JP4722084B2 - Mobile device and communication system - Google Patents

Description

  The present invention relates to a portable device with a function of simultaneously photographing multiple images and a communication system using the same.

  Conventionally, a parallax barrier method, a lenticular lens method, and the like are known as methods for realizing stereoscopic image display without requiring special glasses. These methods are used for right-eye images having binocular parallax and the like. The left-eye image is displayed alternately on the screen, for example, in the form of vertical stripes, and this display image is separated by a parallax barrier, lenticular lens, etc., and guided to the viewer's right eye and left eye for stereoscopic viewing. It is what you want to do.

Meanwhile, in recent years, the improvement of miniaturization technology for communications technology and equipment, and capturing an image in the mobile phone, it becomes possible to transmit the captured image to a desired partner terminal by the mail function (Patent Document 1).
JP 2002-191067 A JP 2001-292440 A JP 2002-091862 A JP 2001-128194 A International Publication No. 97/032437 Pamphlet JP-A-11-355624 JP-A-10-108152 JP 2002-125246 A JP 2001-166259 A

  However, an appropriate system has not been realized in obtaining, using, transmitting, displaying, etc. a plurality of image data for stereoscopic viewing. In view of the above circumstances, an object of the present invention is to provide a portable device with a multiple image simultaneous photographing function that is useful for a system in acquisition, use, transmission, display, and the like of a plurality of image data for stereoscopic viewing.

Multiple images photographed simultaneously function portable device of the present invention, first obtains a stereoscopic image display means for displaying a stereoscopic image, the image data is provided in the stereoscopic image display means is arranged plane with a plurality of image data and 1 of the camera unit, and a second camera means for acquiring et Re images data provided on the back surface of the stereoscopic image display means is disposed surface, wherein the first camera means and said second camera in one possible photographed images for a planar view by selecting mobile devices means, the second camera means, said stereoscopic image display means is provided to be movable in the arranged the plane, the stereoscopic image display means , and displaying the stereoscopic image generated by the image data acquired from the first camera means and said second camera means stereoscopic image display means is moved to arranged the surface.

In the portable device, the second camera means is provided to be rotatable by a hinge.

  These portable devices can be used to construct a system for acquiring, using, transmitting, displaying, etc. a plurality of image data for stereoscopic viewing. It is possible to prevent a disadvantage that a plurality of image data for stereoscopic viewing is transmitted to a model that does not support stereoscopic viewing, and the memory is consumed with unnecessary image data in the model.

In addition, the communication system of the present invention is provided with a first stereoscopic image display means for displaying a stereoscopic image using a plurality of image data and a surface on which the first stereoscopic image display means is arranged to acquire image data. a first camera means and a second camera means for acquiring et Re images data provided on the rear surface of the first stereoscopic image display means is disposed surface, and a first camera means A portable device capable of picking up one of the second camera means and taking a plane view image , wherein the second camera means is movable to a surface on which the first stereoscopic image display means is arranged. It provided are a first transmitting and receiving to the first camera means and said first first image data communication network stereoscopic image display device has acquired from the second camera means has moved to arranged the plane A first portable device comprising communication means; A second stereoscopic image display means for displaying a stereoscopic image using a plurality of image data; a third camera means for obtaining image data provided on a surface on which the second stereoscopic image display means is arranged; a fourth camera means for acquiring et Re images data provided on the back surface of the second three-dimensional image display means arranged, provided with, one of said third camera means and said fourth camera means A portable device capable of selecting and photographing a planar view image , wherein the fourth camera means is provided movably to a surface on which the second stereoscopic image display means is arranged, and the third camera second mobile device comprising a second communication means for transmitting and receiving the second image data acquired from the fourth camera unit means and said second three-dimensional image display device is moved to arranged the plane to a communications network And the first camera means The three-dimensional image generated by the first image data obtained from the preliminary second camera means, characterized in that to be displayed on the second three-dimensional image display means.

  If it is said structure, the system which acquires, uses, transmits, a display, etc. of several image data for stereoscopic vision can be constructed | assembled.

  Further, in the above mobile device, the distance between the stereoscopic camera means for obtaining a plurality of stereoscopic image data by executing simultaneous shooting and the object to be photographed based on the plurality of stereoscopic image data is measured. It is good also as a structure provided with the means to perform, and the means to produce | generate the information based on a measurement distance, and to show to a user. With this configuration, for example, the distance to the front power pole is measured with a plurality of stereoscopic image data obtained by photographing the front of itself, and information based on the distance for those with weak visual acuity Can improve walking safety.

  Further, in the above mobile device, stereoscopic camera means for obtaining a plurality of stereoscopic image data by executing simultaneous photographing, and means for generating three-dimensional data based on the plurality of stereoscopic image data Obtaining detailed position information based on correspondence between the means for performing rough measurement of position information and the three-dimensional data of the local 3D map data obtained by the rough measurement and the plurality of stereoscopic image data. It is good also as a structure provided with the means to show to a user. With this configuration, the detailed position information is based on the correspondence between the three-dimensional data based on a plurality of stereoscopic image data obtained by photographing the front of itself and the local three-dimensional data obtained by rough measurement. Thus, the user can accurately grasp his / her position. The position information may be roughly measured by GPS. Further, it may be configured to transmit three-dimensional data based on the plurality of image data for stereoscopic viewing to a center via a communication network and obtain detailed position information calculated by the center by communication.

  In the above portable device, the stereoscopic camera means may be configured to have two cameras to perform simultaneous shooting and obtain a plurality of image data for stereoscopic viewing. Alternatively, the stereoscopic camera means is provided with a terminal that can be provided with one camera and the other camera detachably, and performs simultaneous shooting with both cameras to obtain a plurality of image data for stereoscopic viewing. It may be configured as follows.

  In addition, the stereoscopic camera means includes one camera and also includes means for remotely operating another camera device and means for receiving captured image data, and performs simultaneous shooting with the camera and camera device for stereoscopic viewing. A plurality of image data may be obtained. In such a configuration, two-image display means is provided, and an image captured by itself is displayed on one image display side, and an image received from another camera device is displayed on the other image display side. Also good. In addition, a stereoscopic image display means for performing stereoscopic viewing with a plurality of image data for stereoscopic viewing is provided, and this stereoscopic image display means is used for confirming an image being captured by itself and an image being received from another camera device. You may be comprised so that a stereoscopic display can be performed.

  Further, in the configuration including two cameras, at least one of the two cameras may be moved, and the interval between the two cameras may be variable. Alternatively, one of the two cameras may be provided on the front side of the device and the other may be provided on the back side of the device, and one of the cameras may be configured to rotate by a hinge and face the front side or the back side. Moreover, you may be comprised so that the rotation angle of a camera can be set.

  In addition, these portable devices with a multiple image simultaneous photographing function may include a stereoscopic image display unit that performs stereoscopic viewing based on a plurality of stereoscopic image data.

  As described above, according to the present invention, there is an effect that it is useful for a system in acquisition, use, transmission, display, and the like of a plurality of image data for stereoscopic viewing.

  Hereinafter, a portable device with a multiple image simultaneous photographing function according to an embodiment of the present invention will be described with reference to FIGS.

  FIG. 1 illustrates a mobile phone 1 with a camera as a mobile device with a function for simultaneously shooting multiple images. The first camera 3 is provided at the upper left position of the surface on which the display screen 2 of the camera-equipped mobile phone 1 is provided, and the second camera is provided at the upper left position of the back surface (upper right position when viewed from the surface). 4 is provided. The second camera 4 is rotatably provided by a hinge 5. By turning the second camera 4 180 degrees by the hinge 5 to be in the open state, the second camera 4 faces the front side, the first camera 3 and the second camera 4 are side by side, and the same direction is set as the imaging range. To do. In the example shown in FIG. 1, the second camera 4 is rotatable by the hinge 5. However, by making the first camera 3 rotatable, both the first camera 3 and the second camera 4 can be rotated. The imaging direction can be the back side. About a rotation form, it is not restricted to the example mentioned above. Further, in normal single image shooting for planar view, one of the first camera 3 and the second camera 4 can be arbitrarily selected by a user operation.

  When the second camera 4 is turned, a switch (not shown) is turned ON, and this ON information is given to the system controller of the mobile phone. In this state, the system controller determines that the mode is a stereoscopic shooting mode, and when the camera shutter is operated by the user, both the first camera 3 and the second camera 4 perform an imaging process, and the obtained two image data are Store in memory. When the memory is stored, information for right eye / left eye is added to the two images.

  The camera-equipped mobile phone 1 has a mail transmission function, and the image data can be read from the memory and transmitted to a desired destination terminal using the mail transmission function. A system for transmitting two image data obtained in the stereoscopic shooting mode will be described in detail later.

  In addition, the imaging range of the 1st camera 3 and the 2nd camera 4 will become different by rotating the 2nd camera 4 only the angle below 180 degrees with the hinge 5, and making it an open state. That is, as a whole, the imaging range becomes wider in the horizontal direction. By determining the overlapping area between the imaging range of the first camera 3 and the imaging range of the second camera 4 by corresponding point matching and performing a process of connecting the two captured image data, a so-called panoramic image that is horizontally long can be obtained. Become.

  In addition, the camera-equipped mobile phone 1 has a function of measuring the distance to the captured object using two image data obtained in the stereoscopic shooting mode. The point where the line connecting the imaging object and the center of the camera lens intersects the camera CCD corresponds to the camera CCD in the two cameras, and the distance on the CCD changes as the distance from the camera to the imaging object changes. The position of the point changes. The positions of the points on the two camera CCDs are determined by corresponding point matching processing, and the distance to the imaging object can be measured based on the relationship with the distance between the CCDs. The camera-equipped mobile phone 1 displays the measured distance on the display screen 2 or outputs a voice by voice synthesis. In addition to notifying the user of the numerical value of the distance, for example, when detecting that the distance to the object is about 1 m, the voice output or warning sound such as “Please be careful. There is an obstacle ahead.” You may make it perform. Thereby, the walking safety of a low vision person improves. In addition, a voice can be transmitted to a low hearing person using a bone conduction earphone.

  The camera-equipped mobile phone 1 is equipped with a GPS (Global Positioning System) so that it can perform a rough measurement of the current position. Conventionally, a mobile phone is equipped with a GPS, and furthermore, the location of the mobile phone is determined at the center based on information between base stations obtained by location registration processing of the mobile phone, and map information corresponding to the location is carried by the mobile phone. A system to give to the telephone is considered. However, the accuracy of the position information measured by GPS is about 3 m at most, and the position information is rough. For example, when the location determination mode is selected by the user and a shutter operation is performed, the camera-equipped mobile phone 1 transmits two image data and GPS position information obtained by the two cameras 3 and 4 to the center. In the center, 3D map data (including building shape information and the like) is acquired from the 3D map database based on the GPS position information (further, position registration information). Further, the center generates 3D video information based on the two image data sent from the camera-equipped mobile phone 1, and compares the 3D video information with the 3D map data to determine the imaging position. It is possible to accurately determine and give this accurate position information to the camera-equipped mobile phone 1. In the camera-equipped mobile phone 1, for example, a mark indicating the current position can be accurately given on the planar map displayed on the display screen 2 based on the accurate position information.

  In addition, the camera-equipped mobile phone 1 is equipped with a small and large-capacity memory so that it itself has 3D map data, and a function for generating 3D video information based on two image data, and 3D video information By mounting the function of comparing the three-dimensional map data, accurate position information can be generated in the camera-equipped mobile phone 1 without depending on the center.

  FIG. 2 shows a flowchart of accurate position information acquisition processing based on 3D video information and 3D map data (steps S1 to S5). By obtaining such accurate position information, for example, voice information such as “I'm approaching the intersection of XX. Let's be careful” is given to low-sighted people. By providing situation information such as under construction or under construction in various locations and transmitting the information to the mobile phone 1 with a camera, for example, voice information such as “I am approaching the construction site. .

  FIG. 3 shows a camera-equipped mobile phone 11 having one camera 10. The camera-equipped mobile phone 11 includes a terminal (for example, a USB terminal) 13 that can be detachably provided with another camera 12. When the camera 12 is attached to the terminal 13, the camera 10 and the camera 12 are set side by side and the same direction is set as the imaging range. In the example illustrated in FIG. 3, the imaging direction of the camera 10 and the camera 12 is front-facing, but the imaging direction may be back-facing. When the camera 12 is attached to the terminal 13, the system controller detects the attachment of the camera 12. In this state, the system controller determines that the mode is a stereoscopic shooting mode, and when the user operates the camera shutter, causes both the camera 10 and the camera 12 to perform imaging processing and stores the obtained two image data in the memory. It is like that. When the memory is stored, information for the right eye is appended to the image of the camera 10, and information for the left eye is appended to the image of the camera 12.

  FIG. 4 shows a camera-equipped mobile phone 21 having one camera 20 and another camera device 22. The camera-equipped mobile phone 21 includes a function of remotely operating another camera device 22, a function of receiving photographed image data from the other camera device 22, and a display-dividable screen 24 that displays two images. In remote operation and image data reception, for example, in addition to wired communication using a USB interface, wireless (radio wave, infrared) can be used. In the stereoscopic imaging mode, the camera-equipped mobile phone 21 displays the image being captured by the camera 20 in the divided area 24a of the display screen 24, and the image being captured by the camera 22a of the other camera device 22 in the divided area 24b. Is received and displayed. The user can adjust the orientation of the camera by looking at the display image on the display screen 24. When the shutter of the camera-equipped mobile phone 21 is pressed after the adjustment, the camera 20 executes a photographing process and simultaneously issues a remote shutter command to the other camera devices 22. Then, the captured image and the captured image that has been captured and transmitted by another camera device 22 are stored in the memory. When the memory is stored, information for the left eye is appended to the image of the camera 20, and information for the right eye is appended to the image of the camera 22a. With such a configuration, the interval between the two cameras can be freely changed.

  In the camera-equipped mobile phone 21 of FIG. 4 described above, one display screen is divided, but two display screens may be provided. Moreover, it is good also as providing the stereoscopic image display function which provides one display screen and performs stereoscopic vision with two image data. As the display means in this case, for example, a liquid crystal display panel is used. Two images (right-eye video and left-eye video) are alternately displayed on the screen in the form of vertical stripes, for example, on this liquid crystal display panel. If a normal flat display is used, only one image may be displayed. A partial retardation plate (for example, a so-called micropole can be used) is pasted on the liquid crystal display panel. This partial phase difference plate is composed of alternating vertical phase stripe portions and non-phase difference portions that are emitted by changing the vibration direction of incident light by 90 °. Corresponding to the image for the right eye and the image for the left eye displayed in the stripe shape, a light shielding part or a light transmitting part necessary for stereoscopic viewing is formed at a position where the light shielding part or the light transmitting part is to be formed. On the partial retardation plate, a polarizing plate can be provided so as to be freely arranged / not arranged.

  By providing a stereoscopic image display function, it is possible to confirm stereoscopic vision at the stage of shooting and to easily adjust the shooting direction and the like of the two cameras. The camera device 22 may include two display screens or a stereoscopic image display function. In addition, the camera-equipped mobile phone shown in FIGS. 1 and 3 described above may have a stereoscopic image display function. Of course, the camera-equipped mobile phone shown in FIGS. 3 and 4 may be provided with a position measurement function and a high-accuracy position information generation function as well as a mail function.

  In addition, in a configuration having a stereoscopic image display function, a head tracking mechanism is preferably provided. As this head tracking mechanism, for example, the mechanism shown in the glassesless stereoscopic video display device (Japanese Patent Laid-Open No. 2001-166259) shown in the previous application of the applicant of the present application can be used.

  Next, a system for transmitting two image data will be described. Note that the two images may be two images including a normal planar image and an image obtained by depth map (distance information generation) processing, in addition to those captured in the above-described stereoscopic shooting mode.

  In the system shown in FIG. 5A, the transmission side terminal 31 sends a plurality of image data for stereoscopic viewing to the communication network. The relay station 32 of the communication network determines whether or not the counterpart terminal (33, 34) is a stereoscopic compatible model. In this determination, it is sufficient to provide a database in which terminal model information (stereoscopic correspondence / non-correspondence) is associated with its mail address. Alternatively, communication with the counterpart terminal (33, 34) may be established, and the corresponding / non-corresponding information may be obtained from the counterpart terminal. The relay station 32 of the communication network transmits a plurality of image data for stereoscopic viewing to the stereoscopic viewing compatible model 33. On the other hand, one of the plurality of image data is transmitted to the non-compatible model 34. As a method for selecting one image, for example, a method for selecting the right eye among information for right eye / left eye added to two transmission images, or an image sent earlier. The method of selecting can be considered.

  In the system shown in FIG. 5B, the transmission side terminal 41 informs the relay station 42 of the communication network that a plurality of image data for stereoscopic viewing is transmitted to the counterpart terminal (43, 44). The relay station 42 of the communication network determines whether the counterpart terminal (43, 44) is a model compatible with stereoscopic vision and transmits the result to the transmission side terminal 41. The transmission-side terminal 41 transmits a plurality of image data for stereoscopic viewing when the destination terminal is the model 43 for stereoscopic viewing. On the other hand, one image data of the plurality of image data is transmitted to the non-compatible model 44. As a method for selecting one image, for example, a method for selecting the right eye among information for right eye / left eye added to two images can be considered.

  With such a system, it is possible to prevent such a problem that a plurality of image data for stereoscopic viewing is transmitted to a model that does not support stereoscopic viewing (34, 44), and memory is consumed with unnecessary image data in the model. it can. For the transmission side, transmission is possible without worrying about the model of the other party.

  A camera-equipped mobile device having a distance measurement function may be attached to the vehicle so that a warning is issued to the driver when a front obstacle is detected. Further, when the front obstacle is an oncoming vehicle and communication can be performed with a device provided in the oncoming vehicle, warning information may be issued to the device. Further, in the above-described example, the two-screen display, the one having two display screens, or the one having two cameras is shown, but more may be provided. If you have three or more cameras, you can perform multi-view stereoscopic display. It is also possible to simultaneously photograph images with different zooms in a completely different direction or the same direction with a plurality of cameras.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a perspective view of a camera-equipped mobile phone (a portable device with a function for simultaneously photographing multiple images) according to an embodiment of the present invention, in which FIG. The open state is shown. It is the flowchart which showed the processing content of high-accuracy position information generation. It is the perspective view which showed the other example of the mobile phone with a camera of embodiment of this invention. It is the perspective view which showed the other example of the mobile phone with a camera of embodiment of this invention. FIGS. 1A and 1B are explanatory views showing a multiple-image communication system according to the embodiment of the present invention.

Explanation of symbols

1, 11, 21 camera-equipped mobile phone 2 Display screen 3 first camera 4 second camera 5 hinge 10,12,20,22a camera 13 terminal 22 camera device 24 display dividable screen (divided regions: 24a, 24b)
31, 41 Transmission side terminal 32, 42 Relay station 33, 34, 43, 44 Destination terminal (stereoscopic model: 33, 43 non-stereoscopic model: 34, 44)

Claims (3)

A first camera means for obtaining a first three-dimensional image display means for displaying a stereoscopic image, the image data provided in the first three-dimensional image display means is arranged plane with a plurality of image data, wherein a second camera means for acquiring provided et Re images data on the back of the surface where the first three-dimensional image display means arranged, provided with, one of the first camera means and the second camera means A portable device capable of selecting and photographing a planar view image , wherein the second camera means is provided movably to a surface on which the first stereoscopic image display means is arranged, and the first camera first portable device comprising a first communication means for transmitting and receiving the means and the first of the first image data communication network stereoscopic image display device has acquired from the second camera means has moved to a plane arranged When,
A second stereoscopic image display means for displaying a stereoscopic image using a plurality of image data; a third camera means for obtaining image data provided on a surface on which the second stereoscopic image display means is arranged; a fourth camera means for acquiring et Re images data provided on the back surface of the second three-dimensional image display means arranged, provided with, one of said third camera means and said fourth camera means A portable device capable of selecting and photographing a planar view image , wherein the fourth camera means is provided movably to a surface on which the second stereoscopic image display means is arranged, and the third camera second mobile device comprising a second communication means for transmitting and receiving the second image data acquired from the fourth camera unit means and said second three-dimensional image display device is moved to arranged the plane to a communications network And comprising
A communication system, characterized in that a stereoscopic image generated by the first image data acquired from the first camera means and the second camera means is displayed on the second stereoscopic image display means. A stereoscopic image display means for displaying a stereoscopic image using a plurality of image data, a first camera means for obtaining image data provided on a surface on which the stereoscopic image display means is arranged, and the stereoscopic image display means are arranged. the a second camera means, comprising a shooting image for planar view by selecting one of the first camera means and the second camera means for acquiring et Re images data provided on the back surface of the the surface In possible portable devices,
The second camera means is provided movably to the surface on which the stereoscopic image display means is arranged,
It said stereoscopic image display means is characterized by displaying a stereoscopic image generated by the image data acquired from the first camera means and said second camera means stereoscopic image display means is moved to arranged the plane A portable device. The portable device according to claim 2, wherein
The portable device characterized in that the second camera means is rotatably provided by a hinge.

Images