JP2009089331A - Camera-attached mobile communication terminal - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a camera-attached mobile communication terminal allowing a user to inexpensively perform panoramic photographing or stereo photographing. <P>SOLUTION: The mobile communication terminal has two casings coupled to each other by a hinge part 108 including a biaxial hinge mechanism, wherein a camera sensor 107 is mounted on a rear-side back face of a casing 102 having a key 103 on its front face and a camera sensor 106 is mounted on a front face of a casing 101 having a main display portion 105. When using the camera-attached mobile communication terminal to perform panoramic photographing or stereo photographing, the casing 101 having display portions 105, 110 is rotated by 180° around a shaft 109 which is orthogonal with a turning shaft around which the mobile communication terminal is folded, through the hinge part 108 having the biaxial hinge mechanism, so that the camera sensor 106 and the camera sensor 107 are set into a positional relationship illustrated in Fig.1(c). Thus, panoramic photographing or stereo photographing can be performed. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a technique for generating a panoramic image or a stereo image by acquiring and processing images simultaneously captured by two cameras in a mobile communication terminal with a camera function having two cameras.

  A technology that enables panoramic photography or stereoscopic photography (stereo photography) by mounting two or more cameras on a mobile communication terminal such as a cellular phone and using images taken simultaneously by these cameras. It is proposed in Documents 1-4.

  In the portable communication device described in Patent Document 1, the first and second casings that are configured to be foldable by the pivot shaft can be pivoted about the first and second pivot shafts at both longitudinal ends. First and second cameras are provided, and the first and second cameras enable image shooting for creating a stereo stereoscopic image or image shooting for creating a panoramic image.

  In the mobile communication terminal described in Patent Document 2, three sensors each capable of shooting a predetermined shooting area (view angle) are mounted on the back and both sides of the mobile phone terminal, and image data shot by these sensors. A technique is described in which images are connected by a data processing circuit, processed to a panorama size, and displayed.

  In the portable information terminal described in Patent Document 3, the first and second casings configured to be foldable by a pivot shaft can be pivoted about the first and second pivot shafts at both longitudinal ends. Even if the opening angle of the housing is changed by providing the first and second cameras and rotating the first and second rotation shafts in conjunction with the rotation shaft for folding the terminal. There is described a technique that enables panoramic shooting or stereo shooting always within a fixed shooting range.

  In the portable device described in Patent Document 4, the first and second imaging optical systems are provided on the same longitudinal surface of the first and second housings that are configured to be foldable by the rotation shaft. , The second photographing optical system enables image photographing for creating a stereo stereoscopic image or image photographing for creating a panoramic image.

JP2003-051872A JP 2004-159014 A Japanese Patent Laid-Open No. 2005-080210 Japanese Patent Laid-Open No. 2005-266023 JP 2007-201727 A JP 2003-174495 A JP 2005-277775 A JP 2003-198676 A

  In the inventions described in the above Patent Documents 1 to 4, in order to perform panoramic photography or stereo photography, such as adding a dedicated camera for performing panoramic photography or stereo photography, or providing a rotation mechanism in the camera built-in unit. In addition, it is necessary to provide the mobile communication terminal with a special mechanism, and accordingly, the structure design becomes complicated and the cost increases.

  On the other hand, as a recent technical trend of mobile phones, cameras are mounted on the front and back of the terminal housing, respectively, and the camera placed mainly on the back of the housing is used for taking pictures of the background etc. In the case of use, there are a large number of cameras (see, for example, FIG. 6 of Patent Document 2 and Patent Document 5) that allow a user to take a picture of himself / herself with a camera arranged in front of the housing.

  In addition, the folding mobile phone has a biaxial hinge mechanism that allows one case to rotate 180 degrees around an axis orthogonal to the folding hinge axis, so that the display screen can be displayed on the front side even in a folded state. There are also products that can be used (see, for example, Patent Documents 6 to 7). In addition, many foldable mobile phones that have improved usability by providing a sub display unit on the back surface of a casing having a main display unit (see, for example, Patent Document 8) are widely used.

  An object of the present invention is to provide a mobile communication terminal with a camera that can perform panoramic photography or stereo photography at low cost by utilizing these technical trends and the like.

  A mobile communication terminal with a camera according to the present invention includes first and second housings that are fastened by a hinge shaft so as to be openable and closable, a first camera that is installed on the front surface of the first housing in an open state, and an open camera. And a second camera installed on the back surface of the second casing in a state, and a panoramic photographing unit that enables panoramic photographing using the first and second cameras. .

  According to another aspect of the present invention, there is provided a camera-equipped mobile communication terminal including first and second housings that are fastened by a hinge shaft so as to be opened and closed, a first camera installed on a front surface of the first housing in an open state, and A second camera installed on the back surface of the second casing in the open state is provided, and a stereo photographing unit that enables stereo photographing using the first and second cameras is provided. To do.

  That is, the present invention includes a configuration having two cameras mounted on the front and the back of a terminal housing, and two configurations configured to be mutually foldable by a hinge shaft, which are provided in many recent mobile communication terminals with cameras. Focusing on a mobile communication terminal equipped with a housing, it is characterized by functionally combining the installation position relationship of two cameras with respect to the two housings and a hinge mechanism, thereby performing panoramic photography or stereo photography A configuration that enables panoramic photography or stereo photography by combining a low-cost module with a camera-equipped mobile communication terminal without adding a dedicated camera for the use, or without using a wide-angle lens or a high-pixel camera module It is characterized by that.

  In the present invention, panoramic photography or stereo photography is enabled by using two cameras provided in the camera-equipped mobile communication terminal, so that panoramic photography or stereo photography by the camera-equipped mobile communication terminal is realized at low cost. be able to.

  Further, since it can be realized without using a special lens, a camera module, or a special mechanism for panoramic photography or stereo photography, it can be easily incorporated into a mobile communication terminal.

  FIG. 1 is a schematic view of a camera-equipped mobile communication terminal according to a first embodiment of the present invention. FIG. 1A is a view of the mobile communication terminal viewed from the front side when the mobile communication terminal is opened. Fig. 1 (c) shows the case where the upper housing with the display unit is rotated 180 degrees around the axis orthogonal to the hinge axis to be in the reverse open state. Each overview is shown.

  The mobile communication terminal with a camera according to the present embodiment includes a housing (102) having a key (103) and a housing (101) having a main display unit (105) on the front surface and a sub display unit (110) on the back surface. Are coupled by a hinge portion (108) having a biaxial hinge mechanism. In the present embodiment, the camera sensor (107) is mounted on the back side (back side) of the casing (102) having the key (103) on the front side, and the front side of the casing (101) having the main display unit (105) is set on the front side. The camera sensor (106) is installed. The camera sensor (106) is mounted on the back side (rear) of the housing (101) having the sub display unit (110), and the camera sensor (107) is a key of the housing (102) having the key (103). It is good also as a structure mounted in the surface (104) side (front surface).

  When performing panoramic shooting or stereo shooting using the camera-equipped mobile communication terminal according to the present embodiment, the housing (101) having the display units (105) and (110) is connected to the hinge unit (108) having the biaxial hinge mechanism. 1), the camera sensor (106) and the camera sensor (107) are shown in FIG. 1 (c) by rotating 180 degrees around an axis (109) orthogonal to the rotation axis for folding the portable communication terminal. Set so that it is in a positional relationship. Thereby, the panorama shooting or the stereo shooting is set to be possible.

  FIG. 2 is a principal block diagram illustrating a configuration example related to the present invention in a mobile communication terminal including two cameras according to the present embodiment.

  The key operation unit (205) is a device for a user to input camera activation, shooting, and storage processing. A camera sensor (201) and a camera sensor (202) correspond to the camera sensor (106) and the camera sensor (107) in FIG. The camera sensor (201) exchanges data with the image processing unit (203) via the data bus (208). The camera sensor (202) exchanges data with the image processing unit (203) via the data bus (209). The display unit (206) is shown as a configuration including the main display unit (105) and the sub display unit (110) of FIG.

  The image processing unit (203) performs processing such as size correction and panorama synthesis of image data captured by the camera sensor (201) and the camera sensor (202). The CPU (204) performs shooting commands and data transfer commands to the camera sensors (201) and (202) and image processing commands and data to the image processing unit (203) by operations input from the key operation unit (205). A transfer command is stored in the memory unit (207).

  In FIG. 2, when a shooting operation is performed by the key operation unit (205), the camera sensor (201) and the camera sensor (202) are simultaneously shot by a command from the CPU (204), and the camera sensor (201) The captured image data is transferred to the image processing unit (203) via the data bus (208), and the camera sensor (202) transfers the captured image data to the image processing unit (203) via the data bus (209).

  The image processing unit (203) performs a panorama synthesis process on the image data transferred from the camera sensor (201) and the camera sensor (202) to create a panorama image. At this time, if the image sizes photographed by the camera sensor (201) and the camera sensor (202) are different, the image processing unit (203) performs a size correction process for aligning the sizes and then performs panorama synthesis.

  The created panoramic image is displayed on the display unit (206) from the image processing unit (203) via the CPU (204). When a save operation is input from the key operation unit (205), the created panoramic image is transferred from the image processing unit (203) to the memory unit (207) via the CPU (204) and stored.

  FIG. 3 is an image diagram showing panoramic photographing and image processing operations according to the present embodiment.

  At the time of panoramic photography, the camera-equipped mobile communication terminal of the present embodiment is set to the state shown in FIG. A camera arranged on the back of the housing (302) having the key (303) by pressing the shooting button while viewing the preview displayed on the main display (305) or the sub display (310) The camera (307) and the camera sensor (306) arranged on the front of the housing (301) having the main display (305) and the sub display (310) Photograph B (312) at the same time.

  The captured image data is sent to the image processing unit (203) as an acquired image A (313) and an acquired image B (314). The image processing unit (203) performs composition processing of the input acquired image upper (313) and the acquired image second (314) to create a panoramic composite image (315).

  Note that the shooting button can be arranged at an appropriate position such as the key (303) of the housing (302) or provided on the side surface of the housing (302). Also, the preview display at the time of shooting is displayed on the sub display (310) when shooting a landscape or a third party, and when shooting a picture of yourself in the panoramic view of the back, the main display (305). Further, when displaying a confirmation image to be stored in the memory, the main display unit (305) is set to display the confirmation image.

  The switching setting of the display unit for preview display can be performed by predetermined key scanning from the key scanning unit (303). In this way, by switching and setting the display unit that performs the preview display according to the panoramic shooting scene, it is possible to press the shooting button while confirming in the preview display, and the main display unit (305) and the sub display unit (310) can be used effectively during panoramic photography.

  FIG. 4 is a flowchart showing an example of panoramic image shooting processing in the present embodiment. The operation of this embodiment will be described mainly with reference to FIGS.

  When the camera-equipped mobile communication terminal of the present embodiment is set to the state of FIG. 1C and the camera panorama mode is designated, the camera panorama mode is started (S401). First, the camera sensor (201) and the camera sensor (202) is activated (S402), and a preview image of the camera sensor (201) and the camera sensor (202) is displayed on the display unit (206) (S403). When the shooting button is pressed (S404, YES), the camera sensor (201) and the camera sensor (202) simultaneously perform shooting and image acquisition (S405). Thereafter, the acquired image is subjected to image size correction / panorama synthesis processing (S406) by the image processing unit (203), and the generated panoramic image is displayed on the display unit (206) (S407).

  When save execution (S408, YES) is input in the key operation unit (205), the panorama image generated by the image processing unit (203) via the CPU (204) is saved in the memory unit (207) ( In step S409, confirmation of camera termination (S410) is performed. When the storage is not executed (S408, NO), the panoramic image data is discarded (S411), and confirmation of the end of the camera (S410) is performed. When the camera end is executed by the key operation unit (205) (S410, YES), the camera panorama mode is ended (S412). When the camera end is not executed (S410, NO), the process returns to the preview video display (S403).

  As shown in FIG. 8 (a), the preview video display of the present embodiment is displayed in the preview image display screen (801) in the video display frame of the shooting area A taken by the camera sensor (301) shown in FIG. (802) and the image display frame (803) of the shooting area B taken by the camera sensor (302) are displayed side by side without being combined. In this way, the preview video display can reduce the load on the image processing unit (203) during the preview by displaying the video that is not subjected to the panorama synthesis process. On the image display screen (804) after shooting, as shown in FIG. 8 (b), the panorama composite image (805) can be displayed to confirm the panorama composite image.

  FIG. 5 is a flowchart showing another example of the panoramic image photographing process in the present embodiment.

  The basic configuration of this embodiment is the same as that of the embodiment shown in FIG. 4, but in this embodiment, images output from the camera immediately after camera activation (S501) until the shutter button is pressed (S505). The panorama image shown in FIG. 8B is displayed from the preview by executing the panorama synthesizing process (S503) for each frame of data.

  When the configuration of the present embodiment is adopted, the load on the image processing unit (203) during preview increases, but before the shutter button is pressed, a panoramic image to be shot can be confirmed, so that a desired panorama image can be confirmed. Captured images can be easily acquired, and there is an effect of reducing unnecessary shooting. The operation of this embodiment is the same as that of the embodiment shown in FIG. 4 except that a panorama composite image is displayed during the preview, and a detailed description of the operation is omitted.

  FIG. 6 is a flowchart showing an example of a stereoscopic image shooting process in the present embodiment.

  The basic configuration of this embodiment is the same as that of the embodiment shown in FIG. 4, but in this embodiment, the image processing unit (203) is an image taken by the camera sensor (201) and the camera sensor (202). By performing stereoscopic (stereo) composition processing (S606) on the data, it is possible to perform stereoscopic image shooting with the mobile communication terminal of the present embodiment.

  In the case of stereo shooting, the image data shot by the camera sensor (201) and the camera sensor (202) are in the same shooting area, so the preview display during stereo shooting is the camera sensor (201) and the camera sensor. Only one of the image data captured in (202) can be displayed on the display unit (206). Since the operation of this embodiment is almost the same as the operation of the embodiment shown in FIG. 4 except for the difference between stereo shooting and panoramic shooting, detailed operation description is omitted.

  FIG. 7 is a schematic view of a camera-equipped mobile communication terminal according to the second embodiment of the present invention, and FIG. 7A is a view of the mobile communication terminal viewed from the front side with the mobile communication terminal open. Fig. 7 (c) shows the positional relationship between the two camera sensors when the portable communication terminal is closed, and Fig. 7 (d) shows the present embodiment. The images at the time of panoramic photography according to the form are shown.

  The camera-equipped mobile communication terminal according to the present embodiment includes a housing (702) having a key (703) and a housing (701) having a main display unit (705) on the front surface and a sub display unit (710) on the back surface. Are foldably coupled by the hinge portion (708). A camera sensor (707) is mounted on the back side (rear) of the housing (702) having a key (703) on the front surface, and a camera sensor (706) is mounted on the front surface of the housing (701) having a main display portion (705). Is installed.

  In this embodiment, the casing (701) is configured to be longer than the casing (702), and the camera sensor (706) installed on the front surface of the casing (701) is connected to the hinge axis of the hinge portion (708). It is necessary to dispose at a position separated by more than the length of the housing (702). Accordingly, when the camera-equipped mobile communication terminal is folded (closed) by the hinge portion (708), as shown in FIG. 7 (c), the camera sensor (706 ) And the camera sensor (707) installed on the back side (rear) of the housing (702) can be photographed in the same direction, and panoramic photography or stereo photography can be performed as shown in FIG. 7 (d). It becomes possible.

  In this embodiment, the sub display unit (710) can be used as a finder for preview display. According to the configuration of the present embodiment, since it is not necessary to use a biaxial hinge as the hinge portion (708), a camera-equipped mobile communication terminal capable of panoramic shooting or stereo shooting can be provided with a simpler configuration and at a low cost. Can do.

  Also in this embodiment, the camera sensor (706) is mounted on the back side (rear) of the housing (701) having the sub display unit (710), and the camera sensor (707) is mounted on the housing having the key (703). It is good also as a structure mounted in the key surface (704) side (front surface) of (702).

  In that case, the chassis (702) is configured to be longer than the chassis (701), and the camera sensor (706) is installed at a position close to the hinge (708) on the back (rear) of the chassis (701). The camera sensor (707) is disposed at a position separated from the hinge shaft of the hinge portion (708) by a length equal to or longer than the length of the housing (701). Therefore, when the mobile communication terminal with camera is folded (closed) by the hinge part (708), the camera sensor (706) installed on the back of the housing (701) and the front of the housing (702) The installed camera sensor (707) is directed substantially in the same direction so that each can be photographed.

  In the connection between the camera sensor and the image processing unit in the above embodiment, the camera sensor (201) transfers the captured image data to the image processing unit (203) via the data bus (208), and the camera sensor (202). Adopts a configuration in which captured image data is transferred to the image processing unit (203) via the data bus (209). As shown in FIG. 9, the image is captured by each of the camera sensors (901) and (902). By shifting the timing and sequentially transferring the captured images, the connection between the camera sensor and the image processing unit can be realized by the shared data bus (908).

  As shown in FIG. 10, each camera sensor has a buffer (1009) and (1010) for holding data, and images taken simultaneously by the two camera sensors (1001) and (1002) are stored in the respective buffers (1009). ) And (1010), and the data bus (1008) can be shared by sequentially transferring the data in the buffers (1009) and (1010) to the image processing unit (1003).

It is a general-view figure of the mobile communication terminal with a camera which shows the 1st Embodiment of this invention. It is a block diagram which shows the principal part structural example in the portable communication terminal with a camera of 1st or 2nd embodiment. It is an image figure which shows the panorama imaging | photography and image processing operation | movement by this embodiment. It is a flowchart which shows the Example of the panoramic image imaging process in this embodiment. It is a flowchart which shows the other Example of the panoramic image imaging process in this embodiment. It is a flowchart which shows the Example of the stereo image imaging | photography process in this embodiment. It is a general-view figure of the mobile communication terminal with a camera which shows the 2nd Embodiment of this invention. It is an image figure which shows the example of a preview video display and a composite video display. It is a block diagram which shows the other principal part structural example in the mobile communication terminal with a camera of 1st or 2nd embodiment. It is a block diagram which shows the further principal part structural example in the mobile communication terminal with a camera of 1st or 2nd embodiment.

Explanation of symbols

101,102,301,302,701,702 Housing
103,303,703 keys
104,704 Key surface
105,305,705 Main display
106,107,201,202,306,307,706,707,901,902,1001,1002 Camera sensor
108 Biaxial hinge
109 Directional axis perpendicular to the folding hinge axis
110,310,710 Sub display
203,903,1003 Image processing unit
204,904,1004 CPU
205,905,1005 Key operation section
206,906,1006 Display section
207,907,1007 Memory part
208,209,908,1008 Data bus

Claims (21)

  First and second housings fastened to be able to be opened and closed by a hinge shaft, a first camera installed on the front surface of the first housing in the open state, and a back surface of the second housing in the open state A camera-equipped mobile communication terminal comprising a second camera installed in the camera and panoramic photographing means that enables panoramic photographing using the first and second cameras.   The panoramic imaging means includes a case reversing mechanism that rotates one of the first and second cases 180 degrees around an axis orthogonal to the hinge axis, and the first and second cases by the case reversing mechanism. A panoramic image for generating a panoramic image from images taken by the first and second cameras in a reverse open state in which any one of the second housings is rotated 180 degrees around an axis orthogonal to the hinge axis The camera-equipped mobile communication terminal according to claim 1, further comprising a generation processing unit.   A main display unit disposed on the front surface of one of the first and second housings, and a sub display unit disposed on the rear surface of the housing, and a preview display during panoramic shooting The camera-equipped mobile communication terminal according to claim 2, further comprising: a preview display unit that enables switching between the main display unit and the sub display unit according to a panoramic shooting scene.   The length of the first housing is configured to be longer than that of the second housing, and the first camera installed on the front surface of the first housing is connected to the second housing from the hinge shaft. The panoramic photographing means is arranged at a position separated from the body by a length longer than the length of the body, and the panoramic photographing means closes the first and second housings with the hinge shaft and is taken from the images photographed by the first and second cameras. The camera-equipped mobile communication terminal according to claim 1, further comprising a panoramic image generation processing unit that generates a panoramic image.   A preview display unit that includes a main display unit disposed on the front surface of the first housing and a sub display unit disposed on the rear surface of the housing, and displays a preview display during panoramic shooting on the sub display unit. The camera-equipped mobile communication terminal according to claim 4.   A key operation unit disposed on the front surface of the first housing, a main display unit disposed on the front surface of the second housing, and a sub-display unit disposed on the back surface of the housing; 5. The camera-equipped mobile communication terminal according to claim 4, further comprising preview display means for displaying a preview display during panorama shooting on the sub display unit.   7. The preview display means at the time of panoramic photographing has a function of displaying the images photographed by the first and second cameras side by side on the display unit. The camera-equipped mobile communication terminal according to any one of the above.   The preview display means at the time of panoramic photographing has a function of displaying a panoramic composite image obtained by synthesizing videos photographed by the first and second cameras on the display unit. The camera-equipped mobile communication terminal according to any one of 3, 5, and 6.   5. The camera according to claim 2, further comprising means for transferring image data photographed by the first and second cameras to the panorama image generation processing unit via a data bus. Mobile communication terminal.   Means for shifting the photographing timing of the first and second cameras, and image data photographed by the first and second cameras are sequentially transferred to the panoramic image generation processing unit via a common data bus. 5. The camera-equipped mobile communication terminal according to claim 2, further comprising means.   A buffer that holds the image data captured by the first and second cameras for each camera, and the image data for each camera held in the buffer is sent to the panoramic image generation processing unit via a common data bus. 5. The camera-equipped mobile communication terminal according to claim 2, further comprising means for sequentially transferring.   First and second housings fastened to be able to be opened and closed by a hinge shaft, a first camera installed on the front surface of the first housing in the open state, and a back surface of the second housing in the open state A camera-equipped mobile communication terminal comprising: a second camera installed in the camera; and a stereo photographing unit that enables stereo photographing using the first and second cameras.   The stereo photographing means includes a case reversing mechanism that rotates either one of the first and second cases around an axis orthogonal to the hinge axis, and the first and second cases by the case reversing mechanism. A stereo image that generates a stereo image from video captured by the first and second cameras in a reverse open state in which any one of the second housings is rotated 180 degrees around an axis orthogonal to the hinge axis The mobile communication terminal with a camera according to claim 12, further comprising a generation processing unit.   A main display unit disposed on the front surface of one of the first and second housings, and a sub display unit disposed on the rear surface of the housing, and a preview display during stereo shooting 14. The camera-equipped mobile communication terminal according to claim 13, further comprising: a preview display unit that enables switching between the main display unit and the sub display unit according to a stereo shooting scene.   The length of the first housing is configured to be longer than that of the second housing, and the first camera installed on the front surface of the first housing is connected to the second housing from the hinge shaft. The stereo photographing means is arranged from a video photographed by the first and second cameras with the first and second housings closed by the hinge shaft. The camera-equipped mobile communication terminal according to claim 12, further comprising a stereo image generation processing unit that generates a stereo image.   A preview display unit that includes a main display unit disposed on the front surface of the first housing and a sub display unit disposed on the back surface of the housing, and displays a preview display during stereo shooting on the sub display unit. The camera-equipped mobile communication terminal according to claim 15.   A key operation unit disposed on the front surface of the first housing, a main display unit disposed on the front surface of the second housing, and a sub-display unit disposed on the back surface of the housing; 16. The camera-equipped mobile communication terminal according to claim 15, further comprising preview display means for displaying a preview display during stereo shooting on the sub display unit.   The preview display means at the time of stereo shooting has a function of displaying any one of the images shot by the first and second cameras on the display unit. The mobile communication terminal with a camera according to claim 14, 16 or 17.   16. The camera according to claim 13, further comprising means for transferring image data photographed by the first and second cameras to the stereo image generation processing unit via a data bus. Mobile communication terminal.   Means for shifting the photographing timings of the first and second cameras, and image data photographed by the first and second cameras are sequentially transferred to the stereo image generation processing unit via a common data bus. The camera-equipped mobile communication terminal according to claim 13 or 15, further comprising: means. A buffer that holds the image data captured by the first and second cameras for each camera, and the image data for each camera held in the buffer is sent to the stereo image generation processing unit via a common data bus. 16. The camera-equipped mobile communication terminal according to claim 13 or 15, further comprising means for sequentially transferring.

Images