JP3959393B2 - Mobile communication terminal - Google Patents

Description

  The present invention relates to a mobile communication terminal including an imaging unit and an image data storage unit that stores image data obtained by imaging with the imaging unit.

  In recent years, mobile communication terminals such as mobile phones equipped with imaging means such as small digital cameras are widely known, and attempts have been made to add various functions using such imaging means to such mobile communication terminals. Has been made. For example, Patent Document 1 discloses a mobile phone device that displays image information captured by a digital camera provided in the apparatus main body on an image display unit together with character information input by a user. According to this mobile phone device, for example, the telephone number and name information of the person who is the other party of call and the image information such as the face image can be simultaneously displayed on the image display unit. Thus, by selecting a call partner from image information such as a face image, it is possible to search for character information such as the telephone number and name of the call partner. Further, in Patent Document 2, when a hinge unit that rotatably couples a first housing that houses an imaging unit and a second housing that is different from the first housing rotates, the imaging unit There has been disclosed a mobile phone device that changes the setting to the setting for close-up shooting. According to this mobile phone device, the setting of the imaging unit can be changed to the setting of the proximity shooting without providing an independent switch for changing the setting of the imaging unit to the setting of the proximity shooting.

  On the other hand, Patent Document 3 discloses a mobile phone provided with a sensor (detection means) for detecting acceleration. Patent Document 3 describes an embodiment in which a user inputs characters by moving a mobile phone so as to write characters in the air while pressing a character input button. Specifically, the movement trajectory of the mobile phone is obtained from the movement distance data obtained by integrating the acceleration data twice to calculate the movement distance of the mobile phone and the movement direction data obtained from the acceleration data. Operation control of recognizing the movement locus as an input character is performed. Patent Document 4 discloses a mobile phone provided with a geomagnetic sensor (detection means) for detecting the direction. In this mobile phone, numerical values are associated with a plurality of directions, and numerical values can be input by directing the mobile phone body in a specific direction.

JP-A-11-205761 JP 2003-188953 A JP 2002-169645 A JP 2003-111142 A

  Recently, mobile communication terminals such as mobile phones equipped with image capturing means have rapidly spread, and it is important to further enhance functions and services using the image capturing means in order to satisfy the demands of users. ing. For this reason, developers of mobile communication terminals and developers of application programs that run on mobile communication terminals are conducting research every day to enhance their functions and services. The provision of functions and services is still not enough.

  The present invention has been made in view of the above background, and an object of the present invention is to provide a mobile communication terminal capable of providing an unprecedented new function or service using an imaging unit. It is.

In order to achieve the above object, the invention of claim 1 is directed to a mobile communication terminal comprising an imaging means and an image data storage means for storing image data obtained by imaging with the imaging means. Detection means for performing detection for specifying the imaging direction, imaging direction data storage means for storing imaging direction data indicating an imaging direction specified by detection of the detection means performed at the time of imaging by the imaging means, The image data obtained by the imaging is stored in the image data storage unit, the imaging direction data at the time of imaging is stored in the imaging direction data storage unit, and the image storage unit captures the image data. Based on the imaging direction data at the time of imaging, the imaging direction changed by one image is calculated from the image data captured at the time of imaging, and an image showing the calculated imaging direction is detected by the detection means. Together with the image showing the current imaging direction specified Ri, it is characterized in that it has a display control means for performing control to display on the display means.
In this mobile communication terminal, image data obtained by imaging by the imaging unit and imaging direction data indicating the imaging direction specified by detection of the detection unit performed at the time of imaging are stored. Therefore, data processing such as processing image data using the imaging direction at the time of imaging becomes possible. The imaging direction here may be a two-dimensional direction or a three-dimensional direction. In the case of a horizontal (two-dimensional) direction, the imaging direction can be grasped as an orientation in which the imaging direction is directed, for example. Moreover, when it is set as the two-dimensional direction which consists of a vertical surface, the imaging direction can be grasped | ascertained as an inclination angle with respect to a horizontal surface, for example. Moreover, you may grasp | ascertain an imaging direction as a relative direction with respect to reference directions, such as the imaging direction at the time of last imaging, for example. In this case, as the imaging direction data indicating the imaging direction, data indicating the inclination angle of the imaging direction with respect to the reference direction can be used.
Here, when a single image is obtained by combining a plurality of images, it is desirable that the image continuity at the combined portion of the images is as high as possible. According to this mobile communication terminal, it is possible to cause the user to perform imaging in an appropriate imaging direction in which the images are properly continuous with the previous captured image. Therefore, even when a user combines images captured while changing the imaging direction, high image continuity can be ensured.

The invention of claim 2 is the mobile communication terminal according to claim 1, said image data a plurality of image data stored in the storage means, the imaging direction data associated with each of the image data of the plurality of And data processing means for performing data processing to be combined into one image data.
In this mobile communication terminal, it is possible to obtain a large image that cannot be captured by one imaging by combining a plurality of image data by the data processing means. For example, when imaging a 360 ° landscape from the top of a mountain as a single image, first, a plurality of image data are obtained by imaging a plurality of times while sequentially changing the imaging direction from a fixed point. Since an appropriate arrangement order of the image data can be specified from the respective imaging direction data, the image data can be appropriately arranged and synthesized regardless of the imaging order. Therefore, a 360 ° landscape image can be obtained as one image data .

As the "mobile communication terminal" is, PDC (Personal Digital Cellular) _{system, GSM (Global S Y stem for} Mobile Communication) system, TIA (Telecommunications Industr _Y Association) system such as a mobile phone, IMT (International Mobile Telecommunications )-Phones such as mobile phones, PHS (Personal Hand _Y phone Service), and automobile phones that are standardized in -2000 are provided with imaging means. Examples of the “mobile communication terminal” include a mobile mobile communication terminal such as a PDA (Personal Digital Assistance) that does not have a telephone function in addition to the above-described telephone.

As described above, according to the invention of claim 1 and 2 and the data processing such as processing the image data becomes possible using imaging Direction at the time of imaging of the imaging unit, a new unprecedented utilizing imaging means functions Or the outstanding effect that it becomes possible to provide a service is show | played.
In particular, according to the invention of claim 2 , there is an excellent effect that a large image that cannot be captured by one imaging can be obtained as one image data .

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
FIG. 2 is an explanatory diagram for explaining the overall configuration of a mobile communication system in which a mobile phone as a mobile communication terminal according to the present embodiment can be used.
In this mobile communication system, the mobile phone 20 used by the user 1 has a configuration capable of executing an application program registered by the user 1. In the present embodiment, the application program is developed by object-oriented programming independent of the platform. Examples of such an application program include an application program written in JAVA (registered trademark), an application program operating in an application execution environment of BREW (registered trademark), and the like. The mobile phone 20 can be connected to a mobile phone communication network 10 as a communication network. Further, an application program download server (hereinafter referred to as “download server”) 11 as a program providing server is connected to the mobile phone communication network 10. When the download server 11 receives a download request from the mobile phone 20, the download server 11 transmits an application program related to the request to the mobile phone 20.

  The application program provided from the download server 11 is provided from the application program developer 2. Specifically, for example, it is uploaded from the personal computer on the application program developer 2 side to the download server 11 via a dedicated line or a public line. Note that a recording medium such as an optical disk or a magnetic disk on which the developed application program is recorded is sent from the application program developer 2 to a communication carrier that manages and operates the download server 11, and the application program in the recording medium is downloaded to the download server 11. You may read and provide. The application program thus provided is registered in the download server 11 in a state where it can be downloaded from the mobile phone 20 via the mobile phone communication network 10.

FIG. 3 is a schematic configuration diagram showing a hardware configuration of the download server 11.
The download server 11 includes a system bus 100, a CPU 101, an internal storage device, an external storage device 104, an input device 105, and an output device 106. The internal storage device includes a RAM 102, a ROM 103, and the like. The external storage device includes a hard disk drive (HDD), an optical disk drive, and the like. The input device 105 includes an external storage device 104, a mouse, a keyboard, and the like. The output device 106 includes a display, a printer, and the like. The download server 11 further includes a mobile phone communication device 107 for communicating with the mobile phone 20 of each user 1 via the mobile phone communication network 10.
The components such as the CPU 101 and the RAM 102 exchange data and program instructions with each other via the system bus 100. A program for operating the download server 11 according to a predetermined procedure is stored in the ROM 103 or the external storage device 104, and is called up and executed in the work area on the CPU 101 or the RAM 102 as necessary. In the download server 11, an application program to be provided to the mobile phone 20 is stored in the external storage device 104. In response to a download request from the mobile phone 20, the download server 11 cooperates with the CPU 101, the RAM 102, the mobile phone communication network communication device 107, and the like to transfer the application program stored in the external storage device 104 to the mobile phone communication It has a function of transmitting to the mobile phone 20 via the network 10. The download server 11 may be configured as a dedicated control device or may be configured using a general-purpose computer system. Further, it may be configured by a single computer, or may be configured by connecting a plurality of computers each having a plurality of functions via a network.

4 is a front view showing the appearance of the mobile phone 20, and FIG. 5 is a schematic configuration diagram showing the hardware configuration of the mobile phone 20. As shown in FIG.
The mobile phone 20 is a clamshell (folding) type mobile phone, and includes an internal control device including a system bus 200, a CPU 201, a RAM 202, a ROM 203, an input device 204, an output device 205, a mobile phone communication device 206, an acceleration. A sensor 207, a geomagnetic sensor 208, and a camera 209 are provided. Components such as the CPU 201 and the RAM 202 exchange various data and instructions of a program to be described later via the system bus 200. The input device 204 includes a data input key (ten key, * key, # key) 21, a call start key 22, an end key 23, a scroll key 24, a multi-function key 25, a microphone 26, and the like. The output device 205 includes a liquid crystal display (LCD) 27 as a display means, a speaker 28, and the like. The mobile phone communication device 206 is for communicating with another mobile phone or the download server 11 via the mobile phone communication network 10. Further, in the RAM 202, there are a platform storage area as a first storage means managed by a telephone platform described later and an application storage area as a second storage means managed in an application execution environment described later. Exists.

The acceleration sensor 207 detects accelerations α _X and α _Y directed in two directions (X-axis direction and Y-axis direction in FIG. 4) orthogonal to each other in a plane parallel to the plane on which the data input key is provided. This is a two-axis acceleration sensor. The acceleration sensor 207 is mounted on a circuit board (not shown) provided inside the mobile phone 20, and a known sensor that can detect the accelerations α _X and α _Y can be used.
The geomagnetic sensor 208 is a three-axis sensor that detects a magnetic field strength component (magnetic flux density component) of the geomagnetism on a three-dimensional coordinate composed of the X axis, the Y axis, and the Z axis orthogonal to these axes. . In the present embodiment, the angles θ _X , θ _Y , and θ _Z around the X axis, the Y axis, and the Z axis are detected using the detection result of the geomagnetic sensor 208. Specifically, the amount of change when the direction of the geomagnetism changes with respect to the reference geomagnetic direction (reference direction) is expressed as the angles θ _X , θ _Y , θ around the _X , _Y , and Z axes. Detect using _Z. Thereby, when the mobile phone changes its attitude from the attitude when the direction of geomagnetism is in the reference direction, the attitude after the change can be specified by each angle θ _X , θ _Y , θ _Z. In the following description, the angle θ _X around the X axis is referred to as the pitch angle, the angle θ _Y around the Y axis is referred to as the roll angle, and the angle θ _Z around the Z axis is referred to as the yaw angle.
Further, by using the geomagnetic sensor 208, for example, it can be detected in which direction the Z-axis is directed with respect to the north direction. In this case, for example, the direction in which the mobile phone is facing is specified by the angle (hereinafter referred to as “azimuth angle”) θ _{N formed} by the Z axis and the north direction. The geomagnetic sensor 208 is also mounted on a circuit board (not shown) provided inside the mobile phone 20.
Note that these sensors 207 and 208 may be configured as separate devices from the main body of the mobile phone 20. In this case, for example, an external device including these sensors 207 and 208 is connected to an external terminal provided on the main body of the mobile phone 20 so that the external device and the main body of the mobile phone 20 are integrated. .

  The camera 209 includes a built-in camera 29 whose lens surface is exposed on the back side of the mobile phone 20, as shown on the left side in FIG. As the built-in camera 29, a digital small camera such as a complementary metal oxide semiconductor (CMOS) camera or a charge coupled device (CCD) camera can be used. In the present embodiment, the built-in camera 29 is fixed to the main body of the mobile phone 20 so that the imaging direction is in the Z-axis direction.

FIG. 6 is a block diagram showing an extracted main part of the mobile phone 20, and FIG. 7 is an explanatory diagram of the software structure in the mobile phone 20.
The cellular phone 20 includes a telephone communication unit 211 and a data communication unit 212 as wireless communication units, an operation unit 213 as operation units, an application program execution management unit 214 as application program execution units, a main control unit 215, and an output unit 216. A sensor detection unit 217 as a detection unit, a camera unit 219 as an imaging unit, and the like.

The telephone communication unit 211 performs wireless communication with a base station of the mobile phone communication network 10 in order to perform telephone communication with other mobile phones or fixed phones. This corresponds to the device 206 and the like.
Similar to the telephone communication unit 211, the data communication unit 212 corresponds to the mobile phone communication device 206 having the above-described hardware configuration. The data communication unit 212 exchanges mail with other mobile phones via the mobile phone communication network 10 or connects to an external communication network such as the Internet from the mobile phone communication network 10 via a gateway server. This is for exchanging e-mails on the Internet, browsing web pages, and the like. The data communication unit 212 is also used to download an application program provided by the download server 11 via the mobile phone communication network 10.

  The operation unit 213 includes the above-described numeric keypad 21, call start key 22, and call end key 23 that can be operated by the user 1. By operating the operation unit 213, the user inputs data such as a URL to the mobile phone 20, starts and ends a call when receiving a call, and selects, starts, and stops an application program. Can be done. The user can also download the application program from the download server 11 by operating the operation unit 213.

  The application program execution management unit 214 includes the system bus 200, the CPU 201, a part of the RAM 202, and the like. The application program execution management unit 214 corresponds to the central “application execution environment” in the software structure of FIG. 7, and includes a class library, an execution environment management library, and an application program developed by object-oriented programming. Software such as application management is provided to manage the execution environment of application programs. This application execution environment is appropriately selected according to the application program to be executed. For example, if the application program to be executed is written in JAVA, the JAVA application execution environment is selected. If the application program to be executed is written in C language that operates in the BREW execution environment, the BREW application execution environment is selected. If the application program to be executed is written in JAVA, it can be executed by constructing a JAVA application execution environment on the BREW application execution environment.

  Here, the application program can be used by calling a class library such as a function in the application execution environment via a class library API (application interface). The call history of the class library such as this function is stored in the application storage area in the RAM 202 until the virtual execution environment (virtual machine: VM) of the application program is terminated. The application execution environment also saves various data used when executing the application program in the application storage area. When these various data are used, they are read from or written to the application storage area. The execution environment management library in the application execution environment can be used by calling a telephone platform library in the telephone platform described later via the telephone platform API.

  The main control unit 215 controls the telephone communication unit 211, the data communication unit 212, the operation unit 213, the sensor detection unit 217, and the camera unit 219, and includes the above-described system bus 200, CPU 201, RAM 202, and the like. ing. The main control unit 215 exchanges control commands and various data with the application program execution management unit 214 and performs control in cooperation with them. The main control unit 215 corresponds to the lowermost “telephone platform” in the software structure of FIG. 7 and executes a control program and user interface for controlling the telephone communication unit 211 and the like, Or provide a library. The telephone platform executes various processes in an application program by sending an event to an execution environment management library in the application execution environment, or performs application management software in the application execution environment via an application management API. Can be called and used. When the application execution environment calls and uses the telephone platform library via the telephone platform API, the telephone platform executes processing according to the telephone platform library. For example, the telephone platform reads data stored in the platform storage area managed by the telephone platform in the RAM 202 based on an instruction from the application execution environment using the telephone platform library, and migrates it to the application storage area. can do.

  The output unit 216 includes the output device 205 including the liquid crystal display 27 and the speaker 28 described above. The output unit 216 displays the Web page screen received by the data communication unit 212 on the liquid crystal display 27. The liquid crystal display 27 of the output unit 216 is used when the telephone communication unit 211 or the data communication unit 212 notifies the user that information has been received. Specifically, when the information is received, the main control unit 215 displays an incoming call notification image on the liquid crystal display 27 of the output unit 216 or outputs a ring tone from the speaker 28. Further, the output unit 216 is also used for displaying a menu screen or the like related to the execution of the program or outputting music during execution of the application program executed in the application execution environment. For example, when shooting is performed by executing a camera application program that uses the camera unit 219, an image to be captured that is captured by the camera unit 219 is displayed, or a notification sound for notifying a release timing described later is output. To do.

The sensor detection unit 217 includes the acceleration sensor 207 and the geomagnetic sensor 208 described above. The sensor detection unit 217 operates under the control of the main control unit 215, and the detection data is acquired by the main control unit 215. As described above, the acceleration α _X and α _Y data, the pitch angle θ _X , the roll angle θ _Y and the yaw angle θ _Z data, and the azimuth angle θ _N data, which are detection data, are stored in the RAM 202 platform storage area. Is remembered. For example, when the mobile phone 20 is displaced by the user 1, acceleration acting in the X-axis direction and the Y-axis direction is detected by the acceleration sensor 207 that constitutes the sensor detection unit 217. When the detection data is input to the main control unit 215, the main control unit 215 can grasp the acceleration α _{X in the X-} axis direction and the acceleration α _Y in the Y-axis direction from the detection data. The acceleration α _X and α _Y data are temporarily stored in the platform storage area in the RAM 202 by the main control unit 215.
Further, when the posture of the mobile phone 20 changes, the magnetic field strength component (magnetic flux density component) after the change of the posture is detected by the geomagnetic sensor 207 constituting the sensor detection unit 217. The sensor detection unit 217 calculates the respective angles θ _x , θ _Y , and θ _Z after the posture change from the detection signal detected by the geomagnetic sensor 207. The data of the calculated angles θ _x , θ _Y , θ _Z is output to the main control unit 215 as in the case of the accelerations α _x , α _Y , and is temporarily stored in the platform storage area in the RAM 202 by the main control unit 215. Saved.
In addition, when the orientation of the mobile phone 20 changes, the magnetic field strength component (magnetic flux density component) after the change of the orientation is detected by the geomagnetic sensor 207 constituting the sensor detection unit 217. The sensor detection unit 217 calculates the azimuth angle θ _N after the change in direction from the detection signal detected by the geomagnetic sensor 207. Similarly, the data of the calculated azimuth angle θ _N is also output to the main control unit 215 and temporarily stored in the platform storage area in the RAM 202 by the main control unit 215.

The following is a method for the main control unit 215 to acquire the data of the accelerations α _x and α _Y and the angles θ _x , θ _Y , and θ _Z stored in the platform storage area from the sensor detection unit 217. Is mentioned. For example, this is an acquisition method in which a request is sent from the main control unit 215 to the sensor detection unit 217, and the data output by the sensor detection unit 217 is received by the main control unit 215 accordingly. In addition, for example, an acquisition method may be employed in which the main control unit 215 appropriately receives data continuously output from the sensor detection unit 217 even when there is no request. The main control unit 215 sends a request to the sensor detection unit 217 in response to a request output from the application program via the application program execution management unit 214, and the data output by the sensor detection unit 217 in response to the request is sent to the main control unit. The acquisition method received by 215 can also be adopted.

  The camera unit 219 includes the above-described camera 209 and the like. The camera unit 219 operates under the control of the main control unit 215, and the main control unit 215 acquires the image data. This imaging data is sent to the output unit 216 via the main control unit 215, and an image to be captured is displayed on the liquid crystal display 27. Therefore, the user can capture the imaging target while viewing the image displayed on the liquid crystal display 27 when shooting using the camera 29. When the user performs a predetermined release operation on the operation unit 213, the main control unit 215 outputs a release command to the camera unit 219. As a result, the camera unit 219 images the imaging target and outputs the image data to the main control unit 215. The main control unit 215 stores this image data in the platform storage area of the RAM 202.

  A control program for constructing a telephone platform for operating the mobile telephone 20 according to a predetermined procedure is stored in the RAM 202 and the ROM 203. Further, a basic OS (operating system) program, a program for building the application execution environment, and an application program are also stored in the RAM 202 and the ROM 203. These programs are called up and executed in the work area in the CPU 201 or RAM 202 as necessary.

[Control example 1]
Next, one control example of data processing for image data obtained by imaging with the camera unit 219 (hereinafter, this control example will be referred to as “control example 1”) will be described.
This control example 1 is an example in the case of taking a landscape 360 ° horizontally from the mountain top as one image. Hereinafter, for the sake of explanation, the shooting angle of the built-in camera 29 is assumed to be 45 ° horizontal and 30 ° vertical. Therefore, in order to capture a horizontal 360 ° landscape as a single image, a total of eight images are captured while shifting the imaging direction by 45 ° in the horizontal direction, and then these images are arranged appropriately. It is necessary to perform data processing to be synthesized. Hereinafter, the present control example 1 will be described separately for an imaging process and a data processing process.

  Before starting the imaging process, first, the user 1 downloads and acquires a panorama shooting application program from the download server 11 and registers it. Specifically, the user 1 accesses the download server 11 by operating a key of the operation unit 213. As a result, a download selection screen for selecting a downloadable application program is displayed on the liquid crystal display 27. On the download selection screen, when the panorama shooting application program to be executed is selected using the scroll key 24 and the multifunction key 25 is pressed, the main control unit 215 controls the data communication unit 212, The application program is downloaded from the download server 11. The application program downloaded in this way is stored in the RAM 102 by the main control unit 215.

FIG. 1 is a flowchart illustrating the flow of the imaging process of the first control example.
At the summit, the user 1 first executes the downloaded application program for panoramic photography. Specifically, the user 1 operates a key of the operation unit 213 to display an application selection screen for selecting an application program to be executed on the liquid crystal display 27. Then, on the application selection screen, the panorama shooting application program to be executed is selected using the scroll key 24 and the multifunction key 25 is pressed. Then, an application program execution instruction is input to the telephone platform shown in FIG. 7, that is, the main control unit 215 shown in FIG. 6 (S1). As a result, the main control unit 215 reads the application program for panoramic photography and activates it (S2). When the application program is started, the application program operates on the application execution environment shown in FIG. 7, that is, the application program execution management unit 214 shown in FIG.

After the application program for panoramic photography is activated, after the application program is activated, the user 1 holds the cellular phone 20 so that the lens surface of the built-in camera 29 of the cellular phone 20 faces the imaging target. At this time, an image to be imaged captured by the built-in camera 29 is displayed on the liquid crystal display 27 of the mobile phone 20. Therefore, the user 1 can perform imaging while confirming the imaging target by viewing the image displayed on the liquid crystal display 27. When the first imaged part is determined from the 360 ° landscape, the user 1 performs a predetermined release operation on the operation unit 213 (S3). Here, it is assumed that the true north landscape is imaged first. Then, the operation signal is sent from the operation unit 213 to the main control unit 215, and the main control unit 215 outputs a release command to the camera unit 219. Upon receiving the release command, the camera unit 219 performs imaging (S4), and sends image data obtained thereby to the main control unit 215.
On the other hand, when receiving the operation signal from the operation unit 213, the main control unit 215 acquires data of the azimuth angle θ _N (imaging direction data) detected by the sensor detection unit 217 (S5). Here, since the azimuth angle θ _N is an angle formed by the Z-axis direction in which the imaging direction of the built-in camera 29 is directed and the north direction, the azimuth direction in which the imaging direction is directed can be specified by the azimuth angle θ _N. . Here, since a true north landscape is imaged, the azimuth angle θ _N is 0 °. The main control unit 215 functions as a data storage unit, and stores the platform data in the RAM 202 in a state in which the image data acquired from the camera unit 219 is associated with the data of the azimuth angle θ _N acquired from the sensor detection unit 217. Save to area. Specifically, the image data is stored in an image data storage area (image data storage means) in the platform storage area, and the data of the azimuth angle θ _N at the time of imaging the image data is management information of the image data Is stored in the management information storage area (imaging direction data storage means) (S6).

  After the first imaging is completed in this way, the user 1 next changes the orientation of the mobile phone 20 in the horizontal direction until the imaging direction of the built-in camera 20 changes by one image data. In the present embodiment, since the horizontal imaging angle of the built-in camera 29 is 45 °, the user 1 changes the orientation of the mobile phone 20 by 45 ° to the east side in the horizontal direction from the imaging direction at the time of first imaging. . At this time, in order to increase the continuity of the image in the composite image data obtained by the data processing process described later, the user 1 needs to properly change the orientation of the mobile phone 20 by 45 °. Therefore, in the present control example 1, in order to enable the user 1 to properly change the orientation of the mobile phone 20 by 45 °, the image pickup direction along with the image to be picked up by the built-in camera 29 is displayed on the liquid crystal display 27. An image showing is also displayed.

More specifically, when the user 1 performs a release operation and receives an operation signal from the operation unit 213, the main control unit 215 sends an event indicating that to the application program via the application execution environment. As a result, the application program for panoramic photography performs display control by the output unit 216 via the application execution environment so that the screen of the liquid crystal display 27 is vertically divided into two as shown in FIG. 8 (S7). . In this screen, the upper screen portion A displays an image to be imaged by the built-in camera 29 as described above, and the lower screen portion B shows an image indicating the imaging direction of the built-in camera 29. An indicator frame b ₁ is displayed. In addition, on the lower screen portion B, an arrow b ₂ indicating the target (northeast direction) of the imaging direction to be imaged _second is displayed at the upper center of the indicator frame b ₁ .
When the application program receives the event from the main control unit 215, the application program sends a measurement start command to the application program execution management unit 214. In response to this, the application program execution management unit 214 sends a measurement start command to the main control unit 215 of the telephone platform. Receiving this, the main control unit 215 thereafter starts processing to continuously acquire data of the azimuth angle θ _N detected by the sensor detection unit 217 (S8), and stores this in the application storage area in the RAM 202. Save temporarily. Thereby, when the user 1 who has finished the first imaging changes the orientation of the mobile phone to the horizontal direction, the data of the azimuth angle θ _N corresponding to the change is continuously output from the sensor detection unit 217 to the main control unit 215. The Therefore, the main control unit 215 can continuously acquire data of the azimuth angle θ _{N in} substantially real time.

When the user 1 who finished the first imaging changes the orientation of the mobile phone to the horizontal direction, the application program changes the current imaging direction in near real time based on the azimuth angle θ _N data stored in the application storage area. I can grasp it. Then, according to the imaging direction that is grasped, performs display control so that the length of the indicator b ₃ indicated by hatching in FIG. 8 is changed (S9). By performing such display control, the user 1 can change the orientation of the mobile phone 20 in the horizontal direction so that the tip of the indicator b ₃ reaches the arrow b ₂ . Then, when the tip of the indicator b ₃ reaches the arrow b ₂ , the imaging direction of the built-in camera 29 is changed by 45 ° eastward in the horizontal direction from the imaging direction at the first imaging. As described above, the user 1 uses the indicator b ₃ and the arrow b ₂ displayed on the liquid crystal display 27, and after the first imaging is completed, the imaging direction of the built-in camera 29 is 1 of the image data obtained by the first imaging. It is notified that the image has changed by the amount. Therefore, if the user 1 performs the release operation when the tip of the indicator b ₃ reaches the arrow b ₂ , the user 1 can capture an image having high continuity with respect to the first captured image. Note that when the tip of the indicator b ₃ is present at the position of the arrow b ₂ , that is, when the data of the azimuth angle θ _N matches the target of the next imaging direction (north azimuth direction: 45 °), the output unit 216 You may make it emit a sound from the speaker 28 of this. As a result, the user 1 can more easily recognize the direction in which the imaging direction of the built-in camera 29 matches the target of the next imaging direction, and can further improve image continuity.

As described above, when shooting the second and subsequent images, the user 1 performs the release operation while checking the indicator b ₃ , and the main control unit 215 determines the image data and the azimuth angle θ _N at the time of imaging. The process of saving data is repeated (S3 to S9). Then, after imaging 8 times and capturing all 360 ° scenery, the user 1 operates the keys of the operation unit 213 to end the imaging process (S10). As a result, eight image data as shown in Table 1 below are stored in the platform storage area of the RAM 202.

FIG. 9 is a flowchart showing the flow of the data processing process of the first control example.
When the user performs a predetermined operation on the key of the operation unit 213 in order to start the data processing process after the imaging process is finished, the main control unit 215 causes the application program execution management unit 214 as a data processing unit. The application program for panoramic photography starts a data processing process. When the data processing process starts, first, the main control unit 215, based on an instruction from the application execution environment using the telephone platform library, the eight image data stored in the platform storage area of the RAM 202 and the associated image data. Data of the azimuth angle θ _N is read, and these data are transferred to the application storage area (S11). Thereafter, the application program execution management unit 214 reads out the data of the azimuth angle θ _N from the data transferred to the application storage area according to the contents of the application program (S12), and based on the data of each azimuth angle θ _N An appropriate arrangement order of each image data is specified (S13). Then, a data composition process is performed to synthesize the image data into one image data so that the images of the image data become one image arranged in the arrangement order (S14). The synthesized image data synthesized in this way is stored in the application storage area of the RAM 202 (S15).

As described above, according to the present control example 8, eight image data obtained by the user 1 capturing a landscape 360 ° horizontally from the top of the mountain in eight portions are obtained as azimuth θ _N data at the time of capturing each image data. Based on (imaging direction data), it is possible to combine appropriately. Therefore, a horizontal 360 ° landscape seen from the top of the mountain can be obtained as one image. Moreover, in the present control example 1, since the captured image data is associated with the data of the azimuth angle θ _N at the time of imaging, it is appropriate when combining image data even if the arrangement order of the images and the imaging order are different. The arrangement order can be specified. Therefore, the convenience of the user 1 when performing panoramic shooting is very high.

In the present control example 1, the indicator for guiding the user 1 so that the user performs imaging in the appropriate imaging direction to be imaged next is not limited to that shown in FIG.
For example, a circular indicator frame c ₁ as shown in FIG. 10 can be used. In this case, when the user 1 having been subjected to the first imaging changing the orientation of the mobile phone 20 in the horizontal direction, according to the imaging direction is grasped based on the data of azimuth theta _N, three diamond mark circular indicator Display control is performed so as to move along the circumference of the frame c ₁ . In this indicator frame c ₁ , a rhombus mark c ₂ indicating the imaging direction when the image is first captured and a rhombus mark c ₃ indicating the target of the imaging direction to be imaged second (the northeast direction and the northwest direction) are also displayed. ing. Moreover, this indicator frame c _1, the cross of the aiming mark is displayed and a line extending in the vertical direction of the sight indicates the current imaging direction. Therefore, the user 1 changes the orientation of the mobile phone 20 in the horizontal direction so that the next target rhombus mark c ₃ overlaps the line of the aiming mark indicating the current imaging direction. Imaging can be performed in an appropriate imaging direction to be imaged.
Further, a circular indicator frame d ₁ as shown in FIG. 11 can be used. In this case, when the user 1 having been subjected to the first imaging changing the orientation of the mobile phone 20 in the horizontal direction, according to the imaging direction is grasped based on the data of azimuth theta _N, indicates the current imaging direction arrow d ₄ performs display control to move along a circumference of the circular indicator frame c _1. Among the rhombus marks displayed in the indicator frame d ₁ , the _one indicated by the symbol d ₂ is a mark indicating the imaging direction when first imaged, and the one indicated by the symbol d ₃ is an image to be imaged second. It is a mark indicating a direction target (northeast direction and northwest direction). Therefore, the user 1 changes the direction of the mobile phone 20 in the horizontal direction so that the arrow d ₄ indicating the current imaging direction points to the next target diamond mark d ₃ , so that Imaging can be performed in an appropriate imaging direction to be imaged.

In the present control example 1, as shown in FIGS. 8, 10, and 11 , the user 1 is guided by the indicator so that the user performs imaging in an appropriate imaging direction to be imaged next. As described above, the user 1 may be guided by other notification means that outputs voice or the like.
Further, in the present control example 1, the case where an imaging target that cannot be captured at once in the horizontal direction has been described, but the same applies to the case where an imaging target that cannot be captured at once in the vertical direction is captured. In this case, instead of the azimuth angle θ _N data, the pitch angle θ _X data may be stored in association with the image data as imaging direction data. Combining image data obtained by capturing images in such a plurality of times in the vertical direction is effective, for example, when capturing the entire high-rise building at a relatively close location. Also, the process of combining a plurality of images in the horizontal direction and the process of combining a plurality of images in the vertical direction can be combined. In this case, in addition to the data of the azimuth angle θ _N, the data of the pitch angle θ _X is also stored in association with the image data as the imaging direction data, and 2 based on the data of the azimuth angle θ _{N and} the data of the pitch angle θ _X. What is necessary is just to specify a dimension array and to perform a data composition process.

[Modification 1]
According to the control example 1, the indicator 1 allows the user 1 to recognize an appropriate imaging direction to be imaged next only in the horizontal direction. However, even if the image is picked up in an appropriate image pickup direction only in the horizontal direction, the continuity of the image is lowered if the image pickup direction in the vertical direction is different from other image data. Therefore, it is preferable to guide the user 1 by displaying not only a horizontal indicator but also a vertical indicator on the liquid crystal display 27. When displaying such an indicator in the vertical direction, the pitch angle θ _X detected by the sensor detection unit 217 may be used. Even if the imaging directions in the horizontal direction and the vertical direction are in appropriate directions, if the mobile phone 20 is rotated about the imaging direction as a rotation axis, the vertical direction of the image does not match the vertical direction. The captured image is tilted. Also in this case, the continuity of the image is lowered. Therefore, it is preferable to guide the user 1 by displaying not only horizontal and vertical indicators on the liquid crystal display 27 but also an indicator about the rotation of the mobile phone 20. In order to display an indicator for the rotation of the mobile phone 20, the acceleration α _X and α _Y data detected by the sensor detection unit 217 may be used. Even by utilizing the yaw angle theta _Z detected by the sensor detecting unit 217 can display an indicator for the rotation of the mobile phone 20.

Specifically, for example, an indicator frame e ₁ as shown in FIGS. 12A and 12B is used instead of the indicator frame b ₁ shown in FIG. The pitch angle θ _X detected by the sensor detection unit 217 when the user 1 performs the release operation for the first image pickup is stored, and this is set as the reference pitch angle θ _X0 . When the indicator frame e ₁ shown in FIGS. 12A and 12B is used, when the user 1 who has finished the first imaging changes the orientation of the mobile phone 20 to the horizontal direction, the data is based on the azimuth θ _N data. Display control is performed so that the solid rhombus mark e ₂ indicating the imaging direction moves along the circumference of the circular indicator frame e ₁ according to the grasped imaging direction. In the indicator frame e ₁ , a broken-line rhombus mark e ₃ indicating a target of the imaging direction to be imaged secondly (northeast direction and northwest direction) is also displayed. Therefore, the user 1 changes the orientation of the mobile phone 20 in the horizontal direction so that the solid rhombus mark e ₂ indicating the current imaging direction reaches the target broken rhombus mark e ₃ . Next, imaging can be performed in an appropriate imaging direction to be imaged.

In addition, while changing the orientation of the mobile phone 20 in the horizontal direction in this way, the orientation of the mobile phone may change in the vertical direction. This change can be grasped by the difference between the pitch angle θ _X data detected in real time and the reference pitch angle θ _X0 data. And according to this difference, display control is performed so that the solid rhombus mark e ₂ indicating the imaging direction moves in the vertical direction of the circular indicator frame e ₁ . Thereby, the user 1 adjusts the orientation of the mobile phone 20 so that the vertical position of the solid rhombus mark e ₂ in the circular indicator frame e ₁ coincides with the vertical position of the target dashed rhombus mark e _3. In this way, it is possible to suppress the mobile phone from being displaced in the vertical direction with respect to the first captured image during the next imaging.

Furthermore, as described above, the mobile phone 20 rotates around the imaging direction as the rotation axis while the orientation of the mobile phone 20 is changed to the horizontal direction or adjusted in the vertical direction in this way. Sometimes. That is, in the first modification, the Y axis shown in FIG. 4 needs to coincide with the vertical direction at the time of imaging, and if the mobile phone 20 rotates about the Z axis as the rotation axis, it coincides with the Z axis direction. Although the captured image direction is not changed, the vertical direction of the captured image is shifted from the vertical direction, and the continuity of the image is deteriorated. Therefore, in the first modification, such rotation of the mobile phone 20 is grasped based on acceleration α _X and α _Y data obtained by detecting gravitational acceleration with the acceleration sensor 207 of the sensor detection unit 217. Specifically, the inclination angle of the mobile phone 20 with respect to the vertical direction obtained based on the acceleration α _X and α _Y data is grasped as the rotation angle of the mobile phone 20. Then, display control is performed so that the solid diamond mark e ₂ indicating the imaging direction rotates (autorotates) according to the grasped rotation angle. Accordingly, the user 1 adjusts the posture of the mobile phone 20 so that the rotation posture of the solid rhombus mark e ₂ matches the rotation posture of the target broken rhombus mark e _3. Can be prevented from shifting from the vertical direction.

According to the first modification, when the solid rhombus mark e ₂ in the circular indicator frame e ₁ matches the target broken rhombus mark e ₃ , the image capturing direction of the built-in camera 29 is first captured. In this case, the image is changed from the imaging direction to the east side by 45 ° in the horizontal direction, there is no deviation in the vertical direction, and there is no deviation between the vertical direction and the vertical direction of the captured image. In the first modification, a sound is output when the solid rhombus mark e ₂ matches the dashed rhombus mark e ₃ , so that the user 1 performs a release operation when the sound is output, An image having high continuity with respect to the image captured first can be captured.

As described above, when image capturing is completed eight times and all 360 ° landscapes are imaged, eight image data as shown in Table 2 below are stored in the platform storage area of the RAM 202.

Even if the user 1 is guided by the indicator frame e ₁ , as shown in Table 2 above, the acceleration α _X data is −10 during the second and sixth imaging, and the Y axis of the mobile phone 20 is vertical. The mobile phone 20 has been rotated with respect to the direction. In addition, at the time of the fourth imaging, a difference occurs between the data of the pitch angle θ _X at the time of imaging and the data of the reference pitch angle θ _X0 , and the imaging direction Y axis is shifted in the vertical direction. . As in the case of the control example 1, the image continuity cannot be obtained simply by arranging the image data in an appropriate arrangement order based on the data of each azimuth angle θ _N and combining them. Therefore, in the first modification, the image continuity is based on the acceleration α _X , α _Y data and the difference between the pitch angle θ _X data and the reference pitch angle θ _X0 data associated with each image data. Process the image data to increase. For example, for the second and sixth image data in which the mobile phone 20 has been rotated, the image data is rotated by the rotation angle obtained from the acceleration α _X and α _Y data. Process.

[Control example 2]
Next, another control example of data processing on image data obtained by imaging with the camera unit 219 (hereinafter, this control example will be referred to as “control example 2”) will be described.
This control example 2 is an example in which a horizontally long poster that cannot be photographed once is captured as a single image. In order to capture such a horizontally long poster as a single image, data processing is performed in which images are captured while the image capturing position is shifted by one image in the horizontal direction, and then these images are appropriately arranged and combined. Need to do. Hereinafter, the second control example will be described separately for the imaging process and the data processing process. Note that the shooting angle of the built-in camera 29 is the same as in the control example 1 described above. In addition, the poster is assumed to have a vertical length of 0.5 m and a horizontal length of 4 m.

FIG. 13 is a flowchart illustrating the flow of the imaging process of the second control example.
Before starting this imaging process, the user 1 downloads and acquires the application program for landscape shooting from the download server 11 and registers it, as in the first control example. Then, the user 1 executes the downloaded application program for landscape shooting (S21 to S22). After the application program is activated, the user 1 holds the mobile phone 20 so that, for example, the left end portion of the landscape poster is directed to the lens surface of the built-in camera 29 of the mobile phone 20 so as to be imaged. Then, the user 1 performs a predetermined release operation on the operation unit 213 (S23). Thereby, the camera unit 219 performs imaging (S24), and the image data obtained thereby is sent to the main control unit 215. The main control unit 215 functions as a data storage unit, and stores the image data acquired from the camera unit 219 in the platform storage area in the RAM 202 in a state in which the image data is associated with movement distance data (imaging position data) described later. To do. Specifically, the image data is stored in an image data storage area (image data storage means) in the platform storage area, and the management information of the image data is stored as the movement distance data when the image data is captured. Stored in the management information storage area (imaging position data storage means) (S25). Since the current shooting is the first shooting, the moving distance data is zero.

After completing the first imaging in this manner, the user 1 next changes the position of the mobile phone 20 in the horizontal direction until the imaging position of the built-in camera 20 changes by one image data. Here, since the vertical imaging angle of the built-in camera 29 is 30 ° and the vertical length of the poster is 0.5 m, in order to capture the vertical length of the poster at a time, the built-in camera 29 The shooting distance from the lens surface to the poster needs to be 1 m. Since the horizontal imaging angle of the built-in camera 29 is 45 °, in order to move the imaging position of the built-in camera 20 by one image of the image data, the user 1 must move horizontally from the imaging position when the image was first captured. It is necessary to move along the poster by about 0.8m. In addition, this imaging distance can be obtained by measuring before the first imaging, for example. As a specific example, the user 1 holds the mobile phone 20 so that the XY plane shown in FIG. 4 is horizontal, and contacts the bottom surface (lower end surface in FIG. 4) of the mobile phone 20 with the poster surface. In the posture, the operation unit 213 is operated to instruct the start of measurement. Upon receiving this instruction, the application program causes the main control unit 215 to continuously acquire the acceleration α _Y data from the sensor detection unit 217 and receives the data. Then, the user 1 moves to the shooting point in a direction away from the poster while holding the mobile phone 20 in the above posture. During this movement, the application program calculates the moving distance of the mobile phone by integrating the acceleration α _Y data twice. Thereby, the application program can obtain the shooting distance.

  Also in this control example 2, in the same manner as in the above control example 1, in order to improve the continuity of the image in the composite image data obtained by the data processing step described later, the imaging target imaged by the built-in camera 29 on the liquid crystal display 27 is selected. Along with the image, an image indicating the imaging position is also displayed. More specifically, when the user 1 performs a release operation and receives an operation signal from the operation unit 213, the main control unit 215 sends an event indicating that to the application program via the application execution environment. As a result, the application program for landscape shooting performs display control by the output unit 216 via the application execution environment so that the screen of the liquid crystal display 27 is vertically divided into two in the same manner as shown in FIG. S26). In the case of this control example 2, the arrow displayed on the lower screen portion of the screen indicates the target of the imaging position to be imaged second.

When the application program receives the event from the main control unit 215, the application program sends a measurement start command to the application program execution management unit 214. In response to this, the application program execution management unit 214 sends a measurement start command to the main control unit 215 of the telephone platform. Receiving this, the main control unit 215 thereafter starts processing to continuously acquire the data of the acceleration α _X detected by the sensor detection unit 217 (S27), and temporarily stores it in the application storage area in the RAM 202. To save. Accordingly, when the user 1 who has finished the first imaging moves the position of the mobile phone in the horizontal direction along the poster while the imaging direction, that is, the Z-axis is directed toward the poster, the acceleration α _X corresponding to the movement is obtained. Is continuously output from the sensor detection unit 217 to the main control unit 215. Therefore, the main control unit 215 can continuously acquire the acceleration α _X data substantially in real time.

When the position of the mobile phone moves in the horizontal direction in this way, the application program integrates the acceleration α _X data stored in the application storage area twice to continuously obtain the movement distance data of the mobile phone 20. While calculating, indicator display control is performed (S28). Since this moving distance data indicates the distance that the mobile phone 20 has moved from the imaging position when the first image was taken, the current position with respect to the first shooting position can be specified from this moving distance data. Then, the application program performs display control according to the movement distance data so that the length of the indicator changes as in the control example 1 described above. By performing such display control, the user 1 can move the position of the mobile phone 20 in the horizontal direction so that the tip of the indicator reaches the arrow indicating the target. Then, by performing a release operation when the tip of the indicator reaches the arrow, it is possible to capture an image having high continuity with respect to the image captured first. Similar to the control example 1 described above, sound may be emitted from the speaker 28 of the output unit 216 when the tip of the indicator exists at the position of the arrow.

As described above, when shooting the second and subsequent images, the user 1 performs the release operation while checking the indicator, and the main control unit 215 moves the image data and the movement distance data (imaging position data) at the time of imaging. ) Is repeatedly performed (S23 to S28). Then, after imaging five times and capturing the entire poster, the user 1 operates the keys of the operation unit 213 to end the imaging process (S29). As a result, five image data as shown in Table 3 below are stored in the platform storage area of the RAM 202.

FIG. 14 is a flowchart showing the flow of the data processing step of the present control example 2.
When the user performs a predetermined operation on the key of the operation unit 213 to start the data processing process after the imaging process is completed, the application program for landscape shooting is processed by the data processing similarly to the control example 1 described above. Start the process. Then, similarly to the control example 1, the main control unit 215 shifts the five image data and the movement distance data associated therewith to the application storage area (S30). Thereafter, the application program reads movement distance data from the data transferred to the application storage area (S31), and specifies an appropriate arrangement order of the image data based on the movement distance data (S32). Then, after combining each image data to form one image data so that the image of each image data becomes one image arranged in the arrangement order (S33), the combined image Data is stored in the application storage area of the RAM 202 (S34).

  As described above, according to the second control example, the five image data obtained by the user 1 capturing the horizontally long poster in five times are obtained based on the movement distance data (imaging position data) at the time of capturing each image data. Can be synthesized appropriately. Therefore, a horizontally long poster can be obtained as one image. In addition, in this control example 2, since the captured image data is associated with the movement distance data at the time of imaging, an appropriate arrangement order can be set when combining the image data even if the arrangement order of the images and the order of the imaging are different. Can be identified. Therefore, the convenience of the user 1 when photographing such a horizontally long subject is very high.

In the second control example, the case where the user guides the user 1 to take an image at an appropriate imaging position to be imaged next by the indicator has been described. However, the user is notified by other notification means that outputs sound or the like. 1 may be induced.
Further, according to this control example 2, the indicator allows the user 1 to recognize an appropriate imaging position to be imaged next only in the horizontal direction. However, even if the image is picked up at an appropriate image pickup position only in the horizontal direction, the image continuity deteriorates if the image pickup direction in the vertical direction is different from other image data. Therefore, it is preferable to guide the user 1 by displaying not only the horizontal indicator but also the vertical indicator on the liquid crystal display 27 as in the control example 1 described above. The method for displaying such a vertical indicator is the same as in the first control example.
Further, in the present control example 2, a case has been described in which an imaging target that cannot be captured once in the horizontal direction is described, but the same applies to a case where an imaging target that cannot be captured in the vertical direction is captured once. In this case, instead of the movement distance data calculated from the acceleration α _X in the X-axis direction, movement distance data obtained by integrating the acceleration α _Y in the Y-axis direction twice is associated with the image data as imaging position data. Just remember. Also, the process of combining a plurality of images in the horizontal direction and the process of combining a plurality of images in the vertical direction can be combined. In this case, in addition to the travel distance data calculated from the acceleration alpha _X in the X-axis direction, also the moving distance data calculated from the acceleration alpha _X in the Y-axis direction, and stored in association with the image data as the image position data, these A two-dimensional array may be specified based on the movement distance data and data synthesis processing may be performed.

[Modification 2]
Next, a modified example of the mobile phone 20 (hereinafter, this modified example is referred to as “modified example 2”) will be described.
When imaging is performed using the built-in camera 29 fixed to the mobile phone 20, the user 1 needs to change the imaging direction and the imaging position by changing the orientation or moving the position of the mobile phone 20. . When the user changes the imaging direction or imaging position, even if there is guidance by the indicator as described above, the imaging direction or imaging position is likely to be misaligned, and the synthesized image has high image continuity stably. Difficult to get. Therefore, in the second modification, a configuration in which the imaging direction and the imaging position are automatically performed on the mobile phone 20 side will be described.

FIG. 15 is an external view showing an external camera device 30 as imaging means connected to an external terminal of the mobile phone 20.
The external camera device 30 includes a connection unit 31 connected to an external terminal (not shown) provided on the bottom of the mobile phone 20 so as to be able to perform data communication, and a movable camera unit 32 rotatably attached to the connection unit 31. It is composed of A camera similar to the built-in camera 29 is accommodated inside the movable camera unit 32, and its lens surface 32a is exposed to the outside. Further, the rotation shaft 32b extends from a position shifted by 90 ° with respect to the lens surface 32a, and an end portion of the rotation shaft 32b is rotatably supported by the connection portion 31. The end of the rotating shaft 32 b is connected to a drive motor (not shown) provided inside the connection portion 31. Thereby, the imaging direction of the movable camera unit 32 can be changed by rotating the drive motor.

FIG. 16 is a block diagram showing an extracted main part of the mobile phone 20 to which the external camera device 30 is connected. Although the main body configuration of the mobile phone 20 is the same as that described above, in the second modification, the external camera device 30 is used instead of the built-in camera 29, and thus the camera unit 219 configured by the built-in camera 29 is used. The illustration about is omitted.
The external camera device 30 includes a camera unit 319, a camera driving unit 320, a communication interface not shown, and the like. The camera unit 319 corresponds to the movable camera unit 32 on the hardware configuration described above, and the camera driving unit 320 corresponds to the connection unit 31 on the hardware configuration described above. The camera unit 319 and the camera driving unit 320 are connected to the main control unit 215 of the mobile phone main body via a communication interface and an external terminal. The camera driving unit 320 functions as a changing unit that changes the imaging direction of the camera unit 319 under the control of the main control unit 215 of the mobile phone body. Specifically, in accordance with a drive control command sent from the main control unit 215, a drive motor (not shown) is drive-controlled to change the imaging direction of the camera unit 319. Since the camera unit 319 operates in the same manner as the camera unit 219 configured by the built-in camera 29 described above, the description thereof is omitted.

FIG. 17 is a flowchart illustrating the flow of the imaging process of the second modification.
In the second modification, as in the case of the control example 1 described above, a case where a landscape from the top is taken as a single image is taken as an example. Here, a landscape at a horizontal angle of 180 ° from the top is taken as a single image. It shall be. Before starting the imaging process, the user 1 downloads and acquires the panorama shooting application program using the external camera device 30 from the download server 11 and registers it, as in the control example 1. . Then, the user 1 executes the downloaded application program for landscape shooting (S41 to S42). After the application program is activated, the user 1 operates the keys of the operation unit 213 to specify the shooting range (S43). Specifically, the angle range for photographing in the horizontal direction is input using the keys of the operation unit 213. In the second modification, this angle range is 180 ° as described above. The input angle range is sent to the main control unit 215. When the shooting range is designated in this way, the user 1 moves the mobile phone 20 so that the lens surface 32a of the movable camera unit 32 of the mobile phone 20 faces the leftmost part of the landscape of, for example, 180 °. Hold it. Then, the user 1 performs a predetermined release operation on the operation unit 213 (S44). Then, the operation signal is sent from the operation unit 213 to the main control unit 215, and the main control unit 215 sends an event indicating that a release has occurred to the application program for panoramic photography through the application execution environment.

The application program that has received the event first sends a release instruction to the main control unit 215 using the telephone platform library via the application execution environment. Accordingly, the main control unit 215 outputs a release command to the camera unit 319, and the camera unit 319 that has received the release command performs imaging (S45). The image data obtained in this way is sent to the main control unit 215. Further, the main control unit 215 acquires data (imaging direction data) of the azimuth angle θ _N detected by the sensor detection unit 217 (S46), and uses the image data acquired from the camera unit 319 as the azimuth angle θ _N. In a state associated with the data, the data is stored in the image data storage area of the platform storage area in the RAM 202 (S47). The azimuth angle θ _N data is stored in the management information storage area of the platform storage area (S47). After completing the first imaging in this manner, the application program then issues a camera drive instruction to rotate the movable camera unit 32 by a certain angle using the telephone platform library via the application execution environment. Send to. As a result, the main control unit 215 outputs a drive command to the camera unit 320 of the external camera device 30. Then, according to the drive command, the camera unit 320 rotates the movable camera unit 32 so that the imaging direction is deviated by 45 ° from the time of the first shooting (S49). Thereafter, the application program again sends a release instruction to the main control unit 215 using the telephone platform library via the application execution environment, and repeats the processes of S45 to S49. When the rotation angle of the movable camera unit 32 from the position at the time of the first shooting reaches the shooting range specified in S43, that is, the angle range (180 °) input by the user 1, the imaging process is ended (S48). ). As a result, four image data items are stored in the platform storage area of the RAM 202 in association with the azimuth angle θ _N data (imaging direction data) at the time of imaging.
The data processing step of combining the four image data obtained in this way into one image data is the same as that in the control example 1 and will not be described.

  According to the second modification, when shooting a landscape of 180 ° horizontally from the top of the mountain, the user 1 simply holds the mobile phone 20 so as not to move and performs a release operation once. An imaging process is automatically performed according to the application program. In addition, in the second modification, since the shooting direction is automatically changed by the camera driving unit 320, as compared to the case where the user changes the shooting direction by changing the orientation of the mobile phone 20 as in the first control example. High image continuity can be stably obtained for the synthesized image.

In the second modification, the case has been described in which an imaging target that cannot be captured once in the horizontal direction is described. However, the same applies to the case where an imaging target that cannot be captured in the vertical direction is captured once. In this case, the cellular phone 20 is gripped and photographed so that the X-axis direction is parallel to the vertical direction, and the rotation angle data of the movable camera unit 32 is replaced with the imaging direction data instead of the azimuth angle θ _N data. May be stored in association with the image data. Further, depending on the movable range of the movable camera unit 32, it is possible to combine a process of combining a plurality of images in the horizontal direction and a process of combining a plurality of images in the vertical direction.
Further, in the second modification, the configuration in which the movable camera unit 32 of the external camera device 30 rotates and changes the imaging direction while the main body of the mobile phone 20 is stationary has been described. It may be changed. For example, a tripod 33 as shown in FIG. 18 may be used. More specifically, the illustrated tripod 33 has a built-in drive unit for rotating the main body of the mobile phone 20 in the connection portion 33 with the external terminal of the mobile phone 20. This drive unit has the same configuration as the camera drive unit 320 of the external camera device 30. If the mobile phone 20 is attached to such a tripod 33 and shooting is performed, panoramic shooting as in Modification 2 can be performed using the built-in camera 29 fixedly provided on the mobile phone 20.
In the second modification, the case where the image data obtained by imaging is stored in association with the data of the azimuth angle θ _N at the time of imaging has been described as an example. The same effect can be obtained even in the case of saving in association with the movement distance data calculated from the acceleration data.

In the above-described embodiment (including the above-described control examples and the above-described modification examples; the same applies hereinafter), a program that executes the above-described imaging process and data processing process is an application execution environment of the application program execution management unit 214. Although the case of the application program operating above has been described as an example, the application program may operate directly on the telephone platform of the main control unit 215. In this case, the present invention can also be applied to a mobile phone that does not include the application program execution management unit 214 as shown in FIG. In this case, the application program depends on the telephone platform. Further, the main control unit 215 performs the processing performed by the application program execution management unit 214.
In the above-described embodiment, image data in a state associated with imaging direction data such as the azimuth angle θ _N data and imaging position data such as the moving distance data is obtained from a plurality of image data. However, other usage forms may be used. For example, when displaying an image obtained by imaging, there is a usage form in which an image based on imaging direction data and imaging position data at the time of imaging is displayed together.
Further, the present invention can be applied to a case of a mobile phone, a phone such as a PHS and a car phone, and a portable PDA, and similar effects can be obtained.

The flowchart which shows the flow of the imaging process of the control example 1 of the mobile telephone in an embodiment. Explanatory drawing for demonstrating the whole structure of the mobile communication system which can use the mobile phone. The schematic block diagram which shows the hardware constitutions of the download server which comprises the mobile communication system. The front view which shows the external appearance of the mobile phone. The schematic block diagram which shows the hardware constitutions of the mobile phone. The block diagram which extracted and showed the principal part of the mobile phone. Explanatory drawing of the software structure in the mobile phone. Explanatory drawing which shows the display content of the liquid crystal display of the mobile telephone in the imaging process of the control example 1. FIG. 7 is a flowchart showing a flow of a data processing process of the control example 1; Explanatory drawing which shows the other example of the indicator in the example 1 of the control. Explanatory drawing which shows the further another example of the indicator in the control example 1. FIG. Explanatory drawing which shows the indicator displayed on the liquid crystal display of the mobile phone in the imaging process of the modification 1. The flowchart which shows the flow of the imaging process of the control example 2 of the same mobile phone. 7 is a flowchart showing a flow of a data processing process of the control example 2. FIG. 14 is an external view showing an external camera device connected to an external terminal of a mobile phone in Modification 2. The block diagram which extracted and showed the principal part of the mobile telephone to which the same external camera apparatus was connected. 9 is a flowchart showing a flow of an imaging process in the second modification. The perspective view of the state which attached the mobile phone to the tripod which shows the other structure of the mobile phone. The block diagram which shows the other internal structure of the mobile phone.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Mobile phone communication network 11 Download server 20 Mobile phone 29 Built-in camera 30 External camera device 32 Movable camera part 32b Rotating shaft 207 Acceleration sensor 208 Geomagnetic sensor 213 Operation part 214 Application program execution management part 215 Main control part 216 Output part 217 Sensor detection Sections 219 and 319 Camera section 320 Camera drive section

Claims (2)

In a mobile communication terminal comprising an imaging means and an image data storage means for storing image data obtained by imaging with the imaging means,
Detection means for performing detection for specifying the imaging direction of the imaging means;
Imaging direction data storage means for storing imaging direction data indicating an imaging direction specified by detection of the detection means performed at the time of imaging by the imaging means;
Data storage means for storing image data obtained by the imaging in the image data storage means, and for storing imaging direction data at the time of imaging in the imaging direction data storage means ;
After imaging by the imaging means, the imaging direction changed by one image is calculated from the image data captured at the time of imaging based on the imaging direction data at the time of imaging, and an image indicating the calculated imaging direction is detected. A mobile communication terminal comprising: a display control unit that performs control to display the image on the display unit together with an image indicating a current imaging direction specified by the detection of the unit . The mobile communication terminal 請 Motomeko 1,
Said plurality of image data stored in the image data storage means, and synthesized based on the imaging direction data associated with each of the image data of the plurality of data processing means for performing data processing for one image data mobile communication terminal characterized by having a.

Images