JP5243280B2 - Mobile terminal and synthesis program - Google Patents

Description

  The present invention relates to a mobile terminal, and more particularly to a mobile terminal capable of taking a panoramic photograph, for example.

2. Description of the Related Art Conventionally, portable terminals capable of taking panoramic photographs, for example, have been known, and an example of this type of apparatus is disclosed in Patent Document 1. In the camera of this background art photographic system, position data capable of identifying a photographic location is acquired by wireless communication with a station arranged around or communication with a GPS satellite. Further, the image taken by the camera is associated with position data and stored in the memory. The image associated with the location data is then associated with the stored content stored in the database by the processor of the processing subsystem to create an extended image product such as a panorama and 360 degree photo. Further, the developer can print the content information (for example, the location where the image was taken) of the photographed image on a sticker and paste it on the back side of the photograph by the developer himself.
JP 2004-289825 [H04N 5/76, G06T 1/00, H04N 5/907]

  However, the camera in the photographic system in Patent Document 1 does not take into consideration that a panoramic photograph is divided into a plurality of parts and developed. For example, photographic paper for panoramic photographs has a special size and is expensive. Therefore, when storing a panoramic photo in an album, the developer may use a plurality of L-size photographic papers and divide the panoramic photo into a plurality of photos for development. However, after a while after the photo is stored in the album, the developer forgets the arrangement of the plurality of divided and developed photos and cannot be rearranged. Patent Document 1 discloses a technique for printing content information indicating a photographed location on a sticker, but the user only remembers the photographed location even when looking at the sticker on the back side.

Therefore, a main object of the present invention is to provide a novel portable terminal , an image storage program, and an image storage method .

Another object of the present invention is to provide a portable terminal , an image storage program, and an image storage method capable of easily rearranging a plurality of images.

  The present invention employs the following configuration in order to solve the above problems. The reference numerals in parentheses, supplementary explanations, and the like indicate the corresponding relationship with the embodiments described in order to help understanding of the present invention, and do not limit the present invention.

Orientation first invention, a plurality of images captured in a panoramic mode, respectively stored as separate images, a portable terminal, to acquire imaging unit for capturing an image, a rectangular position information in accordance with the photographing operation information acquisition means, tilt detecting means for detecting the tilt of the portable terminal, when the photographing operation is in a state where the panoramic mode is set, the inclination of the rectangular position information and the portable terminal, in association with each of the plurality of images first storage means for storing, and plurality of order information based rather panoramic disposed orientation information associated with each image, a second memory means for storing given to a plurality of images, the panoramic arrangement sequence information, the position to be imparted based on the slope associated with each of the plurality of images Ru determined, a portable terminal.

In the first invention, the mobile terminal (10) stores a plurality of images taken in the panorama mode as individual images. The user can take an image with the photographing means (36, 38) by performing a photographing operation . Further, the orientation information acquisition means (20, 34, S3) acquires the orientation information indicated by the optical axis direction of the imaging device. The inclination detecting means (20, 32, S5) detects the inclination of the portable terminal. The first storage means (20, 30, S9) stores the azimuth information and the inclination of the mobile terminal in association with each of the plurality of images when a photographing operation is performed in a state where the panorama mode is set. The second storage means (20, S17 , S21 ) assigns and stores the panorama arrangement order information based on the azimuth information associated with each of the plurality of images. Sequence information of the panoramic arrangement, Ru position granted under the slope associated with each of the plurality of images are determined.

According to the first aspect of the invention, the user can easily rearrange the panoramic photographs that are divided and developed by referring to the order information determined based on the orientation information.
In particular, the user can refer to the order information given based on the inclination of the mobile terminal, so that the user can accurately recognize the direction of the ground with respect to the subject and rearrange panoramic photos, for example. become.

A second invention is according to the first inventions, when further comprising a time obtaining means for obtaining a time, first storage means, by the time the acquisition by time acquisition means, corresponding to each of the plurality of images give stored, the second storage means, if the same orientation data is at least two images associated with the sequence of a panoramic arranged based on direction information and the time associated with each of the plurality of images Decide information.

In the second invention, the time acquisition means (20, 20a, S7) acquires the time. The first storage means stores the time acquired by the time acquisition means in association with each of the plurality of images. Second storage means, the same if the direction information there are two or more images associated with, Ru determine the order information panorama arranged based on direction information and the time associated with each of the plurality of images.

According to the second invention, when images of the same orientation are associated there is more than one, Ru order information are determined based on the captured time.

A third invention is the first invention or dependent on the second inventions, the second storage means, it stores imparted information indicating the orientation in the image.

According to the third invention, for example, when a user takes a plurality of panoramic photos at the same place, the user distinguishes the subject (mountain or the like) shown in each panoramic photo from information on the orientation given to each panoramic photo. Will be able to.

The fourth invention has a photographing means (36, 38) for photographing an image according to a photographing operation, and stores a plurality of images photographed in the panorama mode as individual images, respectively. of the processor (20), the orientation information acquiring means for acquiring the rectangular position information (S3), the inclination detecting means for detecting the tilt of the portable terminal, when the photographing operation is in a state where the panoramic mode is set, square position the inclination of the information and the mobile terminal, a first storage means (S9) for storing in association with each of the plurality of images, and a plurality of order information based rather panoramic disposed orientation information associated with each image , to function as a second storage means (S17, S21) for storing assigned to a plurality of images, the sequence information of the panoramic arrangement is given on the basis of the slope associated with each of the plurality of images Location is Ru determined, an image storage program.
5th invention is an image storage method of a portable terminal which has a photography means which picturizes an image according to photography operation, and memorizes a plurality of pictures photoed in panorama mode as an individual picture, respectively. Obtains azimuth information, detects the tilt of the mobile terminal, and stores the azimuth information and the tilt of the mobile terminal in association with each of a plurality of images when a shooting operation is performed with the panorama mode set. Panorama arrangement order information based on the orientation information associated with each of the plurality of images is assigned to and stored in the plurality of images, and the panorama arrangement order information is associated with each of the plurality of images. This is an image storage method in which a position to be given is determined based on an inclination.

In the fourth and fifth inventions , as in the first invention, the user can easily create a panoramic photograph that has been divided and developed by referring to the order information determined based on the orientation information. Can be rearranged.
In particular, since the user can refer to the order information given based on the tilt of the mobile terminal, for example, the user can accurately recognize the direction of the ground in the image and rearrange panoramic photos. It becomes like this.

According to the present invention, the user can easily rearrange the panoramic photographs developed by dividing into a plurality of parts by referring to the order information given to each image.

  The above object, other objects, features, and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

FIG. 1 is a block diagram showing a portable terminal of the present invention. FIG. 2 is an illustrative view showing an appearance of the portable terminal shown in FIG. FIG. 3 is an illustrative view showing one example of the inclination of the mobile terminal shown in FIG. 4 is an illustrative view showing a display example of a through image and a GUI displayed on the LCD monitor shown in FIG. FIG. 5 is an illustrative view showing an example when a certain landscape is shot as a panoramic photograph. 6 is an illustrative view showing the position of the center of gravity in the image data photographed by the portable terminal shown in FIG. FIG. 7 is an illustrative view showing one example of a panoramic photographing table stored in the RAM shown in FIG. FIG. 8 is an illustrative view showing one example of a sort table stored in the RAM shown in FIG. FIG. 9 is an illustrative view showing one example of a panoramic photograph taken in the panorama mode of the portable terminal shown in FIG. FIG. 10 is an illustrative view showing one example of a memory map of the RAM shown in FIG. FIG. 11 is a flowchart showing the panorama shooting process of the CPU shown in FIG.

  Referring to FIG. 1, portable terminal 10 includes a CPU (also referred to as a processor or a computer) 20 and a key input device 22 that is an input device. The CPU 20 controls the radio communication circuit 14 corresponding to the CDMA system and outputs a call signal. The output call signal is transmitted from the antenna 12 and transmitted to the mobile communication network including the base station. When the other party performs a response operation, a call ready state is established.

  When a call end operation is performed by the key input device 22 after shifting to a call ready state, the CPU 20 controls the wireless communication circuit 14 to transmit a call end signal to the other party. Then, after transmitting the call end signal, the CPU 20 ends the call process. Also, when the call end signal is received from the call partner first, the CPU 20 ends the call process. Further, the CPU 20 ends the call process even when a call end signal is received from the mobile communication network regardless of the call partner.

  When a call signal from a call partner is captured by the antenna 12 while the mobile terminal 10 is activated, the wireless communication circuit 14 notifies the CPU 20 of an incoming call. Further, the CPU 20 controls the LCD monitor 26 which is a display device by the LCD driver 24 and causes the LCD monitor 26 to display the caller information described in the incoming call notification. Then, the CPU 20 outputs a ring tone from an incoming call notification speaker (not shown).

  In the call ready state, the following processing is executed. The modulated audio signal (high frequency signal) sent from the other party is received by the antenna 12. The received modulated audio signal is subjected to demodulation processing and decoding processing by the wireless communication circuit 14. The obtained received voice signal is output from the speaker 18. On the other hand, the transmission voice signal captured by the microphone 16 is subjected to encoding processing and modulation processing by the wireless communication circuit 14. Then, the generated modulated audio signal is transmitted to the other party using the antenna 12 as described above.

  In addition, the mobile terminal 10 includes a tilt sensor 32 that detects the tilt of the mobile terminal 10 and an electronic compass 34 that detects the orientation. The tilt sensor 32 is a semiconductor triaxial acceleration sensor, and outputs the gravitational acceleration of each axis to the CPU 20. And CPU20 calculates the inclination of the portable terminal 10, ie, an angle, using an inverse trigonometric function with respect to the value of the gravitational acceleration of each axis. The electronic compass 34 is a three-axis electronic compass, and includes three geomagnetic sensors and a control circuit. Further, the control circuit extracts the geomagnetic data from the magnetic data detected by the three geomagnetic sensors and outputs it to the CPU 20. Then, the CPU 20 obtains an azimuth angle (azimuth) based on the geomagnetic data output from the control circuit and the angle output from the tilt sensor 32. Further, the obtained orientation coincides with the optical axis direction in which the image sensor 38 and the focus lens 40 are arranged. In this embodiment, the azimuth is 90 degrees north (N), 90 degrees south (S), 180 degrees south and 270 degrees west (W), with north (N) being 0 degrees. In addition, although the Hall element is used for each magnetic sensor, an MR (Magnet-Resistive) element or an MI (Magnet-Impedance) element may be used.

  Further, the portable terminal 10 executes the camera function by causing the camera control circuit 36 and the image sensor 38 to function as photographing means. The camera function records image data focused on the subject by autofocus processing. Specifically, when an operation for executing a camera function is performed by the key input device 22, the CPU 20 gives a command necessary for executing the camera function to the camera control circuit 36. The camera control circuit 36 controls the image sensor 38 and the focus lens 40 to convert the optical image of the show field acquired by the image sensor 38 into image data. Then, the CPU 20 converts the image data obtained from the camera control circuit 36 and focused on the subject into JPEG compressed image data and records it in the flash memory 28. In this camera function, the size of an image to be captured can be selected from WQVGA (240 × 400), VGA (640 × 480), and UXGA (1600 × 1200). Of course, the size of an image to be captured by the camera function of the present invention is not limited to these, and may be other sizes as appropriate. Further, the method for compressing the image may be a method such as GIF or PNG instead of the JPEG method.

  Further, in a state where the camera function is executed, a process of displaying a real-time moving image of the object scene (hereinafter referred to as a through image) on the LCD monitor 26 is performed. Specifically, the CPU 20 activates an image sensor driver built in the camera control circuit 36 and instructs the image sensor driver to perform an exposure operation and a charge readout operation corresponding to a designated readout region.

  The image sensor driver performs exposure of the imaging surface of the image sensor 38 and reading of electric charges generated by the exposure. As a result, a raw image signal is output from the image sensor 38. The raw image signal output from the image sensor 38 is input to the camera control circuit 36. The camera control circuit 36 performs processes such as color separation, white balance adjustment, and YUV conversion on the input raw image signal to generate image data in YUV format. The CPU 20 receives YUV format image data.

  The YUV format image input to the CPU 20 is temporarily stored in the RAM 30 by the CPU 20. Further, the stored YUV format image data is provided from the RAM 30 to the LCD driver 24 via the CPU 20. At the same time, the CPU 20 issues a thinning readout command to the LCD driver 24. Then, the LCD driver 24 outputs YUV format image data to the LCD monitor 26 in accordance with the thinning readout command issued from the CPU 20. Therefore, a low-resolution through image representing the object scene is reproduced on the LCD monitor 26.

  In YUV format image data, Y means luminance, U means a color difference obtained by subtracting luminance from blue, and V means a color difference obtained by subtracting luminance from red. That is, the YUV format image data is composed of luminance signal (Y) data, blue color difference signal (U) data, and red color difference signal (V) data.

  Note that the through image update cycle is 15 fps, and the image sensor 38 is a CMOS image sensor.

2A and 2B are illustrative views showing the appearance of the mobile terminal 10. 2A is a front external view of the mobile terminal 10, and FIG. 2B is a back external view of the mobile terminal 10.
With reference to FIG. 2A, the portable terminal 10 has a case C formed in a plate shape. A microphone 16 and a speaker 18 (not shown in FIG. 2A) are built in the case C. The opening OP2 leading to the built-in microphone 16 is provided on one surface in the length direction of the case C, and the opening OP1 leading to the built-in speaker 18 is provided on the other surface in the length direction of the case C. That is, the user listens to the sound output from the speaker 18 through the opening OP1, and inputs the sound to the microphone 16 through the opening OP2.

  The key input device 22 includes a call key, an end key, a cursor key, a confirmation key, and numeric keys 0 to 9, and each key is provided on the surface of the case C. The LCD monitor 26 is attached so that the monitor screen is exposed on the surface of the case C.

  The user performs a response operation by pressing a call key, and performs a call end operation by pressing an end call key. Further, a GUI (Graphical User Interface) is operated with a cursor key or a numeric key, and the result of the operation is determined with a confirmation key. Also, the telephone number of the other party is input using the numeric keys 0-9. Then, the power-on / off operation of the mobile terminal 10 can be performed by long-pressing the end call key 22c. Note that the confirmation key functions as a shooting key if the camera function is executed. That is, when the camera function is being executed, the user performs a shooting operation by pressing the enter key, and causes the flash memory 28 to record image data.

  2B, the image sensor 38 and the focus lens 40 (not shown in FIG. 2B) are built in the case C, and communicate with the built-in image sensor 38 and the focus lens 40. The opening OP3 is provided on the other back surface in the longitudinal direction of the case C. That is, the user can shoot an arbitrary subject by directing the opening OP3 provided on the back side of the LCD monitor 26 toward the subject. The opening OP3 is covered with a transparent plastic cover.

  2A and 2B, the inclination sensor 32 has three axes in the vertical direction (y-axis direction), the horizontal direction (x-axis direction), and the front-rear direction (z-axis direction) of the mobile terminal 10, respectively. Detect gravitational acceleration.

  Next, the inclination of the mobile terminal 10 will be described with reference to FIGS. FIG. 3A is an illustrative view showing a state in which the mobile terminal 10 is held vertically, and the angle of the mobile terminal 10 is 0 degree. FIG. 3B is an illustrative view showing a state in which the mobile terminal 10 is held tilted from the vertical direction to the left, and the angle of the mobile terminal 10 is 90 degrees. Further, FIG. 3C is an illustrative view showing a state in which the mobile terminal 10 is held in the opposite direction from the vertical orientation, and the angle of the mobile terminal 10 is 180 degrees. 3D is an illustrative view showing a state in which the mobile terminal 10 is tilted to the right from the vertical orientation, and the angle of the mobile terminal 10 is 270. FIG. In other words, the angle of the mobile terminal 10 detected by the tilt sensor 32 increases as it rotates counterclockwise with the vertical orientation being 0 degrees, and returns to 0 degrees when rotated 360 degrees.

  Here, the camera function of the present embodiment can set a mode for performing panoramic shooting (panoramic mode) separately from a mode for performing normal shooting (normal mode). In this panorama mode, a plurality of image data continuously photographed is handled as one panoramic photograph. That is, the user can perform panoramic shooting by setting the camera function of the mobile terminal 10 to the panoramic shooting mode and continuously shooting a plurality of images. Hereinafter, an operation for switching between the normal mode and the panoramic shooting mode will be described with reference to FIGS.

  FIG. 4A is an illustrative view showing a display example of the LCD monitor 26 that displays a through image photographed by the camera function. Referring to FIG. 4A, LCD monitor 26 includes a status display area 50, a function display area 52, and a key display area 54. In the status display area 50, the radio wave reception status by the antenna 12, the remaining battery capacity of the rechargeable battery, the current date and time, and the like are displayed. The current date and time displayed in the status display area 50 is acquired from the RTC 20a built in the CPU 20.

  In the function display area 52, a through image photographed by the camera function and a mode display 58 indicating the camera function mode are displayed. In the state shown in FIG. 4A, the character string “normal shooting” is displayed in the mode display 58, indicating that the camera function is in the normal mode.

  The key display area 54 includes function keys 56. When a key (for example, a right cursor key) corresponding to the function key 56 is operated, a function setting display 60 shown in FIG. 4B is displayed in the function display area 52. In this function setting display 60, “1. Mode switching”, “2. Time stamp”, “3. Self-timer”, and “4. Image size” are displayed as menus. When the same numeric key as the number indicating each menu is pressed, the menu with the number indicated by the number key is designated.

  When “1. Mode switching” is designated, the mobile terminal 10 switches to the panorama mode if the camera function is the normal mode, and switches to the normal mode if the camera function is the panorama mode. When “2. Time stamp” is designated, the mobile terminal 10 is in a state of combining a photographed image data with a numeric string indicating time. When “3. Self-timer” is designated, the portable terminal 10 records (photographs) image data in the flash memory 28 after a certain time has elapsed after the photographing operation. “4. Image size” further displays a menu screen for selecting the size of an image to be captured from WQVGA, VGA, and UXGA. That is, the user switches the size of the image data by designating a menu of “4. Image size”.

  When the numeric key “1” is pressed, as shown in FIG. 4C, the character string in the mode display 58 is switched to the character string “Panorama shooting”. When the user presses the enter key continuously with the panorama mode set, the image data is stored in the RAM 30. When the panorama mode is finished, each image data is recorded in the flash memory 28 as a panoramic picture. In addition, what is necessary is just to switch to normal mode in order to complete | finish panorama mode.

  Here, when shooting is performed in the panorama mode, a number (order information) indicating the panorama arrangement is combined with each image data. Specifically, when the image is captured in the panorama mode, the captured image data is associated with the time acquired from the RTC 20a, the angle of the mobile terminal 10 detected by the tilt sensor 32, and the orientation acquired by the electronic compass 34. And temporarily stored in the RAM 30. Each temporarily stored image data is sorted and assigned a number based on the direction and time. Further, the position where the numbers are combined is determined based on the angle of the mobile terminal 10 associated with the image data. As a result, each image data is combined with a number indicating a panoramic arrangement based on the associated azimuth and time, regardless of the order of shooting. Hereinafter, detailed processing will be described with reference to the drawings.

  First, image data captured in the panorama mode will be described with reference to FIGS. FIG. 5A is an illustrative view showing a landscape in the direction of north (N), northeast (NE), and east (E) in a certain place. FIG. 5B shows six image data obtained by shooting the landscape shown in FIG. 5A in the panorama mode. The file names of the image data are “001.jpg”, “ 002.jpg ”,“ 003.jpg ”,“ 004.jpg ”,“ 005.jpg ”, and“ 006.jpg ”.

  The image data “001.jpg” and “006.jpg” are taken in the state shown in FIG. 3B or 3D, and are referred to as landscape image data. The image data “002.jpg”, “003.jpg” and “004.jpg” are taken in the state shown in FIG. 3 (A) or FIG. 3 (C), and are referred to as portrait image data. . The image data “005.jpg” is taken with the mobile terminal 10 tilted to the right, and is referred to as right tilt image data.

  Each of the vertical image data, the horizontal image data, and the right tilt image data is associated with the azimuths of the positions shown in FIGS. Referring to FIG. 6A, in the portrait image data, since the position of the point Pa matches the optical axis direction, the orientation indicated by the position of the point Pa in the image is associated with the portrait image data. . Next, referring to FIG. 6B, in the horizontal image data, the position indicated by the position of the point Pb in the image corresponds to the horizontal image data because the position of the point Pb matches the optical axis direction. It is done. 6C, in the right tilt image data, the position indicated by the position of the point Pc in the image corresponds to the right tilt image data because the position of the point Pc matches the optical axis direction. Attached.

  Note that the coordinates indicated by the point Pa, the point Pb, and the point Pc are the barycentric coordinates in the image data. Therefore, even if the right-tilted image data is tilted at another angle, the position coordinates of the corresponding orientation in the image are There is no change.

  Although not shown, when the mobile terminal 10 is photographed in a state where the mobile terminal 10 is tilted to the left, the image data is referred to as left tilt image data. And when it does not distinguish each of right inclination image data and left inclination image data, it is called inclination image data.

  Next, a procedure for temporarily storing (temporarily storing) each of the image data “001.jpg” to “006.jpg” in the buffer of the RAM 30 will be described with reference to FIG. 5B again. FIG. 7 is an illustrative view showing the contents of a panorama shooting table showing a list of image data once stored in the buffer. This panorama shooting table is composed of columns of “time”, “file name”, “azimuth angle”, and “tilt”, and data is stored in each row in the shooting order.

  For example, when the panorama mode is set and the mobile terminal 10 faces the direction of an azimuth angle of 33.75 degrees (approximately north-northeast) at 10 o'clock and the mobile terminal 10 takes a picture shown in FIG. Data is stored in the top row. That is, “10:00:00” is stored in the time column, “001.jpg” is stored in the file name column, “33.75” is stored in the azimuth column, and “90.00” is stored in the tilt column. The horizontal image data of “001.jpg” is stored in this buffer.

  Next, when the mobile terminal 10 faces the direction of 60.00 degrees at 10: 5 and shoots in the state shown in FIG. 3A, data is stored in the second row of the panorama shooting table. . That is, “10:00:05” is stored in the time column, “002.jpg” is stored in the file name column, “60.00” is stored in the azimuth column, and “0.00” is stored in the tilt column. The horizontal image data of “002.jpg” is further stored in the buffer of the RAM 30.

  Then, the shooting operation is continued three times while changing the shooting direction of the user and the inclination of the mobile terminal 10, and shooting is performed at the same direction and the same tilt as the first shot at 10:50 seconds. Data is stored in the row of the tier (the last row). That is, “10:00:00” is stored in the time column, “001.jpg” is stored in the file name column, “33.75” is stored in the azimuth column, and “90.00” is stored in the tilt column. The horizontal image data “006.jpg” is further stored in this buffer.

  Subsequently, when the panorama mode is ended, each captured image data is sorted based on the columns of “azimuth” and “time” in the panorama shooting table. Referring to FIG. 7, the “azimuth” column includes “33.75”, “60.00”, “116.25”, “7.50”, “88.13”, and “33.75”. The azimuth angles are stored in the order of "." Therefore, the direction of the “azimuth” column is “7.50”, “33.75”, “33.75”, “60.00”, “88.13”, and “116.25”. Sorted in ascending order of corners. Further, since there are two pieces of image data associated with the azimuth angle of “33.75”, the image data is further sorted based on the “time” column. That is, in the time column, the time is sorted in the order of early so that “10:00: 00” and “10:00: 50” are in this order. As a result, when there are a plurality of image data associated with the same orientation, a number (order information) is synthesized based on the time when the image was taken. The sorted result is stored in the sort table shown in FIG.

  The sort table shown in FIG. 8 shows the result of sorting the contents of the panorama shooting table shown in FIG. 7 based on the azimuth and time. This sort table is composed of columns of “azimuth”, “time”, “file name”, and “tilt”. In the top row, the file name and inclination of the image data associated with the smallest azimuth angle are stored.

  That is, in the top row, “7.50” is stored in the azimuth column, “10:00:30” in the corresponding time column, and “004.jpg” in the file name column. In addition, “0.00” is stored in each of the columns of inclination and inclination. In the second row, “33.75” is stored in the azimuth column, “10:00:00” in the corresponding time column, and “001.jpg” in the file name column. In addition, “90.00” is stored in the column of the inclination. In the third row, “33.75” is stored in the azimuth column, “10:00:50” in the corresponding time column, and “006.jpg” in the file name column. In addition, “90.00” is stored in the column of the inclination. In the next fourth row, “60.00” is stored in the azimuth column, “10:00:05” in the corresponding time column, and “002.jpg” in the file name column. In the slope column, “0.00” is stored. In the next fifth row, “88.13” is stored in the azimuth column, “10:00:35” in the corresponding time column, and “005.jpg” in the file name column. In addition, “335.00” is stored in the column of the inclination. In the sixth row (last row), “116.25” is stored in the azimuth column, “10:00:10” in the corresponding time column, and “10:00:10” in the file name column. “0.00” is stored in each of “003.jpg” and the slope column.

  Based on the sort table, numbers are combined with the image data stored in the buffer provided in the RAM 30, as shown in FIG.

  Referring to FIG. 9, numbers (1) to (6) are respectively synthesized on the upper left in the image data “001.jpg” to “006.jpg”. That is, the number (1) is combined with the vertical image data “004.jpg”, the number (2) is combined with the horizontal image data “001.jpg”, and the horizontal direction “006.jpg” is displayed. The number of (3) is combined with the image data, the number of (4) is combined with the vertically oriented image data of “002.jpg”, and the right inclined image data of “005.jpg” is combined with the number of (5). The numbers are combined, and the number (6) is combined with the vertically oriented image data “003.jpg”.

  Also, the direction and position where the numbers are combined is determined from the inclination associated with each image data. First, the direction of the number to be combined is set to be perpendicular to the ground in the captured scene. Specifically, if the inclination associated with the image data is 90 degrees, it can be understood that the image data was photographed with an inclination of 90 degrees to the left with respect to the ground, and therefore the direction perpendicular to the ground can be obtained. . As a result, the user can accurately recognize the direction of the ground in the image data based on the synthesized number, and can arrange the panorama.

  Of course, in the present invention, the direction of the number to be combined is not limited to the direction perpendicular to the ground in the photographed scene as described above. An appropriate direction such as a horizontal direction can be taken.

  Next, the position of the number to be combined is set to the upper left of the position farthest from the ground in the image. Specifically, the image data is rotated on the virtual plane so that the direction of the ground is downward, and the position of the number to be combined is determined based on the coordinates indicating the upper left.

  Here, the position where the numbers are combined is set on the upper side, since the subject (such as a mountain) required by the photographer in the panoramic photograph is included in the lower part, and the upper part (such as the sky) of the subject is not important for the photographer. This is because there are many cases. Further, the left side is the position starting from the raster scan of the image, so that the process of determining the position by the raster scan is accelerated. For this reason, in this embodiment, the position of the number to be combined is set to the upper left. As a result, in the panoramic photograph taken by the mobile terminal 10, the number can be synthesized at a position that does not get in the way for the user. Only when the raster scan starts from the upper right end, the position of the number to be combined may be the upper right end. Furthermore, in another embodiment, the position where the order information is combined may be arbitrarily determined by the user.

  “001.jpg”, “004.jpg”, “005.jpg”, and “006.jpg” indicate eight directions (north, northeast, east, southeast, south, southwest, west, northwest). Is further synthesized. For example, “NE” meaning northeast is combined with the image data “001.jpg” and “006.jpg” respectively, and “N” meaning north is added to the image data “004.jpg”. The combined image data “004.jpg” is combined with “E”, which means east. Further, an auxiliary line is added to the direction information to be synthesized so that the ground direction can be recognized more clearly. Accordingly, when the user takes a plurality of panoramic photos at the same place, the user can distinguish a subject (such as a mountain) that appears in each panoramic photo from the direction combined with each panoramic photo.

  Here, when compositing azimuth information, the angle of view is calculated from the distance between the image sensor 38 and the focus lens 40 on the optical axis, and the target (direction) indicated by the optical axis and the calculated angle of view are calculated. It is determined whether 8 directions are included in the image data. Then, when 8 orientations are included in the image data, an alphabet such as “N” is synthesized. Also, in the panorama mode in this embodiment, the angle of view is not changed every time the image is taken, and therefore the angle of view data is not stored in the panorama shooting table.

  Note that the direction information is synthesized on the lower side of the image data, but it can be determined by the user whether to synthesize. That is, if the user determines that the direction information to be combined is an obstacle, the user can prevent the direction information from being combined. Further, in FIG. 9, the drawing is greatly drawn for easy recognition of the combined direction information, but the combined direction information, that is, the size of the alphabet may be reduced so as not to stand out. .

  FIG. 10 is an illustrative view showing a memory map of the RAM 30. Referring to FIG. 10, a memory map 300 of RAM 30 includes a program storage area 302 and a data storage area 304. A part of the program and data is read from the flash memory 28 all at once or partially and sequentially as necessary, stored in the RAM 30, and then processed by the CPU 20 or the like.

  The program storage area 302 stores a program for operating the mobile terminal 10. A program for operating the mobile terminal 10 includes a camera function program 310 and the like. The camera function program 310 is a program for executing a camera function, and further includes a panorama shooting program 312. The panorama shooting program 312 is a program for combining order information and azimuth information with a plurality of image data shot in the panorama mode.

  In addition, although illustration is abbreviate | omitted, the program for operating the portable terminal 10 contains the program for performing a telephone call, etc.

  In the data storage area 304, a through image buffer 320, a panorama shooting buffer 322, and a sort buffer 324 are provided. In addition, number image data 326 and direction image data 328 are stored.

  The through image buffer 320 is a buffer in which YUV format image data read from the camera control circuit 36 is temporarily stored (stored once). The panorama shooting buffer 322 is a buffer that temporarily stores each of the images shot in the panorama mode, and further stores the panorama shooting table shown in FIG. 7 temporarily. The sort buffer 324 is a buffer used when the panorama shooting table is sorted, and temporarily stores the sort table shown in FIG. The number image data 326 is image data indicating the numbers (1) to (6) (including (6) and above) combined with the image data. The azimuth image data is azimuth information combined with the image data, that is, image data indicating alphabets such as “N”, “NE”, and “E”.

  Although illustration is omitted, the GUI image data for displaying the GUI shown in FIGS. 4A to 4C is stored in the data storage area 304, and other data necessary for the operation of the mobile terminal 10 is stored. Counters and flags are also provided.

  The CPU 20 executes a plurality of tasks including a panorama shooting process shown in FIG. 11 in parallel under the control of an RTOS (real-time operating system) such as “Linux” and “REX”.

  FIG. 11 is a flowchart showing the panorama shooting process. For example, when the user performs an operation for setting the panorama mode, the CPU 20 determines whether or not the operation is a shooting operation in step S1. That is, it is determined whether or not the confirmation key has been pressed. If “NO” in the step S1, that is, if the photographing operation is not performed, the process of the step S1 is repeatedly executed. On the other hand, if “YES” in the step S1, that is, if the confirmation key is pressed, the orientation data is acquired from the electronic compass 34 in a step S3. That is, the azimuth corresponding to the optical axis direction when the image is taken is acquired.

  Subsequently, in step S5, an angle is calculated from the tilt sensor 32. That is, the angle of the mobile terminal 10 is calculated using an inverse trigonometric function with respect to the gravitational acceleration of each axis output from the tilt sensor 32 in accordance with the shooting. Subsequently, in step S7, the current time is acquired. That is, the current time is acquired from the RTC 20a. Subsequently, in step S9, the image data and the corresponding time, azimuth data, and angle are stored. That is, image data is temporarily stored in the panorama shooting buffer 322, and the time, data name, azimuth angle, and inclination (angle) corresponding to the image data are stored in the panorama shooting table. Subsequently, in step S11, it is determined whether or not an end operation is performed. That is, it is determined whether or not an operation for ending the panorama mode has been performed.

  If “NO” in the step S11, that is, if the panorama mode is not ended, the process returns to the step S1. On the other hand, if “YES” in the step S11, that is, if the panorama mode is ended, the image data in the panorama shooting buffer 322 is sorted in a step S13. In other words, each data recorded in the panorama shooting table is sorted based on the azimuth and time, and the sort table is created in the sort buffer 324.

  Subsequently, in step S15, a synthesis position is determined based on the angle. That is, the image data is rotated on the virtual plane so that the direction of the ground is downward, and the coordinates indicating the upper left are obtained. Subsequently, in step S17, a number is combined with each image data based on the sorting result and the angle. That is, in each image data, the vertical direction with respect to the ground is obtained, and numbers (order information) are synthesized based on the created sort table. Subsequently, in step S19, the orientation is combined with the image data based on the angle. That is, as in the process of step S17, the vertical direction with respect to the ground is obtained, and if the object field includes eight directions, the direction information is synthesized.

  Subsequently, in step S21, each image data is recorded, and the panorama shooting process is ended. That is, in step S21, each of the image data in which the number and direction information are combined is recorded in the flash memory 28 as a panoramic photograph. That is, the user can save each image data photographed individually as a panoramic photograph.

  In the panorama mode, each of the image data in which the order information and the direction information are combined and the image data in which the order information and the direction information are not combined may be recorded.

  As can be understood from the above description, the mobile terminal 10 includes the camera control circuit 36, the image sensor 38, and the focus lens 40, and the user can take image data with the mobile terminal 10. The mobile terminal 10 has a built-in electronic compass 34 and can acquire the orientation indicated by the optical axis direction of the image sensor 38 and the focus lens 40. For example, when the panorama mode is set, each shot image data is temporarily stored with an azimuth angle or the like associated therewith.

  When the panorama mode is finished, each image data is combined with a number indicating the order of panorama arrangement based on the azimuth angle associated with each image data.

  As a result, the user can easily rearrange the panoramic photographs that have been divided and developed with reference to the order information determined based on the azimuth angle.

  The image sensor 38 may be a CCD image sensor instead of a CMOS image sensor. Further, the order information is not limited to a number, and may be a character string indicating an order such as “abc” or “left middle right up / down”. That is, as the order information in the present invention, information that allows the user to recognize the order may be used. Therefore, as the order information, appropriate information such as numbers, symbols, symbols, characters and other information, or any combination thereof may be used. Can be used. Furthermore, as shown in FIG. 9, in addition to enclosing numbers with parentheses, such parentheses () need not be used, and other parentheses such as parentheses [] can also be used. In addition, the information of the direction to be combined is a combination of 16 directions including 8 directions, north-northeast, east-northeast, east-southeast, south-southeast, south-southwest, west-southwest, west-northwest and north-northwest, and other appropriate types of orientation information. It is good also as the information of the direction synthesize | combined.

  Further, the communication method of the mobile terminal 10 is not limited to the CDMA method, but may be a W-CDMA method, a TDMA method, a PHS method, a GSM method, or the like. Not only the portable terminal 10 but also a portable information terminal such as a PDA (Personal Digital Assistant) having a camera function may be used.

10 ... Mobile terminal 20 ... CPU
22 ... Key input device 26 ... LCD monitor 28 ... Flash memory 30 ... RAM
32 ... Tilt sensor 34 ... Electronic compass 36 ... Camera control circuit 38 ... Image sensor 40 ... Focus lens

Claims (5)

A mobile terminal that stores a plurality of images shot in panorama mode as individual images,
Photographing means for photographing an image in accordance with a photographing operation;
Orientation information acquisition means for acquiring a rectangular position information,
Inclination detecting means for detecting the inclination of the portable terminal;
When the panorama mode is the photographing operation in a state that has been set, square position information and the inclination of the portable terminal, a first storage means for storing in association with each of the plurality of images, and
Order information based rather panoramic disposed orientation information associated with each of the plurality of images, a second storage means for storing given to the plurality of images,
The sequence information of the panoramic arrangement is Ru determined position granted under the slope associated with each of the plurality of images, the portable terminal. Further comprising a time acquisition means for acquiring at time,
The first storage means stores the time acquired by the time acquisition means in association with each of a plurality of images,
Said second storage means, if the same orientation data is at least two images associated with the sequence information of the panoramic arranged based on direction information and the time associated with each of the plurality of images determining, according to claim 1 Symbol placement of the portable terminal. It said second storage means, you store imparted information indicating the orientation in the image, according to claim 1 or 2 mobile terminal according. A processor of a portable terminal that has a photographing means for photographing an image according to a photographing operation and stores a plurality of images photographed in the panoramic mode as individual images ;
Orientation information acquisition means for acquiring a rectangular position information,
Inclination detecting means for detecting the inclination of the portable terminal;
When the panorama mode is the photographing operation in a state that has been set, square position information and the inclination of the portable terminal, a first storage means for storing in association with each of the plurality of images, and
Wherein the plurality of order information based rather panoramic disposed orientation information associated with each image, to function as a second storage means for storing given to the plurality of images,
The sequence information of the panoramic arrangement, Ru position granted under the slope associated with each of the plurality of images are determined, an image storage program. An image storage method for a mobile terminal, which has an image capturing unit that captures an image according to a shooting operation, and stores a plurality of images captured in a panoramic mode as individual images,
Get direction information,
Detecting the tilt of the mobile device;
When a shooting operation is performed with the panorama mode set, the orientation information and the inclination of the mobile terminal are stored in association with each of a plurality of images, and
Panoramic arrangement order information based on the orientation information associated with each of the plurality of images is assigned to the plurality of images and stored;
The panorama arrangement order information is an image storage method in which a position to be given is determined based on an inclination associated with each of a plurality of images.

Images