JP2005071332A - Portable terminal equipment and image display method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide portable terminal equipment that improves convenience of viewing and searching of image information by a user, displaying more image information arranged for easy understanding even if a small-sized screen is used, by using three dimensional computer graphics technology. <P>SOLUTION: The portable terminal equipment 100 includes an object section 100a for generating and storing various objects constructing three dimensional objects, a database section 100b for storing information to be displayed with the three dimensional objects, a key input section 100c for executing input processing according to input keys such as cursor keys, a drawing section 100d for drawing the various objects transferred from the object section 100a based on arrangement information, and a display section 100e for generating and displaying an image to be displayed on a screen. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a portable terminal device such as a mobile phone or a PDA that displays a digital image or a digital moving image on a screen, and more particularly to a portable terminal device that generates an object using a three-dimensional display technology and displays the object on a screen. .

  With the spread of digital cameras and video cameras in recent years, the number of digital images and digital images possessed by individuals has increased, and most current mobile phones are equipped with a digital camera function. Stores a large number of images taken by the user. Under such circumstances, it has become an issue how to use and select a large amount of digital images that are photographed by the user and stored in the terminal device.

  For example, digital images can be used by collecting a series of related images as an album, deforming a photographed person's face, or combining another border with a digital image such as a so-called photo book. However, even in the use of such digital image applications, it is necessary to select a digital image to be used from many digital images.

  Conventionally, in a PC, a mobile phone, a PDA, a camera with a monitor, etc., in order to view and select a favorite photo from a large number of digital images, a thumbnail display in which a plurality of digital images are reduced and arranged on the screen The law is common.

As a display method in which the thumbnail display method is extended to three dimensions, a three-dimensional display processing apparatus and method and a medium on which a control program is recorded are disclosed (for example, see Patent Document 1). In this three-dimensional display device, by arranging a lot of input data having evaluation values as display images that can be displayed as a list in the three-dimensional space, the user can walk through the three-dimensional space and compare the input data. Make it possible.
Japanese Patent Laid-Open No. 11-231993

  However, in the conventional thumbnail display method described above, when a large amount of digital images are displayed on a small screen of a mobile terminal device, the size of the display screen is limited and the display area of the digital image increases. The user must scroll and switch the screen frequently. Therefore, the thumbnail display method used for a small display screen of a mobile terminal device or the like has a problem that the user operation becomes complicated.

  Further, in the conventional thumbnail display method, thumbnail images are reduced and arranged in the order of normal shooting, so that it is difficult for the user to determine the relationship between a large number of images and the time relationship such as date and time. Further, the user needs to switch the display screen in order to search for information related to the image. As described above, in the thumbnail display, there is a problem that it is difficult for the user to find a variety of complicated information for each large amount of digital images and it is difficult to search for a favorite image.

  In view of the above-mentioned problems, the present invention displays information related to more images in an easy-to-understand manner even when the screen size is small on a small screen included in a mobile terminal device, so that the user can search for images, browse information, etc. The first object is to provide a portable terminal device with improved convenience.

  A second object of the present invention is to provide a mobile terminal device that displays various related information of an image together with the image in an easy-to-understand manner and allows the user to select a favorite image without switching the screen. It is a third object of the present invention to provide a mobile terminal device that displays a picture visually and beautifully on the screen of the mobile terminal device and performs a display method with enhanced entertainment in user operations.

  In order to solve the above-described problem, a mobile terminal device according to the present invention includes an object generation unit that generates an object, a texture generation unit that generates a texture image for each content of related information that is information related to a digital image, Based on the related information, the digital image and the texture image generated by the texture generating unit are texture-mapped to the object generated by the object generating unit to create an image object, and the plurality of image objects are based on the related information. Block generating means for generating block objects arranged in a three-dimensional space, image generating means for generating images of the block objects, and display means for displaying images generated by the image generating means. It is characterized by that.

  Further, the object generation means of the mobile terminal device according to the present invention generates a three-dimensional object, and the provision means provides the digital image in front of the three-dimensional object and the related information as a side surface of the three-dimensional object. The block generation means generates a block object by arranging a plurality of the image objects in a three-dimensional space so as to form a polyhedron based on the related information. The image generation means generates an image of the block object, and the display means displays the image.

  Therefore, a 3D object can be created on the screen of the mobile terminal device using 3D computer graphics technology, and various images and various information included in the images can be easily displayed on a small screen of the mobile terminal. Two pieces of related information can be referred to, and the search efficiency of a digital image by a user can be improved using more information.

  Furthermore, the object generation means of the mobile terminal device according to the present invention generates a two-dimensional object, and the assigning means places the first texture image in front of the two-dimensional object and the second texture image as the two-dimensional object. An image object is created by pasting a part or all of the outer periphery of the object, and the block generation unit can recognize at least a part of the image object located behind based on the related information. A plurality of two-dimensional image objects are arranged obliquely in a three-dimensional space to generate a block object, the image generation unit generates an image of the block object, and the display unit displays the image It is characterized by displaying.

  For this reason, since the user can confirm the display contents of other image objects located in the depth direction in addition to the frontmost two-dimensional image object, the convenience in image selection can be further improved. Further, since related information is given to the image object, the user can select an image more easily.

  The present invention can be realized not only as a portable terminal device as described above, but also as an image display method using steps included in the portable terminal device as a step, and as a program for realizing the image display method on a computer or the like. can do. Needless to say, the program can be distributed via a recording medium such as a CD-ROM or a transmission medium such as a communication network.

  As described above, the user of the portable terminal device according to the present invention creates a three-dimensional object on the screen by using the three-dimensional computer graphics technology, and thereby displays a plurality of images and various information included in the images on a small screen of the portable terminal. However, it becomes possible to display it in an easy-to-understand manner. That is, by displaying a photograph using a three-dimensional block object, it is possible to refer to at least two pieces of related information simultaneously with the image information. Can be increased. In addition, since colorful block objects are displayed on the screen together with the speech bubble display and the like, the image can be displayed beautifully on the screen, and the entertainment of the user of the mobile terminal device can be enhanced.

  Then, the user of the mobile terminal device can search while confirming the contents of the image object positioned in the depth direction by using a two-dimensional image object, and can select a preferred method of arranging the image object from a plurality of arrangement methods. The display method of the image object can be diversified.

Hereinafter, a mobile terminal device according to an embodiment of the present invention will be described with reference to the drawings. The mobile terminal device according to the present invention is, for example, a mobile phone having a small screen, a PDA, a car navigation device, or a digital camera machine.
(Embodiment 1)
FIG. 1 is a block diagram illustrating an example of a functional configuration of the mobile terminal device 100 according to the first embodiment. The mobile terminal device 100 includes an object unit 100a, a database unit 100b, a key input unit 100c, a drawing unit 100d, and a display unit 100e. Details will be described below.

  The mobile terminal device 100 according to the first embodiment has a small screen for displaying a digital image, a digital moving image, and related information associated with the image or moving image, and is not a thumbnail display method. A feature is that a large amount of images are handled as block objects arranged in the depth direction based on related information of the images using a three-dimensional CG technique.

  An object unit 100a illustrated in FIG. 1 is a management unit that generates and stores various objects constituting an image object, and includes an object management unit 200, an object generation unit 210, a texture generation unit 220, a model generation unit 230, and an object. The storage unit 240 is configured.

  The texture generation unit 220 is related information related to each image passed from the data table of the object management unit 200 via the object generation unit 210, for example, “genre”, “priority”, “shooting time”, “reference count”. Etc. are combined with font image data stored in advance in advance to generate texture images that are color-coded, shape-divided, patterned, or gradated. The texture image also includes character information for displaying help information of the mobile terminal device.

  The model generation unit 230 receives an instruction from the object generation unit 210 and generates an object model to which the texture image generated in the texture generation unit 220 is pasted. In addition, a texture object photographed on the object model is pasted to generate an image object, and a block object displayed on the screen is generated by arranging a plurality of image objects in a three-dimensional space.

  An object model is a polygon model having three-dimensional coordinates. This polygon model has vertex coordinates in a three-dimensional space of four vertices and texture coordinates corresponding to each vertex. The polygon model is not limited to a plate-like polygon model having four vertices, but may be a primitive or polygonal object such as a sphere or a rectangular parallelepiped.

  The object generation unit 210 pastes the texture image generated by the texture generation unit 220 to the object model generated by the model generation unit 230 and generates an image object in which related information is pasted on the side surface. A block object is generated by arranging the image objects in the depth direction so as to form a cube.

  The object storage unit 240 stores block objects, image objects, and the like generated by the object generation unit 210 in accordance with instructions from the object management unit 200.

  The object management unit 200 instructs the object generation unit 210 to generate various objects necessary for scene generation in accordance with instructions from the drawing control unit 600, and also transmits data for each object such as an image object and a block object to the information management unit 101. Request creation of a table.

  The database unit 100b illustrated in FIG. 1 stores various information related to images displayed on the three-dimensional object, and includes an information management unit 101, an image storage unit 110, an information storage unit 120, an image processing unit 130, and an information input unit 140. These five processing units are included.

  The information management unit 101 manages information stored in the image storage unit 110 and the information storage unit 120 using the image ID and the related information ID. The information management unit 101 creates a data table in which information stored in each of the storage units 110 and 120 is written according to an instruction from the object management unit 200 and passes the data table to the object management unit 200.

  The image storage unit 110 is a hard disk that stores an entity of a digital image displayed on the screen, and may be a memory card such as an SD card in a mobile phone. The image storage unit 110 manages each digital image using the image ID, and transmits the selected digital image according to an instruction from the information management unit 101. Note that the mobile terminal device 100 capable of shooting a moving image can record a digital moving image in the image storage unit 110.

  The information storage unit 120 stores related information associated with an image. Examples of the related information include, for example, information automatically provided at the time of shooting in the format of the Exif format, which is a standard of a digital camera, the mobile terminal device manufacturer, model, focal length, image creation time, and shooting time. When displaying a date and a movie, there are recording time, image size, brightness, and hue. In addition, information extracted from the image by the image processing unit 130 and information input by the user via the information input unit 140, such as various shooting contents such as a shooting location, a favorite level, a priority, a genre, a reference count, and the like. From information and images taken with a camera with GPS, it is information on the shooting location. The information storage unit 120 manages and stores related information, which is unique information for each digital image, using an ID associated with an image ID.

  The image processing unit 130 extracts feature information from the image. Examples of feature information extracted from this image include the size of the human area included in the image obtained from the area of the skin color area included in the image, and identification of “wife” or “child” using face recognition technology. Information such as whether or not a person is included, the appearance frequency of a specific person, and information on a natural landscape with a lot of green are extracted from information on hue. In addition, information such as day and night, in the room, and information such as fine rain are also extracted using the luminance information included in the image. Then, the image processing unit 130 records the feature information extracted from the image in the information storage unit 120 as related information.

  The information input unit 140 is a processing unit for inputting, updating, and the like related information related to an image directly input by a user and stored in the information storage unit 120. Input information includes “favorite degree”, “photographing place”, “photographer”, and the like. The newly input information is passed to the information storage unit 120 as related information via the information management unit 101 in association with the image ID.

The key input unit 100c includes an input unit such as an input button provided on the mobile terminal device 100 for user operation and a control unit.
The cursor key input unit 300 is composed of four-way keys provided in the mobile terminal device 100, and is an operation button generally called a cross key. The cursor key control unit 310 transmits control of the cursor position on the screen to the event control unit 400 according to the input of the cursor key input unit 300 by the user of the mobile terminal device 100.

  Here, how to obtain the coordinates at which the cursor object is arranged will be described. In response to an input from the user of the mobile terminal device 100, the cursor key input unit 300 has a one-to-one correspondence with the cursor control unit 310 on each of the upper, lower, left, and right keys. Send a key code that is the identifier of the key.

  The cursor control unit 310 passes to the event control unit 400 which direction the key code has been input. In the present invention, since the direction in which the cursor moves in the three-dimensional space differs depending on the display mode key code input for each display mode, the event control unit 400 includes a display mode set by the mode control unit 370, The correspondence between the cursor's up / down / left / right direction and the direction in the three-dimensional space is stored in advance as a data table. Then, the event control unit 400 passes the movement direction of the cursor in the three-dimensional space to the drawing control unit 600 according to the data table. For example, in the block display mode, the top, bottom, left, and right of the cursor mean movement in the y-axis negative direction, the y-axis positive direction, the x-axis negative direction, and the x-axis positive direction, respectively, in a three-dimensional space.

  Here, the arrangement method of the cursor object will be described. The drawing control unit 600 determines which object is selected from the object arrangement information and the moving direction of the cursor, and the ID of the object selected by the cursor. To the scene generation unit 610. The scene generation unit 610 obtains the coordinates of the cursor from the coordinates of the selected object, and arranges the cursor object.

  The enter key input unit 320 is an operation button used when the user of the mobile terminal device 100 determines information from a plurality of display information, and the user of the mobile terminal device 100 selects a desired image object from among a plurality of image objects. Used to determine

  The determination key control unit 330 passes the state of the determination key to the event control unit 400 from the key code of the determination key input by the determination key input unit 320. For example, when a specific image is determined by the determination key input unit 320, the event control unit 400 passes the selected image to the information output unit 500.

The cancel key input unit 340 is an operation button for canceling information once selected in accordance with a user input.
The cancel key control unit 350 passes the cancel key state to the event control unit 400 from the key code of the cancel key input by the cancel key input unit 340. The event control unit 400 performs display before cancellation is performed based on information from each control unit. For example, the mode control unit 370 holds the history of the previously selected display mode, and the viewpoint control unit 390 holds the history of the previously selected viewpoint position. The cancel key input unit 340 is used to select the previously selected display mode history. You can return to the display mode or viewpoint position.

  The mode selection unit 360 is an input unit for the user of the mobile terminal device 100 to select a display mode according to the present invention, and the mode control unit 370 selects the display mode selected by the mode selection unit 360 from the event control unit 400. To tell.

  The viewpoint changing unit 380 is a zoom-up key and zoom-down key for zooming, scrolling, and rotating the image displayed on the screen in order for the user of the mobile terminal device 100 to change the viewpoint of the object displayed on the screen. This is an operation button composed of nine key input units, a key, an up / down / left / right scroll key, an x-axis rotation key, a y-axis rotation key, and a z-axis rotation key.

  The viewpoint control unit 390 obtains viewpoint coordinates by receiving a key code that is an identifier corresponding to each key, sent from the viewpoint changing unit 380, and sends the viewpoint coordinates to the scene generation unit 610 via the drawing control unit 600. Pass the coordinates. Also, the event control unit 400 is notified of the viewpoint change.

The drawing unit 100d is a processing unit that draws the object passed from the object management unit 200 based on the arrangement information.
The drawing control unit 600 receives a display mode instruction from the event control unit 400. The object management unit 200 is instructed to generate an object necessary for the selected display mode, and receives the object generated from the object management unit 200. In addition, when an instruction to change the viewpoint such as zooming up or down is received from the viewpoint control unit 370, the scene generation unit 610 is instructed to generate an image whose viewpoint has been changed.

  The display unit 100 e is a processing unit that generates and displays an image displayed on the screen of the mobile terminal device 100, and includes a scene generation unit 610, an image generation unit 620, and a display unit 630.

  The scene generation unit 610 generates a block object by arranging the generated image object in the three-dimensional space according to the arrangement information stored in the arrangement information storage unit 640 in accordance with an instruction from the drawing control unit 600 and displaying the block object. Other objects are also arranged based on the mode.

  When the scene generation unit 610 finishes arranging all objects, the image generation unit 620 calculates how the 3D image looks from the viewpoint coordinates selected by the user via the viewpoint change unit 360, The result is output to the display unit 630 as image information. For example, when the thumbnail display mode is selected, the drawing control unit 600 sets the viewpoint to the initial position of the viewpoint corresponding to the display mode on a one-to-one basis.

The display unit 630 performs processing for displaying the image generated by the image generation unit 620 on the screen of the mobile terminal device 100.
The event control unit 400 receives instructions from the control units 310 and the like and performs instructions to the control units 310 and the like in order to execute operations such as display mode switching requested by the user to the mobile terminal device 100. .

  The information output unit 500 is a processing unit for outputting information to an external device, and other devices such as a mail creation device provided in the mobile terminal device 100 in accordance with an instruction from the event control unit 400 such as mail transmission with an image attached. Output images and related information. As an example of an external device, a mail creation device for creating a mail addressed to a personal information email address, a telephone transmission device for making a call to a personal information telephone number, a personal information address, and information editing There are an editing device, a printing device for printing an image, an external storage device, and the like.

  FIG. 2 is a reference diagram of screen display of the mobile terminal device 100 according to the first embodiment. The screen 201 of the mobile terminal device 100 according to the present invention displays a thumbnail display mode in which a speech bubble is displayed in accordance with the position of the cursor.

  In the thumbnail display mode, an image object 301, a block object 401, a cursor object 501, a frame cursor object 601, an axis object 701, and a speech bubble object 801 are arranged on the screen based on the arrangement information. Then, by displaying the block object on the screen 201, the user can refer to related information in addition to the image information.

  Next, each object generated in the object generation unit 210 of the mobile terminal device 100 according to the first embodiment will be described. There are six types of objects used in the present invention: an image object 301, a block object 401, a cursor object 501, a frame cursor object 601, an axis object 701, and a speech bubble object 801. The objects used in each display mode are different.

FIG. 3 is a reference diagram and a data table 302 of the image object 301 displayed on the mobile terminal device 100 according to the first embodiment.
An image object 301 shown in FIG. 3A is an object that visualizes an image and related information related to the image, and is arranged in a three-dimensional space with a two-dimensional texture image stored in the image storage unit 110. And a polygon model having a three-dimensional coordinate generated in the model generation unit 230 for drawing.

  In the data table 302 shown in FIG. 3B, information associated with the image object 301 is described. Specifically, the image data ID stored in the image storage unit 110 is generated by the model generation unit 230. Polygon model ID, image object width, height, image size, date taken, color depth, Exif tag, user-defined tag, and the like. The Exif tag is a standard of a digital camera, and is information automatically given at the time of shooting, for example, the manufacturer, model, focal length, etc. of the mobile terminal device. The user-defined tag is information input via the information input unit 140 and information extracted by the image processing unit 130, and indicates information such as position information and favorite degree.

FIG. 4 is a reference diagram and a data table 402 of the block object 401 displayed on the mobile terminal device 100 according to the first embodiment.
A block object 401 shown in FIG. 4A is a three-dimensional object in which a plurality of image objects are arranged in a rectangular parallelepiped shape, and related information for each image object 301 included in the block object 401 is visualized on the side surface. Paste as texture. The image objects 301 included in the block object 401 have a series of album-like relationships with each other.

  The data table 402 shown in FIG. 4B includes a frame ID indicating the title ID such as the album name of the block object 401, each image object ID, information ID of information given to each surface, the number of images, and the number of image objects. An ID, a user-defined tag such as a shooting date of each image object, travel location information, the priority of each image object, an ID array of the image object, a polygon model ID, and the like are described.

  The image displayed in front of the block object 401 is, for example, a representative image in the image such as “group photo at Kyoto travel” in the Kyoto travel album, or any information held by the block object 401. It is also conceivable to map the image with the highest value or the lowest value when sorted, or the image selected by the user as a texture image on the front of the block object 401.

FIG. 5 is a reference diagram and a data table 502 of the cursor object 501 displayed on the mobile terminal device 100 according to the first embodiment.
A cursor object 501 shown in FIG. 5A is an arrow or the like displayed on the screen. For example, the cursor object 501 is used when the user of the mobile terminal device 100 selects the image object 301 from the block object 401 in the thumbnail display mode. It is operated via the cursor key input unit 300.

In the data table 502 shown in FIG. 5B, the ID, the ID of the instructed object, and the polygon model ID are described.
FIG. 6 is a reference diagram and a data table 602 of the frame cursor object 601 displayed on the mobile terminal device 100 according to the first embodiment.

  A frame cursor object 601 shown in FIG. 6A is displayed to more clearly indicate the position of the image object 301 indicated by the cursor object 501 in the block object 401 in the thumbnail display mode or the like.

  In the data table 602 shown in FIG. 6B, an ID, an instructed frame ID, a block object ID to which it belongs, a polygon model ID, and the like are described.

FIG. 7 is a reference diagram and a data table 702 of the axis object 701 displayed on the mobile terminal device 100 according to the first embodiment.
An axis object 701 shown in FIG. 7A displays time information and the like of the block object 401 in the thumbnail display mode. Also, in the data table 702 shown in FIG. 7B, the ID, the content of information indicated by the axis object 701, the block object ID to which it belongs, the polygon model ID, and the like are described.

FIG. 8 is a reference diagram and a data table 802 of the speech bubble object 801 displayed on the mobile terminal device 100 according to the first embodiment.
A speech bubble object 801 shown in FIG. 8A displays a thumbnail of the image object 301 at the position of the cursor object 501 in the thumbnail display mode together with the frame, and related information of the image object 301 such as a date, for example, together with the thumbnail display. Information can be displayed.

  In the data table 802 shown in FIG. 8B, an instructed frame ID, a block object ID to which the speech bubble belongs, an image ID for thumbnail display, a polygon model ID, and the like are described.

  FIG. 9 is an explanatory diagram showing the relationship between the display modes of the mobile terminal device 100 according to the first embodiment. Types of display modes of objects displayed on the mobile terminal device 100 include a display information selection mode 901, an information input mode 902, a block display mode 903, a thumbnail display mode 904, and an image display mode 905.

  The user selects a display mode in the mode selection unit 360 and shifts to another display mode by performing key input in each display mode. For example, when the user selects one block object 401 from a plurality of block objects 401 in the block display mode 903 using the enter key input unit 320, the thumbnail display mode 904 is automatically entered. When the user selects the image object 301 in the thumbnail display mode 904 using the enter key input unit 320, the image display mode 905 for displaying the image is displayed.

  Then, in order to give the related information of the image selected by the user, for example, in the image display mode 905, the mode selection unit 360 can be used to shift to the information input mode 902. In addition, the history of the selected mode is held in the mode control unit 370, and it is possible to return to the previous mode by using the cancel key input unit 340.

  In the display information selection mode 901, it is possible to shift to the block display mode 903 or the thumbnail display mode 904 via the mode selection unit 360 in which information displayed on each surface is determined. Further, the display information selection mode 901 can be shifted to the information input mode 902 by selecting a digital image by the enter key input unit 320.

  As described above, in the mobile terminal device 100 according to the first embodiment, the user can shift to another display mode using the mode selection unit 360 or the like, and the mobile terminal device 100 can be improved in user operation. .

FIG. 10 is a flowchart showing a process procedure of each mode transition of the mobile terminal device 100 according to the first embodiment.
The user of the mobile terminal device 100 selects one display mode from the key information input from the mode selection unit 360 using the mode key having a correspondence with the key input and the display mode (S1001).

  When the block display mode is selected by the user of the mobile terminal device 100, the event control unit 400 performs block display mode processing (S1002), when the thumbnail display mode is selected, thumbnail display mode processing (S1003), and image display mode. When selected, image display mode processing (S1004), when information input mode is selected, information input mode processing (S1005), and when display information selection mode is selected, display information selection mode processing (S1006) is performed. Do.

  Hereinafter, five display modes, a block display mode, a thumbnail display mode, an image display mode, an information input mode, and a display information selection mode displayed on the screen of the mobile terminal device 100 according to the present embodiment will be described in order. These display modes are examples, and the display modes displayed on the mobile terminal device 100 according to the present invention are not limited to these display modes.

FIG. 11 is a reference diagram of the thumbnail display mode displayed on the screen of the mobile terminal device 100 according to the first embodiment.
In the thumbnail display mode, you can search and browse the image objects included in the block objects displayed on the screen together with multiple information, group your favorite images into an album, or delete unnecessary images. Or editing a large number of images with reference to a three-dimensional block object.

  FIGS. 11A and 11B are reference diagrams when rearrangement is performed based on the related information of the image objects. Image objects can be arranged in “time order” as shown in FIG. 11A, or image objects can be arranged in “content order” as shown in FIG. 11B. In addition, if related information related to “priority” is pasted to an image object as a texture image, it can be rearranged in order of priority.

  In this way, in the thumbnail display mode, image search can be performed using related information other than images, and displayed image objects can be displayed as block objects rearranged according to the information. This image object rearrangement is performed by the user via the key input unit 100c which is set to be rearranged, and the requested image is displayed from the front side based on the favorite information. You can select an image.

Hereinafter, the operation of the thumbnail display mode will be described.
FIG. 12 is a flowchart showing a display processing procedure when the thumbnail display mode is selected in the display mode of the mobile terminal device 100 according to the first embodiment.

  First, when the thumbnail display mode is selected by the mode selection unit 360, the event control unit 400 designates a necessary object in the thumbnail display mode to the drawing control unit 600. In the thumbnail display mode, designated objects are a block object, a cursor object, a frame cursor object, a speech bubble object, an image object, and an axis object (S1201).

  Next, the event control unit 400 informs the drawing control unit 600 to draw the designated object (S1202). The drawing control unit 600 requests the object management unit 200 to create these objects, and the object management unit 200 requests the information management unit 101 for images and related information for each image. If there is an object that is not stored in the object storage unit 240, the object management unit 200 requests the information management unit 101 for a digital image and related information added to the image.

  The information management unit 101 creates an image object data table indicating the related information of each image and each digital image for each image object ID, and sends the image object data table to the object management unit 200 (S1203). The information management unit 101 uses the related information ID held in the information management unit 101 and the correspondence information of the address held in the information storage unit 110 to acquire the related information from the information storage unit 110. Thus, a data table is created. The data table for each object is illustrated in FIGS. 3 to 8 described above.

  In addition, the information management unit 101 creates a block object data table indicating the block information and the ID of the image object included in the block object, and sends it to the object management unit 200 (S1203). Next, the information management unit 101 creates a data table of a cursor object, a frame cursor object, a speech bubble object, and an axis object in which default values stored in the information storage unit 120 are similarly set (S1203). Send to 200.

  Then, the object management unit 200 requests the object generation unit 210 to generate an image object included in the block object data table. The object generation unit 210 acquires necessary image object information from the information management unit 101 from the IDs of the image objects included in the block object data table.

Further, the object generation unit 210 passes the image object data table acquired from the object management unit 200 to the texture generation unit 220 and the model generation unit 230.
The texture generation unit 220 generates a two-dimensional texture image by combining image data in which a texture image corresponding to each related information included in the image object is held and font image data (S1204).

  The model generation unit 230 generates a rectangular parallelepiped polygon model according to the description of the image object data table (S1205). The polygon model has vertex coordinates in a three-dimensional space of eight vertices and texture coordinates corresponding to each vertex.

  The object generation unit 210 generates an image object from information acquired using the texture image generated by the texture generation unit 220 and the polygon model generated by the corresponding model generation unit 230 (S1206). The generated image object is stored in the object storage unit 240 by the object management unit 200 (S1206). Similarly, in the object generation unit 210, other objects necessary in the thumbnail display mode are generated.

  When all the objects are created and stored, the object management unit 200 informs the drawing control unit 600 that all the objects have been created, and ends the loop in object creation (S1207).

  Then, when the drawing control unit 600 is notified from the object management unit 200 that all objects necessary for drawing have been created, the drawing control unit 600 sends each object to the scene generation unit 610 along with the viewpoint information acquired from the viewpoint control unit 390. .

  The scene generation unit 610 determines the arrangement coordinates of each object (S1208), and uses the determined arrangement coordinates to arrange a block object, a cursor object, a frame cursor object, a balloon object, and an axis object in the three-dimensional space. Then, a scene displayed on the screen is generated (S1209). In the thumbnail display mode, the arrangement information is described as a three-dimensional coordinate array as shown in FIG.

  When all the objects are arranged by the scene generation unit 610, the image generation unit 620 calculates how the three-dimensional space looks from the viewpoint coordinates acquired from the viewpoint control unit 390, and displays the result as image information. To 630. When the viewpoint change unit 380 has a viewpoint change (Y in S1210), the image generation unit 620 determines the arrangement coordinates from the new viewpoint position again.

  Next, when the viewpoint changing unit 380 does not change the viewpoint (N in S1210), it is determined whether or not the cursor object is moved via the cursor key input unit 300 (S1211). For example, in the thumbnail display mode, the top and bottom of the cursor control unit 310 mean the z-axis negative direction and the z-axis positive direction in the three-dimensional space, respectively. When the up / down key of the cursor control unit 310 is pressed, the “ID of the pointing object”, which is information held by the cursor object, goes up and down. Then, the cursor object is placed at the same z coordinate position as the z coordinate of the image object to be pointed.

When the cursor object has moved (Y in S1211), similarly, the “ID of the pointing object” of the frame cursor object goes up and down, and the frame cursor object is similarly arranged (S1212).
Similarly, a balloon object is arranged (S1213), and a thumbnail image of the image object pointed by the cursor is displayed together with the balloon (S1214). Therefore, the user can easily browse the image included in the block object by moving the cursor key up and down. If there is no movement of the cursor object (N in S1211), the processing after the image selection (S1215) is performed.

  Next, the user selects an image for image display from the images displayed on the speech bubble object (S1215). When an image is selected (Y in S1215), the image display mode is entered (S1216).

  Next, it is confirmed whether the user of the mobile terminal device 100 has changed the mode using the mode selection unit 360 (S1217), and if changed, the selected mode display process is performed (S1218). If not, next, it is confirmed whether or not the cursor object is moved from the cursor key input unit 300 (S1211). As described above, the thumbnail display mode as shown in FIG. 11 is displayed.

  As described above, in the thumbnail display mode of the mobile terminal device 100 according to the first embodiment, the three-dimensional block object with the related information pasted together with the image is displayed, and the user confirms the image using the balloon display. It can be performed. Therefore, the digital image search efficiency for the user can be increased by using more information.

  In addition, since colorful block objects are displayed together with images, it is possible to display objects beautifully on the screen of the mobile terminal device 100 and enhance entertainment in user operations.

Hereinafter, the block display mode will be described.
FIG. 13 is a reference diagram of screen display in the block display mode in the mobile terminal device according to the present embodiment. The block display mode is characterized in that image object groups grouped according to a series of related information are displayed as block objects together in an album form.

  FIG. 13A shows a screen 1301 in which a plurality of block objects 1301a are arranged in a three-dimensional space. For each block object, for example, a series of related events such as “September athletic meet” and “November trip to Kyoto” are shown. The album is displayed as a group of digital images, and the user selects a block object desired to be edited from the plurality of block objects by the cursor object 1301b via the cursor key input unit 300, and the thumbnail display shown in FIG. The mode screen 1304 can be entered.

  In FIG. 13B, the block object 1302a is sequentially displayed on the screen 1302, and the user can change the block object 1302a to be displayed according to the arrow 1302b displayed via the cursor key input unit 300. When the user selects the block object 1302a via the enter key input unit 320, the user can shift to the display screen 1304 in the thumbnail display mode shown in FIG.

  Although a block object is not displayed on the screen 1303 shown in FIG. 13C, a list 1303a in which titles of album groups stored in the image storage unit 110 are arranged in order of ID, priority, and date is displayed. The In this list 1303a, for example, “Kyoto trip”, “Sun drinking party” and the like are displayed. When the user of the mobile terminal device 100 selects one list item via the enter key input unit 320, the user can move to the display screen 1304 in the thumbnail display mode illustrated in FIG.

Hereinafter, the operation of the block display mode will be described.
FIG. 14 is a flowchart illustrating a display processing procedure when the block display mode is selected in the display mode of the mobile terminal device 100 according to the first embodiment.

  First, when the block display mode is selected in the mode selection unit 360, the event control unit 400 designates a necessary object in the block display mode to the drawing control unit 600. In the block display mode, designated objects are an image object, a block object, and a cursor object (S1401).

  Next, the event control unit 400 informs the drawing control unit 600 to draw the designated objects, and starts creating these objects (S1402). The drawing control unit 600 requests the object management unit 200 to create these objects, and the object management unit 200 requests the information management unit 101 for images and related information for each image.

  The information management unit 101 creates an image object data table indicating the related information of each image and each digital image for each image object ID, and sends the image object data table to the object management unit 200 (S1403). In addition, the information management unit 101 creates a block object data table indicating the block information and the ID of the image object included in the block object, and sends it to the object management unit 200 (S1403). Next, the information management unit 101 creates a cursor object data table in which default values stored in the information storage unit 120 are similarly set (S1403), and sends them to the object management unit 200.

  Then, the object management unit 200 requests the object generation unit 210 to generate an image object included in the block object data table. The object generation unit 210 acquires necessary image object information from the information management unit 101 from the IDs of the image objects included in the block object data table. Further, the object generation unit 210 passes the image object data table acquired from the object management unit 200 to the texture generation unit 220 and the model generation unit 230.

The texture generation unit 220 generates a two-dimensional texture image corresponding to each piece of related information included in the image object (S1404).
The model generation unit 230 generates a rectangular parallelepiped polygon model according to the description in the image object data table (S1405). Then, the object generation unit 210 generates a block object from information acquired using the texture image generated by the texture generation unit 220 and the polygon model generated by the corresponding model generation unit 230 (S1406). The generated block object is stored in the object storage unit 240 by the object management unit 200 (S1406). When all the objects are created and stored, the object management unit 200 informs the drawing control unit 600 that all the objects have been created, and ends the loop for object creation (S1407).

  Then, the drawing control unit 600 determines the arrangement method of each block object in the block display mode according to the input from the mode selection unit 360 and the determination key input unit 340 (S1408). Further, when the drawing control unit 600 is notified from the object management unit 200 that all image objects necessary for drawing have been created, the drawing control unit 600 sends the block object to the scene generation unit 610 together with the viewpoint information acquired from the viewpoint control unit 390. send.

  When arranging a plurality of block objects in the three-dimensional space in the block display mode, the maximum value of the block objects that can be displayed at the same time in the vertical and horizontal directions from the screen size is held in advance in the drawing control unit 600. The block objects are arranged in a matrix, and the viewpoint is set so that the maximum number of block objects that can be displayed on the screen at a time is displayed. The user can view all the block objects by changing the viewpoint with the viewpoint changing unit 380.

  When displaying block objects one by one in the block display mode, the block objects are displayed in order of block ID, order of priority defined by the user, or order of related information of the block object, for example, in order of date. The block object displayed on the left and right of the key input unit 300 is switched. At this time, it is possible to switch the display up and down five times instead of one by one up and down the cursor keys or to have different functions.

  It is also possible to display a list of albums stored in the image storage unit 110 in the block display mode. In this case, the titles of the respective block objects are displayed in a list in the order of the IDs of the respective block objects, the order of priority defined by the user, or the information of the block objects. The block object to be selected can be switched by moving the cursor key up and down, and the left and right cursor keys can have different functions.

  In the block display mode, the top, bottom, left, and right of the cursor control unit 310 mean the y-axis positive direction, the y-axis negative direction, the x-axis negative direction, and the x-axis positive direction in the three-dimensional space shown in FIG. When the cursor up / down key is pressed, the information of the cursor object, “ID of the object being pointed”, goes up and down. Place the cursor object at the same position as the object it points to.

  The scene generation unit 610 determines the arrangement coordinates of each object using the block object arrangement method and viewpoint information (S1409). In the block display mode, the arrangement coordinates are described as a three-dimensional coordinate array. Using the determined arrangement coordinates, a block object and a cursor object are arranged in the three-dimensional space to generate a scene displayed on the screen (S1410). When all the objects are arranged by the scene generation unit 610, the image generation unit 620 calculates how the three-dimensional space looks from the viewpoint coordinates acquired from the viewpoint control unit 390, and displays the result as image information. To 630. As described above, the screen in the block display mode as shown in FIG. 13 is displayed.

  Then, the viewpoint changing unit 380 confirms whether or not there is a viewpoint change (S1411). If there is a viewpoint change (Y in S1411), the image generation unit 620 again sets the arrangement coordinates from the new viewpoint position. (S1409) The following processing is performed.

  Next, when the viewpoint change unit 380 does not change the viewpoint (N in S1411), the user performs block display or editing from the block object displayed in the block display mode via the enter key input unit 320. An object is selected (S1412). When a block is selected (Y in S1412), the block display mode is shifted to the thumbnail display mode of the selected block object (S1413).

  In addition, it is confirmed whether the mode has been changed using the mode selection unit 360 (S1414). If the mode has been changed (Y in S1414), the selected mode display process is performed (S1415), and no change has been made. In the case (N in S1414), the block selection (S1412) and subsequent processes are repeated.

  As described above, in the block display mode of the mobile terminal device 100 according to the first embodiment, a series of related digital images of travel and events can be displayed on the screen as one block object like an album, Further, when the user selects a block object via the enter key input unit 320, it is possible to shift to a thumbnail display mode in which each digital image can be edited.

Therefore, the user can list album groups recorded in the mobile terminal device 100 as block objects, and the operability in the user's image search is improved.
Further, since the block object is displayed in the depth direction in proportion to the data amount of the recorded digital image, the user can visually confirm the data amount of the photographed digital image.

  FIG. 15 is a reference diagram of an image display mode displayed on the screen of the mobile terminal device 100 according to the first embodiment. In the image display mode, the image 1501a selected in the thumbnail display mode or the like is displayed on the screen 1501. Note that related information linked to the displayed image 1501a may be displayed on the screen 1501, or an arrow display 1501b may be provided so that the image can be changed.

Hereinafter, the operation of the display mode will be described.
FIG. 16 is a flowchart showing a processing procedure when the image display mode is selected in the display mode of the mobile terminal device 100 according to the first embodiment.

  When the mode selection unit 360 selects an image display mode, the event control unit 400 designates a digital image necessary for the image display mode to the drawing control unit 600. Then, the drawing control unit 600 acquires a digital image designated by the identification number from the image storage unit 110. The image generation unit 620 transmits the acquired digital image to the display unit 630 as it is, and displays the digital image on the display unit 630 (S1601).

  In the image display mode, the up and down directions in the cursor key input unit 300 mean that the image object is changed. When the up and down keys of the cursor key input unit 300 are pressed, the information that the cursor object has is “pointing”. The “object ID” goes up and down. When the pointing image object changes, the drawing control unit 600 acquires the designated digital image from the image storage unit 110 and changes the digital image to be displayed (Y in S1602).

  Next, in the case where there is no image change (N in S1602), when the user of the mobile terminal device 100 changes the mode using the mode selection unit 360 (Y in S1603), the selected mode display is displayed. Processing is performed (S1604). As described above, the image 1501 in the image display mode is displayed as shown in FIG.

As described above, in the image display mode, an image that the user wants to view can be displayed on the screen.
FIG. 17 is a reference diagram of the display information selection mode displayed on the screen of the mobile terminal device 100 according to the first embodiment. In the display information selection mode, the user inputs and updates information corresponding to each surface of the block object displayed in the block display mode and the thumbnail display mode.

  On the screen of the display information selection mode, a six-sided view that is a development view 1701a of the image object is displayed, and a “selection box 1701b” that allows the user to select related information related to the displayed image is displayed on each side. The The user selects related information desired to be displayed on the screen in the selection box 1701b via the enter key input unit 320. The related information is the same as described above, for example, information such as “priority”, “photographing place”, “priority”, and the like.

Hereinafter, the operation of the display information selection mode will be described.
FIG. 18 is a flowchart illustrating a processing procedure when the display information selection mode is selected in the display mode of the mobile terminal device 100 according to the first embodiment.

  When the display information selection mode is selected in the mode selection unit 360, the event control unit 400 designates a development view of the block object currently designated in the drawing control unit 600 and acquires a development view from the image storage unit 110. To do. In addition, the drawing control unit 600 acquires a selection box displayed together with the development view from the image storage unit 110.

  The image generation unit 620 acquires the development view and the selection box from the drawing control unit 600 and generates an image, and the display unit 630 displays the development view and the selection box (S1801 and S1802).

  In the display information selection mode, the up and down directions in the cursor key input unit 300 each mean change of related information included in the selection box. When there is a selection operation (Y in S1803), the selected related information is set as related information corresponding to the surface (S1804).

  Next, when there is no selection operation (N in S1803) and the user of the mobile terminal device 100 changes the mode using the mode selection unit 360 (Y in S1805), the selected mode display process is performed. (S1806). As described above, the selection information display mode screen 1701 is displayed as shown in FIG.

  As described above, in the display information selection mode according to the present invention, the user can input and update information corresponding to each surface of the block object and the image object to be displayed. Can be displayed on each surface of the block object and the image object.

  FIG. 19 is a reference diagram of the information input mode displayed on the screen of the mobile terminal device 100 according to the first embodiment. On the information input mode screen 1901, the user can update and input related information related to the image object 1901a using the input box 1901b. The related information input by the user is recorded in the information storage unit 120.

Hereinafter, the operation in the information input mode will be described.
FIG. 20 is a flowchart showing a processing procedure when the information input mode is selected in the display mode of portable terminal device 100 according to the first embodiment.

  When the information input mode is selected in the mode selection unit 360, the event control unit 400 specifies the digital image currently specified in the drawing control unit 600 and acquires image data and an input box from the image storage unit 110.

  The image generation unit 620 acquires an image and an input box from the drawing control unit 600 and generates an image, and the display unit 630 displays a development view and an input box (S2001 and S2002).

  Then, the user performs an input operation on the input box via the information input unit 140 (S2003). When there is an input operation (Y in S2003), the input related information is stored in the information storage unit 120 (S2004).

  Next, when there is no input operation (N in S2003) and the user of the mobile terminal device 100 changes the mode using the mode selection unit 360 (Y in S2005), the selected mode display process is performed. (S2006). As described above, the information input mode screen 1901 is displayed as shown in FIG.

  As described above, in the information input mode according to the present invention, related information of an image that is not automatically recorded at the time of shooting in the Exif format can be input via the information input unit 140. Information can be given and user information selection can be improved.

  Further, as described above, the mobile terminal device 100 according to the first embodiment includes the model generation unit 230 that generates an object, an image object by pasting a digital image and related information to the object, and an image object. Are generated in a three-dimensional manner to generate a block object.

  For this reason, the user of the mobile terminal device 100 can display at least two pieces of related information simultaneously with the image information displayed by the speech display by displaying the three-dimensional block object on the screen. Related information can be easily displayed even on a small screen of a portable terminal. Therefore, it is not necessary to switch the screen every time the image or various related information of the image is referred to, and the user can refer to more information, and the digital image search efficiency can be improved.

  In addition, since the image display and related information are pasted on the block object by color coding or the like, the screen of the mobile terminal device 100 can be displayed colorfully and beautifully and can be displayed without getting tired.

  The mobile terminal device 100 includes the viewpoint changing unit 380 that changes the viewpoint position of the object displayed on the screen, so that the block object displayed on the screen can be rotated and displayed in a desired direction. In addition to adding entertainment to the display method of the screen, the user can search for a digital image while referring to related information displayed on another surface hidden on the initial screen.

  Furthermore, since the mobile terminal device 100 according to the first embodiment includes the image processing unit 130 that extracts characteristic information from the image, the face recognition technology, color information, luminance information, and the like included in the digital image are used. Characteristic information is extracted and stored in the information storage unit 120 as related information. Therefore, it is possible to display information such as the appearance frequency of a specific person and photographs of day and night as a texture image on the side of the image object, and to give further related information in the user's image search.

  In addition, since the mobile terminal device 100 according to the first embodiment includes the texture generation unit 220 that generates a texture by changing the color, gradation, pattern, shape, or the like of the texture based on the type of related information, the mobile terminal device 100 is pasted on the block object. Related information can be displayed in an easy-to-understand manner.

  Furthermore, since the mobile terminal device 100 according to the first embodiment includes the drawing control unit 600 that can change the arrangement order of the image objects based on the related information, the user can select a favorite time such as “time” and “favorite degree”. Since it is possible to obtain a block object in which image objects are arranged based on information, search efficiency can be improved. Also, since block objects can be configured by arranging image objects in time order, unlike the conventional thumbnail display, even when a large amount of digital images are recorded by visualizing the flow of time between images, For example, it is possible to perform a search using time information such as “a picture of a drinking party at night two days ago”.

Then, the digital image is edited as an album with a series of related images through the key input unit 100c included in the mobile terminal device 100 according to the first embodiment, so that the image group for each album is stored in the same folder. It can be stored and displayed as a block object for each album in the block display mode.
(Embodiment 2)
Next, the mobile terminal device 100 according to Embodiment 2 will be described. In the second embodiment, the image objects displayed on the screen are arranged as two-dimensional objects instead of three-dimensional objects so that a part of the image objects positioned in the depth direction can be recognized. Further, it is assumed that the related information is pasted on the outer periphery of the image object.

  In the second embodiment, description of the functional configuration and operation procedure similar to those of the mobile terminal device 100 in the first embodiment will be omitted, and variations in the arrangement method of the image objects displayed on the screen will be omitted. Give an explanation.

FIG. 21 is a reference diagram illustrating another display example of the image object of the mobile terminal device 100 according to the second embodiment.
On the screen 2101, two-dimensional image objects 2102 are arranged in the depth direction. The image object 2101 is generated in the object generation unit 210 by pasting a texture image stored in the image storage unit 110 on a two-dimensional board polygon generated in the model generation unit 230. Then, on the outer peripheral portion 2101a of the image object 2102, textures classified by colors, patterns, gradations, shapes, and the like based on related information in the texture generation unit 220 are pasted. For example, a red texture is pasted on the outer peripheral portion 2102 a for a “family” photograph, a green for a “company” photograph, and a blue for a “travel” photograph.

  Accordingly, a two-dimensional image object 2102 is displayed on the screen 2101, and the photo content of another image object 2102 positioned in the depth direction can be confirmed in addition to the foremost photo, and the image object 2102 has a peripheral portion 2102 a on the outer periphery 2102 a. Since the color classification is performed based on the related information, the image is displayed on the screen 2101 visually and beautifully while helping the user to understand the image selection.

  The selected image object 2102 can be moved and arranged above the column to display the image selected by the user conspicuously, and the image arrangement method has been changed to the screen in the first embodiment. It is also conceivable that a display 2103 for displaying the image object 2102 is displayed so that the user can easily change the arrangement method of the image objects 2102.

FIG. 22 is a reference diagram illustrating another display example of the image object of the mobile terminal device 100 according to the second embodiment.
A cylindrical object 2202 is displayed on the screen 2201, and an image object 2202a is pasted on the surface. The outer peripheral portion 2202b of each image object 2202a is divided into patterns based on the related information.

  Then, the user uses the cursor key input unit 300 to rotate the cylindrical object 2202 with the left and right keys, and selects the image object 2202a with the up and down keys. It should be noted that when selecting the array by the cylindrical objects 2202, the image object 2202a in the middle row can always be provisionally selected.

  Therefore, the user can select the image object 2202a pasted on the surface while rotating the cylindrical object 2202, and can display the photograph to enhance the entertainment in the user's image selection.

FIG. 23 is a reference diagram illustrating another display example of the image object of the mobile terminal device 100 according to the second embodiment.
In this figure, the user of the mobile terminal device 100 can select a column with the left and right keys of the cursor key input unit 300 and select an image object 2302 with the up and down keys. Therefore, since many image objects 2302 are displayed on the screen at a time, the search efficiency of the image file for the user is improved, and a large number of image objects 2302 are colorfully displayed together with related information such as color coding. It can be displayed visually and beautifully.

  Therefore, a two-dimensional image object 2102 or the like is displayed on the screen 2101 or the like of the mobile terminal device 100 according to the second embodiment, and the user can have other image objects 2102 or the like positioned in the depth direction in addition to the frontmost photo. Therefore, the convenience in image selection can be improved. In addition, since the peripheries 2102a and the like of the image object 2102 and the like are provided with related information such as color coding, the image selection of the user is further facilitated and displayed visually beautifully.

Furthermore, the user of the mobile terminal device 100 can select a preferred arrangement method of the image object 2101 and the like from a plurality of arrangement methods, and can enhance entertainment by the display method.
Note that it is also conceivable to set an arrangement method selection mode in which a digital image arrangement method can be selected as a display mode in the portable terminal device according to the above embodiment.

  In this arrangement method selection mode, the user can display the arrangement method of the image objects displayed on the screen according to a preset arrangement. Examples of the arrangement method include an arrangement using a block object displayed on the screen of the mobile terminal device 100 according to the first embodiment, an arrangement using a two-dimensional object according to the second embodiment, and the like.

Therefore, the user of the mobile terminal device 100 can select the arrangement method of the image objects arranged on the screen by selecting the arrangement method selection mode.
FIG. 24 is a reference diagram illustrating another display example of the image object of the mobile terminal device 100 according to the second embodiment. In the display example shown in FIG. 24, help information of a mobile phone as a two-dimensional object is displayed in a three-dimensional manner.

  On the screen 2401, help information of the mobile phone is displayed as one two-dimensional object 2402 for each type, and an outline of help information is displayed as character information on the front of each two-dimensional object, and a border frame 2402a is displayed. Displays color information that is color-coded for each category of help information. The border frame 2402a displays, for example, help information related to mail in red and help information related to the portable camera in green.

  In addition, on the screen 2401, each two-dimensional object 2402 is arranged three-dimensionally in an oblique direction so that a part of information of the two-dimensional object 2402 located behind can be recognized. For this reason, the user can recognize a part of the help information located rearward on the same screen, and can search for help information more easily.

  Furthermore, for example, by selecting the camera mark 2402b displayed in the lower part of the two-dimensional object 2402, the downloaded help information can be referred to by a moving image, and a cursor 2403 positioned above the screen 2401 , Shows the position in the whole help information.

  In the above description of the embodiment, the description has been made assuming that a digital image is displayed on the screen. However, the present invention is not limited to this, and the digital moving image is displayed as a block object. You can also think about that. In this case, it may be possible to display the depth width of the three-dimensional object in proportion to the data amount of the digital moving image.

  Furthermore, the description according to the above-described embodiment is an exemplification, and the present invention is not limited to the usage form of the embodiment, and it goes without saying that the present invention can be used within the applicable range. .

  The mobile terminal device according to the present invention relates to an image display device having a function of displaying an image, and is particularly applicable to a mobile phone having a small screen, a PDA, a car navigation device, and the like.

3 is a block diagram illustrating an example of a functional configuration of a mobile terminal device according to Embodiment 1. FIG. 4 is a reference diagram of screen display of the mobile terminal device according to Embodiment 1. FIG. 4 is a reference diagram and a data table of an image object displayed in the mobile terminal device according to the first embodiment. 6 is a reference diagram and data table of a block object displayed in the mobile terminal device according to the first embodiment. 6 is a reference diagram and a data table of a cursor object displayed in the mobile terminal device of the first embodiment. 6 is a reference diagram and a data table of a frame cursor object displayed in the mobile terminal device of the first embodiment. 4 is a reference diagram and a data table of axis objects displayed in the mobile terminal device of the first embodiment. 4 is a reference diagram and a data table of a speech bubble object displayed in the mobile terminal device according to the first embodiment. FIG. 3 is an explanatory diagram showing a relationship between each mode of the mobile terminal device according to the first embodiment. 3 is a flowchart showing a processing procedure for mode transition of the mobile terminal device according to the first embodiment. 6 is a reference diagram of a thumbnail display mode displayed on the screen of the mobile terminal device according to Embodiment 1. FIG. 6 is a flowchart illustrating a display processing procedure when a thumbnail display mode is selected in the display mode of the mobile terminal device according to the first embodiment. It is a reference figure of the screen display of the block display mode in the portable terminal device which concerns on embodiment. 6 is a flowchart illustrating a display processing procedure when a block display mode is selected in the display mode of the mobile terminal device according to the first embodiment. 4 is a reference diagram of an image display mode displayed on the screen of the mobile terminal device according to Embodiment 1. FIG. 5 is a flowchart illustrating a processing procedure when an image display mode is selected in the display mode of the mobile terminal device according to the first embodiment. 6 is a reference diagram of a display information selection mode displayed on the screen of the mobile terminal device according to Embodiment 1. FIG. 6 is a flowchart illustrating a processing procedure when a display information selection mode is selected in the display mode of the mobile terminal device according to the first embodiment. 6 is a reference diagram of an information input mode displayed on the screen of the mobile terminal device according to Embodiment 1. FIG. 5 is a flowchart illustrating a processing procedure when an information input mode is selected in the display mode of the mobile terminal device according to the first embodiment. 12 is a reference diagram illustrating another display example of an image object of the mobile terminal device according to Embodiment 2. FIG. 12 is a reference diagram illustrating another display example of the image object of the mobile terminal device according to Embodiment 2. FIG. 12 is a reference diagram illustrating another display example of an image object of the mobile terminal device according to Embodiment 2. FIG. 12 is a reference diagram illustrating another display example of an image object of the mobile terminal device according to Embodiment 2. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 Mobile terminal device 101 Information management part 110 Image storage part 120 Information storage part 130 Image processing part 140 Information input part 200 Object management part 210 Object generation part 220 Texture generation part 230 Model generation part 240 Object storage part 300 Cursor key input part 310 Cursor control unit 320 Determination key input unit 330 Determination key control unit 340 Cancel key input unit 350 Cancel key control unit 360 Mode selection unit 370 Mode control unit 380 View point change unit 390 View point control unit 400 Event control unit 500 Information output unit 600 Drawing control Section 610 Scene generation section 620 Image generation section 630 Display section

Claims (28)

An object generation means for generating an object;
Texture generating means for generating a second texture image for each content of related information, which is information related to the first texture image;
An assigning unit that texture-maps the first texture image and the second texture image generated by the texture generation unit to the object generated by the object generation unit, and creates an image object;
Block generating means for generating a block object formed by arranging a plurality of the image objects in a three-dimensional space based on the related information;
Image generating means for generating an image of the block object;
And a display unit that displays the image generated by the image generation unit. The object generation means generates a two-dimensional object;
The assigning means creates the image object by pasting the first texture image on the front of the two-dimensional object and pasting the second texture image on a part or all of the outer periphery of the two-dimensional object,
The block generation means arranges a plurality of two-dimensional image objects in an oblique direction in a three-dimensional space so that at least a part of an image object located behind can be recognized based on the related information. Create a block object,
The image generation means generates an image of the block object,
The mobile terminal device according to claim 1, wherein the display unit displays the image. The first texture image is an image including character information, and the second texture image is an image including color information divided for each category of the first texture image. Mobile terminal equipment. The object generating means generates a three-dimensional object;
The giving means creates an image object by texture mapping the first texture image on the front of the three-dimensional object and the second texture image on the side of the three-dimensional object,
The block generation means generates a block object by arranging a plurality of the image objects in a three-dimensional space so as to constitute a polyhedron based on the related information,
The image generation means generates an image of the block object,
The mobile terminal device according to claim 1, wherein the display unit displays the image. The mobile terminal device according to claim 1, wherein the texture generation unit generates a second texture image that is color-coded, pattern-divided, gradation, or shape-based for each content of related information of the first texture image. The portable terminal device further includes:
Image storage means for storing the first texture image;
Comprising information storage means for storing arrangement information regarding the arrangement position of the related information and the block object,
The mobile terminal device according to claim 1, wherein the image generation unit generates an image of the block object based on the arrangement information. The related information includes creation time, shooting time, date, video recording time, image size, brightness, color image information itself, and shooting location, favorite level, genre, shooting content of reference count, mobile phone Contains at least one piece of information about terminal device help,
The portable terminal device further includes:
An arrangement changing means for changing the arrangement of the image objects based on the related information;
The mobile terminal device according to claim 1, wherein the block generation unit generates the block object by arranging the image objects in a three-dimensional space based on the change in the arrangement changing unit. The portable terminal device further includes:
Viewpoint changing means for freely moving the viewpoint on the screen according to the input from the user;
Arrangement information generating means for generating arrangement information viewed from the viewpoint after movement in the viewpoint changing means,
The image generation means generates an image of the block object based on the changed arrangement information,
The mobile terminal device according to claim 1, wherein the display unit displays the image. The portable terminal device further includes:
Extraction means for extracting feature information indicating the feature of the image from the first texture image;
The portable terminal device according to claim 1, wherein the feature information includes at least one of person information included in the image, color information included in the image, or luminance information included in the image. The portable terminal device according to claim 9, wherein the extraction unit records the feature information as related information in the information storage unit. The portable terminal device further includes:
Cursor key input means for moving the position of the cursor displayed on the screen to a user's desired position in accordance with an instruction from the user;
A decision key input means for selecting the image object where the cursor is located;
The portable terminal device according to claim 1, further comprising selection means for determining selection of an image object selected by the determination key input means. The block generation means generates a block object by arranging a plurality of image object groups having a series of relationships as an album in a three-dimensional space based on the related information,
The image generation means generates an image of at least one block object corresponding to the album;
The mobile terminal device according to claim 1, wherein the display unit displays the image. The object generation means further includes:
(A) a cursor object that is an arrow displayed on the screen;
(B) a frame cursor object indicating the viewpoint position of the block object;
(C) an axis object that is an arrow that displays time information of the block object;
(D) A speech bubble object for displaying a thumbnail image of an image object pointed to by the cursor object together with a speech bubble is generated. The portable terminal device further includes:
Mode selection means for selecting one of a plurality of display modes of the three-dimensional object;
The image generating means generates an image by arranging the image object, the block object, the cursor object, the frame cursor object, the axis object, or the balloon object in a three-dimensional space according to the display mode. ,
The mobile terminal device according to claim 1, wherein the display unit displays the image. The display mode is
(A) a block display mode in which at least one grouped block object is arranged in a three-dimensional space;
(B) a thumbnail display mode in which an image included in the block object can be searched by moving the cursor object via the cursor key input means;
(C) an image display mode for displaying the first texture image;
(D) an information input mode for inputting information held by the block object and the image object;
(E) The mobile terminal device according to claim 14, wherein the mobile terminal device is in one of display information selection modes for selecting information displayed on each surface of the block object. In the case where the thumbnail display mode is selected by the mode selection means,
The image generation means generates an image by arranging the block object, the cursor object, the frame cursor object, the balloon object, the image object, and the axis object in a three-dimensional space based on the arrangement information. The mobile terminal device according to claim 15. In the case where the block display mode is selected by the mode selection means,
The image generation means displays an image in which a plurality of the block objects and the cursor object are arranged in a three-dimensional space, or one block object, and sequentially changes the blocks to be displayed via the cursor key input means. The mobile terminal device according to claim 15, wherein an image is generated. 16. When the image selection mode is selected by the mode selection unit, the display unit displays the first texture image determined by the determination key input unit on a screen. Mobile terminal equipment. The portable terminal device further includes:
Comprising information input means for inputting information related to the first texture image from the user;
When the information selection mode is selected by the mode selection unit, the display unit includes an input box in which information is input by the first texture image determined by the determination key input unit and the information input unit. It displays on a screen. The portable terminal device of Claim 15 characterized by the above-mentioned. When the display information selection mode is selected by the mode selection means, the display means is a display information selection box for selecting a development view and display information of the image object determined by the determination key input means. The mobile terminal device according to claim 15, wherein the mobile terminal device is displayed on a screen. The mobile terminal device according to claim 1, wherein the block generation unit generates the block object by arranging a plurality of the image objects so as to form a cylindrical body. The portable terminal device further includes:
22. The apparatus according to claim 1, further comprising: a second assigning unit configured to texture-map a digital moving image and related information related to the digital moving image to the object to create a moving image object. The mobile terminal device described in 1. An object generation step for generating an object;
A texture generation step for generating a second texture image for each content of the related information, which is information related to the first texture image;
An imparting step of creating an image object by texture mapping the first texture image and the second texture image generated in the texture generation step to the object generated in the object generation step;
A block generation step of generating a block object formed by arranging a plurality of the image objects in a three-dimensional space based on the related information;
An image generating step for generating an image of the block object;
A display step of displaying the image generated in the image generation step. In the object generation step, a two-dimensional object is generated,
In the assigning step, the first texture image is pasted on the front of the two-dimensional object, and the second texture image is pasted on a part or all of the outer periphery of the two-dimensional object, thereby creating an image object,
In the block generation step, a plurality of two-dimensional image objects are arranged in an oblique direction in a three-dimensional space so that at least a part of an image object located behind can be recognized based on the related information. To generate a block object,
In the image generation step, an image of the block object is generated,
The image display method according to claim 23, wherein the image is displayed in the display step. In the object generation step, a three-dimensional object is generated,
In the assigning step, an image object is created by texture mapping the first texture image on the front surface of the three-dimensional object and the second texture image on the side surface of the three-dimensional object;
In the block generation step, a plurality of the image objects are arranged in a three-dimensional space so as to constitute a polyhedron based on the related information, and a block object is generated,
In the image generation step, an image of the block object is generated,
The image display method according to claim 23, wherein the image is displayed in the display step. An object generation step for generating an object;
A texture generation step for generating a second texture image for each content of the related information, which is information related to the first texture image;
An imparting step of creating an image object by texture mapping the first texture image and the second texture image generated in the texture generation step to the object generated in the object generation step;
A block generation step of generating a block object formed by arranging a plurality of the image objects in a three-dimensional space based on the related information;
An image generating step for generating an image of the block object;
A program causing a computer to execute a display step of displaying an image generated in the image generation step. In the object generation step, a two-dimensional object is generated,
In the assigning step, the first texture image is pasted on the front of the two-dimensional object, and the second texture image is pasted on a part or all of the outer periphery of the two-dimensional object, thereby creating an image object,
In the block generation step, a plurality of two-dimensional image objects are arranged in an oblique direction in a three-dimensional space so that at least a part of an image object located behind can be recognized based on the related information. To generate a block object,
In the image generation step, an image of the block object is generated,
27. The program according to claim 26, wherein the image is displayed in the display step. In the object generation step, a three-dimensional object is generated,
In the assigning step, an image object is created by texture mapping the first texture image on the front surface of the three-dimensional object and the second texture image on the side surface of the three-dimensional object;
In the block generation step, a plurality of the image objects are arranged in a three-dimensional space so as to constitute a polyhedron based on the related information, and a block object is generated,
In the image generation step, an image of the block object is generated,
27. The program according to claim 26, wherein the image is displayed in the display step.

Images