JP2011041020A - Mobile terminal, display method, and display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display a view located at the other side of a wall, an obstacle or the like on a display screen of a mobile terminal by a simple constitution. <P>SOLUTION: This mobile terminal includes: a communication part 13 for transmitting and receiving data between a server 20 having a plurality of video data obtained by imaging a predetermined landscape or building stored therein and itself; a display part 12 for displaying the video data transmitted from the server 20; a position detection part 15 for detecting the positional information of the mobile terminal 10; and an attitude detection part 16 for detecting the information of a direction which the mobile terminal 10 faces. The mobile terminal is configured to generate and transmit identification information for identifying predetermined video data to the server 20 based on the positional information detected by the position detection part 15 and the information of the direction detected by the attitude detection part 16, and display the picture data extracted and transmitted by the server 20 based on the identification information on the display part 12. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a mobile terminal, a display method, and a display system, and more particularly, to a technique for displaying a scenery on the screen of a mobile terminal outdoors or behind an obstacle in a place such as indoors or behind an obstacle.

  From the viewpoint of displaying a landscape on the display screen of a mobile terminal, conventionally, an image taken by a camera built in the mobile terminal has been displayed on the display screen. However, in this method, only images captured by a camera built in the portable terminal can be displayed on the display screen. That is, when the user of a mobile phone terminal is located indoors or behind an obstacle, the camera cannot capture the scenery outside the room or behind the obstacle, so these scenery are displayed on the display screen. It is not possible.

  It is technically possible to obtain such an image of the other side of the wall or obstacle using X-rays or thermography. In addition, an image of the other side of a wall or an obstacle is generated in advance by CG (computer graphics) so that the CG image can be viewed through a device such as goggles. For example, Patent Literature 1 describes a technique for displaying virtual stereoscopic video information created by CG in a real space viewed by a user through goggles.

JP 2000-155855 A

  However, the conventional techniques as described above have problems such as high cost, realization of the apparatus, an adverse effect on the human body, and poor quality of the obtained image. . As a result, there is a problem that it is difficult to incorporate such a technique into a portable terminal.

  The present invention has been made in view of this point, and an object of the present invention is to display a landscape on the other side of a wall or an obstacle on a display screen of a portable terminal with a simple configuration.

  A portable terminal according to one aspect of the present invention includes a communication unit that transmits and receives data to and from a server that stores a plurality of video data obtained by photographing a predetermined landscape or building, and a video transmitted from the server. And a display unit for displaying data. Moreover, the position detection part which detects the positional information on the said portable terminal, and the attitude | position detection part which detects the information of the direction which the said portable terminal faces are provided. Based on the position information detected by the position detection unit and the direction information detected by the posture detection unit, the communication unit generates specific information for specifying predetermined video data to the server. In addition to the output, the video data extracted and transmitted by the server based on the specific information is displayed on the display unit.

  With this configuration, the specific information for specifying the video data stored in the server is transmitted based on the position information and orientation (direction) information of the mobile terminal, and the server transmits the specific information to the mobile terminal. The video data extracted and transmitted based on the specific information is transmitted. Then, the video data is displayed on the display unit.

  According to the present invention, the video data extracted and transmitted from the server based on the position information and orientation (direction) information of the mobile terminal is displayed on the display unit of the mobile terminal. You will be able to appreciate the scenery behind the obstacles.

It is a block diagram which shows the structural example of the system by one embodiment of this invention. It is explanatory drawing which shows the production | generation example of the transmission data in the mobile telephone terminal by one embodiment of this invention. It is explanatory drawing which shows the structural example of the video database server by one embodiment of this invention. It is a flowchart which shows the example of operation | movement in the mobile telephone terminal and server by one embodiment of this invention. It is explanatory drawing which shows the example of the system outline | summary by one embodiment of this invention.

Hereinafter, a mode for carrying out the invention (hereinafter also referred to as this example) will be described.
[Example of overall system configuration]
FIG. 1 is a block diagram illustrating a configuration example of the display system of this example. The display system shown in FIG. 1 includes a mobile phone terminal 10 as an embodiment of the mobile terminal of the present invention and a server 20. The mobile phone terminal 10 and the server 20 are connected by, for example, a wireless communication network.

  First, a configuration example of the mobile phone terminal 10 will be described. The mobile phone terminal 10 includes an operation input unit 11, a display unit 12, a communication unit 13, a control unit 14, a position detection unit 15, and an attitude detection unit. 16 and a storage unit 17 are included.

  The operation input unit 11 includes dial keys such as numbers and symbols, various function keys, and the like, and receives operation inputs from the user. For example, when the function for realizing the display system of this example is installed in the mobile phone terminal 10 as an application, an operation for starting the application is performed on the operation input unit 11.

  The operation input unit 11 is also used as a user interface for setting, as a parameter, information on the type of a display object (hereinafter also simply referred to as an object), which will be described later. Parameters for specifying the display object include selection information for live video or video image of the object, information for specifying the type of the object (such as building or landscape), and time when the video image was taken , Information on the season, information on the distance to the display object, and the like.

  The display unit 12 includes, for example, a liquid crystal panel, an organic EL (Electro Luminescence) panel, and the like. Based on the operation input content from the user to the operation input unit 11, an e-mail creation screen, a web page, and the other side of the obstacle Displays the scenery on the side.

  The communication unit 13 is a processing unit that communicates with the server 20. The communication unit 13 transmits a video data acquisition request including target identification information (described later) generated by the control unit 14 to the server 20 or is transmitted from the server 20. Receive video data.

  The control unit 14 is configured by, for example, a CPU (Central Processing Unit) and the like, and controls each unit configuring the mobile phone terminal 10. Specifically, based on the operation input information to the operation input unit 11, the position detection information in the position detection unit 15, and the posture detection information in the posture detection unit 16, display objects to be displayed on the display unit 12 are displayed. A process of generating information for specifying (hereinafter referred to as object specifying information) is performed. Details of generation of the object specifying information will be described later with reference to FIG. The control unit 14 also acquires current date information and time information by an RTC (Real Time Clock) not shown.

  The position detection unit 15 is configured by a GPS (Global Positioning System), a geomagnetic sensor, or the like, and detects position information where the mobile phone terminal 10 is located. As position information of the mobile phone terminal 10, latitude information, longitude information, and height (elevation) information where the mobile phone terminal 10 is located are detected by GPS, and a direction (horizontal direction) in which the mobile phone terminal 10 faces by a geomagnetic sensor. Information in the left and right direction) is detected. The posture detection unit 16 includes a gyro sensor or the like, and detects angle information (elevation angle information) in the vertical direction of the mobile phone terminal 10.

  In this example, the configuration in which the information in the direction in which the mobile phone terminal 10 is facing is detected using a geomagnetic sensor is taken as an example, but the present invention is not limited to this. For example, direction information may be detected by a gyro sensor or the like. In this case, when the application is started for the first time, an initialization process for associating the actual orientation information of the mobile phone terminal 10 with a specific direction such as “north” is performed, and thereafter obtained by association What is necessary is just to acquire the deviation | shift amount from a reference point as azimuth | direction information.

  The storage unit 17 is configured by a ROM (Read Only Memory) or a RAM (Random Access Memory), and stores a program necessary for operating the mobile phone terminal 10 of this example, data generated or acquired by the user, and the like. Is done. The storage unit 17 also stores video data transmitted from the server 20.

  In FIG. 1, illustration of a microphone, a speaker, and an audio processing unit that controls them is omitted, but the mobile phone terminal 10 of this example also has these functions.

  Next, a configuration example of the server 20 will be described with reference to FIG. The server 20 includes a web server function unit 21, a data control unit 22, and a video database 23. The Web server function unit 21 transmits the video data extracted from the video database 23 by the data control unit 22 to the mobile phone terminal 10 in response to a video data acquisition request from the mobile phone terminal 10 as a client. The video data acquisition request here includes the object specifying information described above.

  A video database 23 and cameras C1 to C3 are connected to the data control unit 22. Then, the data control unit 22 selects either the video data stored in the video database 23 or the video obtained in real time by the cameras C1 to C3 based on the operation content from the user input to the operation input unit 11. Obtain and transmit to the Web server function unit 21.

  The video database 23 stores video data related to landscapes such as famous mountains and seas, video data related to buildings such as famous buildings, and the like. As these video data, what was image | photographed with the cameras C1-C3 is accumulate | stored. For all video data, latitude information, longitude information, height information, and direction information are set as parameters. When a video data acquisition request is transmitted from the mobile phone terminal 10, the video database is based on the request. Specific video data is extracted from 23. Details of the video database 23 will be described later with reference to FIG.

  Cameras C <b> 1 to C <b> 3 are cameras for photographing a landscape, a building, or the like displayed on the display unit 12 of the mobile phone terminal 10. The video taken and stored in advance is output to the video database 23, or the video currently being shot is output in real time in response to a request from the mobile phone terminal 10.

  In the present example, three cameras C1 to C3 are illustrated as an example, but it is assumed that more cameras are actually used. For example, when it is desired to photograph 360 degrees around a famous building La from each direction of east, west, south, and north, four cameras are required corresponding to each direction. Alternatively, if it is desired to shoot 360 ° images at 30 ° intervals, 12 cameras are required.

  In each of the cameras C1 to C3, latitude information, longitude information, height information, and direction information of each camera C are set as parameters, and these parameters are the images captured by the cameras C1 to C3. It is stored in the video database 23 as data attribute information. In addition, when a request for acquiring a video acquired in real time is transmitted from the mobile phone terminal 10, a specific camera C is selected based on these parameters.

  Next, with reference to FIG. 2, an example of the object specifying information generation process by the control unit 14 of the mobile phone terminal 10 will be described. An example of information acquired by the mobile phone terminal 10 is shown on the left side of FIG. 2, and the control unit 14 generates the information on the right side of FIG. 2 based on various information acquired by the mobile phone terminal 10. An example of the object specifying information is shown.

  As shown on the left side of FIG. 2, in the mobile phone terminal 10, latitude information, longitude information, and height (altitude) information are acquired as position information of the mobile phone terminal 10 by the position detection unit 15 (see FIG. 1). . Further, the distance from the mobile phone terminal 10 to the display object is designated by an operation input to the operation input unit 11.

  The distance from the mobile phone terminal 10 to the display object is the distance on the screen of the display unit 12 of the mobile phone terminal 10 in the landscape or building in the direction in which the mobile phone terminal 10 is facing. It is information for specifying whether to display what is in. When a user in Tokyo specifies 500 km as the distance while facing the west-southwest direction, it is possible to display a Kyoto landscape or a building on the display unit 12 of the mobile phone terminal 10.

  Further, the orientation detection unit 16 detects the direction information of the mobile phone terminal 10 and the elevation angle information which is the vertical direction information.

  The control unit 14 of the mobile phone terminal 10 calculates the latitude of the target object from the position information of the mobile phone terminal 10 thus obtained, the distance information from the mobile phone terminal 10 to the target object, and the orientation information of the mobile phone terminal 10. Calculate longitude, height, and direction. And based on the result obtained by calculation, object specific information is produced | generated as shown on the right side of FIG.

  Further, in the mobile phone terminal 10, as shown on the left side of FIG. 2, information on the current date and time (current date information) is acquired by the control unit 14, and depending on the operation input content to the operation input unit 11, The type of object (landscape or building) is selected. Furthermore, depending on the content of the operation input to the operation input unit 11, either an image captured in real time by the cameras C <b> 1 to C <b> 3 or an image stored in the image database 23 is selected.

  The control unit 14 of the mobile phone terminal 10 generates time information and season information for specifying the object from the current date and time information, and determines the type of video to be extracted based on the type information of the selected object. Generate information to identify. Further, control information for selecting the stored video in the camera C1 to the camera C3 or the video database 23 is generated based on the information of the real-time video or the stored video selected through the operation input unit 11.

  That is, in the control unit 14 of the mobile phone terminal 10, object specifying information for specifying an object to be displayed on the screen of the display unit 12 based on various information acquired by each unit in the mobile phone terminal 10. Is generated.

  Next, a configuration example of the video database 23 in the server 20 will be described with reference to FIG. The video database 23 manages video attribute information and actual video data in association with each other.

  As the attribute information of the video, the name and type (building or landscape) of the object shown in the video and its position information (latitude, longitude, height) are set. Different video data is prepared for each direction, time, and season.

  In the example illustrated in FIG. 3, information on latitude = XX °, longitude = XX °, and height = ZZm is registered as the position information of the video named “XX Tower”. In addition, as the type information, it is described that the video is “landscape”.

  Further, as the attribute information of the video named “XX Tower”, information indicating a live video and information indicating a pre-captured video video are registered for each direction, every time, and every season. As the information indicating the live video, the type of camera (camera C1) that is shooting the live video is described. Information that shows pre-captured video images includes the corresponding images for each direction of east, west, south, and north, every hour in the morning, noon, evening, and night, and each season in spring, summer, autumn, and winter. Information (video V1-1 to video V1-64) is described.

  It is assumed that similar attribute information is prepared for each video image of “ΔΔ mountain” and other scenery.

  When the object specifying information is transmitted from the mobile phone terminal 10, data matching the contents described in the object specifying information is extracted from the video database 23 configured as described above, and the mobile phone terminal 10 is transmitted.

[System operation example]
Next, an example of operation of the display system by the mobile phone terminal 10 and the server 20 will be described with reference to FIG. First, when an operation for starting an application is accepted via the operation input unit 11 in the mobile phone terminal 10 (step S11), the type information of the object to be displayed on the display unit 12 through the operation input unit 11 The input of video type information (real-time video or stored video) is also accepted (step S12).

  Next, based on control by the control unit 14 (see FIG. 1), position information and orientation information of the mobile phone terminal 10 are respectively acquired from the position detection unit 15 and the posture detection unit 16 (step S13). And in the control part 14, the information of the latitude of the target object, the longitude, height, and a direction is produced | generated based on the positional information and orientation information which were acquired by step S13 (step S14).

  Subsequently, in addition to the information on the latitude, longitude, height, and direction of the object, the object specifying information generated by the control unit 14 in consideration of the object type information and the image type information is obtained as video data acquisition. The request is transmitted to the server 20 through the communication unit 13 (step S15).

  In the server 20 that has received the video data acquisition request including the object specifying information, the data control unit 22 extracts optimal video data from the video database 23 (see FIG. 1) based on the received object specifying information, or A video image taken by the camera C designated in the object specifying information is selected (step S16). Then, the extracted or selected video data is transmitted to the mobile phone terminal 10 by the Web server function unit 21 (step S17).

  In the mobile phone terminal 10 that has received the video data transmitted in step S17, the received video data is stored in the storage unit 17 (see FIG. 1) (step S18). Subsequently, the video data stored in the storage unit 17 is displayed on the screen of the display unit 12 based on the control of the control unit 14 (step S19). Next, it is determined whether or not there has been a change in the position information or orientation information (orientation information) of the mobile phone terminal 10 (step S20). If there has been a change, the process returns to step S12 to continue the process.

  If there is no change in the position information or orientation information (orientation information) of the mobile phone terminal 10, it is determined whether or not an operation for ending the application by the user has been received (step S21). If an application termination operation is accepted, the process ends. If the operation for ending the application is not accepted, the process returns to step S20 and the determination is continued.

  By configuring and processing the display system in this way, as shown in FIG. 5, even when the user U is in a room surrounded by walls on all sides, the mobile phone terminal 10 located on the other side of the wall. It is possible to display a landscape or a building on the extended line in the direction facing the screen on the screen of the display unit 12. In the example shown in FIG. 5, the user U can see the building La ahead of the direction Aa on the screen by pointing the mobile phone terminal 10 in the direction Aa, and point it in the direction Ab. The building Lb can be seen, and the landscape Lc can be seen by facing the direction Ac.

[Effects of the embodiment]
According to the above-described embodiment, the user U can see the scenery that cannot be seen originally in a room or the like through the screen of the mobile phone terminal 10. For example, if you go to a tourist spot but cannot go out of the room due to bad weather, or if the beautiful scenery that you could see from your home has disappeared due to the construction of a new building, etc. 10 allows the user U to see the desired scenery.

  In addition, according to the above-described embodiment, since the user can set an arbitrary time such as morning, noon, evening, and night, the cellular phone terminal 10 can display a daytime scenery that cannot be seen because it is night. Can be displayed on the screen. Conversely, it is possible to see the night view during the day.

  In addition, according to the above-described embodiment, since a landscape and a building can be selected as an object to be displayed on the display unit 12, an image desired by the user U is appropriately displayed on the screen of the mobile phone terminal 10. Can be displayed.

  Further, according to the embodiment described above, the user U can select and acquire the video captured in real time by the cameras C1 to C3 and the stored video stored in the video database 23. You will be able to select and view the video. For example, by storing video data of beautiful scenery or buildings as pictures taken when the weather is good in the video database 23, the user U can have the beautiful scenery and the live video currently being shot. You can select your favorite video.

  Further, according to the above-described embodiment, since the distance from the mobile phone terminal 10 to the object can be specified via the operation input unit 11, the operational feeling of performing the zoom (telephoto) of the camera C. Thus, the user U can display a desired distant building or landscape on the screen of the mobile phone terminal 10.

  In addition, according to the above-described embodiment, it is not necessary to use X-rays or thermography, or to generate a video by CG, etc., and the building or landscape beyond the obstacle is displayed on the screen of the mobile phone terminal 10. Therefore, the manufacturing cost of the mobile phone terminal 10 can be reduced.

  In the above-described embodiment, an example in which the mobile terminal is applied to the mobile phone terminal 10 has been described. However, the present invention is not limited to this. For example, the present invention may be applied to other portable terminals such as a PDA (Personal Digital Assistants), a game machine having a communication function, and a music data reproducing apparatus having the same communication function. Or you may apply to the structure which made the display part 12 independent as goggles.

  DESCRIPTION OF SYMBOLS 10 ... Mobile telephone terminal, 11 ... Operation input part, 12 ... Display part, 13 ... Communication part, 14 ... Control part, 15 ... Position detection part, 16 ... Attitude detection part, 17 ... Memory | storage part, 20 ... Server, 21 ... Web server function unit function unit, 22 ... data control unit, 23 ... video database, C1 to C3 ... camera

Claims (11)

A communication unit that transmits and receives data to and from a server that stores a plurality of video data obtained by photographing a predetermined landscape or building;
A display unit for displaying video data transmitted from the server;
A position detector for detecting position information of the mobile terminal;
A posture detection unit that detects information of a direction in which the mobile terminal is facing;
Based on the position information detected by the position detection unit and the direction information detected by the posture detection unit, specific information for specifying predetermined video data to the server is generated, and the communication And a control unit that performs control to display the video data extracted and transmitted by the server based on the specific information on the display unit,
Mobile device. The position information detected by the position detection unit includes latitude, longitude, and altitude information of the position where the mobile terminal is located, and the direction information detected by the posture detection unit is the horizontal information of the mobile terminal. The mobile terminal according to claim 1, comprising left and right orientation information in a direction and elevation angle information of the mobile terminal. It has an operation input unit for inputting instructions from the user,
The portable terminal according to claim 2, wherein type specifying information for specifying a type of video data stored in the server is input from the operation input unit. The mobile terminal according to claim 3, wherein distance specifying information for designating a distance between a position of a subject in the video data stored in the server and a position of the mobile terminal is input from the operation input unit. . The control unit generates the specific information based on the position information detected by the position detection unit, the direction information detected by the posture detection unit, the type specifying information, and the distance specifying information. The mobile terminal according to claim 4. The control unit describes, in the specific information, video data type specifying information for selecting either video data stored in the server or video shot by a camera connected to the server. Item 6. A portable terminal according to Item 5. Detecting position information of the mobile terminal;
Detecting information on the direction in which the mobile terminal is facing;
Based on the detected position information and the information on the detected direction, predetermined video data is stored in a server that stores a plurality of video data obtained by photographing a predetermined landscape or building. Generating and transmitting specific information for identification;
And displaying the video data extracted and transmitted by the server based on the specific information on the display unit. A first communication unit that transmits and receives data to and from a server that stores a plurality of video data obtained by photographing a predetermined landscape or building;
A display unit for displaying video data transmitted from the server;
A position detector for detecting position information of the mobile terminal;
A posture detection unit that detects information of a direction in which the mobile terminal is facing;
Based on the position information detected by the position detection unit and the direction information detected by the posture detection unit, the server generates specific information for specifying predetermined video data to the server, and A portable terminal having a first control unit that outputs to the first communication unit and controls the display unit to display the video data extracted and transmitted by the server based on the specific information;
A second communication unit that transmits and receives data to and from the portable terminal;
A video data storage unit for storing the video data;
A server having a second control unit that extracts specific video data from the video data storage unit based on the specific information transmitted from the portable terminal and outputs the specific video data to the second communication unit. system. The display system according to claim 8, wherein the server is connected with a camera that captures video in real time. The said 2nd control part acquires the picked-up image by the said camera, and outputs to the said 2nd communication part, when the acquisition request | requirement of the picked-up image by a camera is transmitted from the said portable terminal. Display system. In the video data stored in the video data storage unit, the position information of the subject in the video data, the time information when the subject was shot, and the subject were shot as attribute information of the video data. Seasonal information and information on the subject type are defined,
The display system according to claim 10, wherein the first control unit of the mobile terminal generates specific information for specifying attribute information of the video data.

Images