JP5772067B2 - Portable terminal device and program - Google Patents

Description

The present invention relates to a portable terminal apparatus and a program.

  Conventionally, as a mobile terminal device that can be used as an operation device for a display device as an external device, there is a technology that enables a mobile phone to be used as a pointing device (mouse) for the external display device (see Patent Documents 1 to 4). In these prior arts, the mobile phone can be used as a mouse on the assumption that the mobile phone and the external display device are connected in advance. Further, in the above-described prior art (Patent Document 4), when the mobile phone is used as a pointing device (mouse) for the external display device, the object content displayed on the mobile phone is enlarged and displayed on the external display device. I am doing so.

JP 2007-123962 A JP 2008-263454 A JP 2007-150695 A JP 2010-165289 A

  However, in each of the above prior arts, even if the mobile phone can be used as a pointing device (mouse) for the external display device, only the operation instruction from the mobile phone to the external display device is provided, The device was only used as an output monitor.

An object of the present invention is to enable a process related to an object displayed on a mobile terminal device to be executed using a display device which is an external device.

Claim 1 is a portable terminal device that processes an object in response to an operation on the object to be processed, and a communication unit that directly or indirectly performs data communication with a display device as an external device; When the object to be processed is a terminal object and an access operation is performed on the terminal object, the terminal object or the processing result is displayed as a monitor link object on the display device via the communication unit. Display control means for performing control for causing the display device, management means for storing and managing the display position of the monitor link object on the display apparatus side received and acquired via the communication means in association with the terminal object, and the display apparatus The display operation signal is received via the communication means in response to the operation on the display content on the side. Processing means for executing processing corresponding to the operation content indicated by the display operation signal for the terminal object corresponding to the monitor link object when acquired, and the communication means distributes information via a network. Network communication means for indirectly communicating with the display device via a device, the terminal object is link information that specifies the location of data to be displayed, and the display control means By acquiring the identification information of the display device and notifying the information distribution device of the identification information and the link information, the data specified by the link information from the information distribution device is specified by the identification information. By transmitting the data to the display device, the data is displayed on the display device as the monitor link object. Carry out the control, characterized in that.

According to the present invention, by using the display device as an external device, Ru can perform processing on the objects to be displayed on the portable terminal device.

The block diagram which showed the communication network system which the mobile telephone 1 can utilize. FIG. 2 is a block diagram showing basic components of the mobile phone 1. The figure for demonstrating the display position management table DT. The block diagram which showed the basic component of the display apparatus 6 (TV receiver 6A) as an external apparatus with respect to the mobile telephone 1. FIG. FIG. 2 is a schematic external view of the mobile phone 1, in which (1) is a view of the mobile phone 1 as viewed from the surface direction, and (2) is a view as viewed from the back direction. (1), (2) is a schematic external view seen from the direction of one side of the mobile phone 1, and (2) is an operation device (mouse) for the display device 6 (television receiver 6A). ) Including a partially enlarged schematic cross-sectional view in a state of functioning as The flowchart which showed operation | movement of the mobile telephone 1 started execution by monitor link button operation (pairing operation). The flowchart which showed the operation | movement following FIG. The flowchart which showed the operation | movement following FIG. The flowchart by which execution is started by receiving a power-on signal from the outside as operation | movement of the display apparatus 6 (TV receiver 6A). 7 is a flowchart that starts execution when a monitor search signal is received from the mobile phone 1 as an operation of the display device 6 (television receiver 6A). (1)-(3) is the figure which showed the example of a display in the state where the screen by the side of the mobile telephone 1 and the screen by the side of the display apparatus 6 (TV receiver 6A) are linked. (1), (2) is the figure which showed the other example of a display where the screen by the side of the mobile telephone 1 and the screen by the side of the display apparatus 6 (TV receiver 6A) are linked. FIG. 5 is a diagram illustrating an operation example and a display example when the mobile phone 1 is used as an operation device for the display device 6 or the display device 6 is used as an operation device for the mobile phone 1. The figure which showed the other operation example and display example when using the mobile telephone 1 as an operation device with respect to the display apparatus 6, or using the display apparatus 6 as an operation device with respect to the mobile telephone 1. FIG.

Hereinafter, embodiments of the present invention will be described in detail with reference to FIGS.
This embodiment is applied to a mobile phone as a mobile terminal device, and FIG. 1 is a block diagram showing a communication network system in which the mobile phone 1 can be used.
The mobile phone 1 has a voice call function, an e-mail function, an Internet connection function (Web access function), a short-range communication function, and the like, and is connected to a wireless communication network (mobile communication network) 3 from the nearest base station 2. Then, it becomes possible to make a call with another mobile phone (not shown) via the wireless communication network 3.

  In addition, when the mobile phone 1 is connected to the Internet 4 via the wireless communication network 3, it is possible to send and receive e-mails, access a Web site and browse, and the information distribution device 5. Downloads various data such as image data distributed from the Internet 4 via the wireless communication network 3 and uploads various data such as image data from the mobile phone 1 via the wireless communication network 3 and the Internet 4. It is possible to do. The information distribution device 5 provides a cloud server service on a vendor-side network server that can secure highly secure and secure communication by using a protocol for encrypting and transmitting / receiving data. Various data provided by the user is stored or distributed to the terminal device on the user side.

  In addition, the mobile phone 1 can directly or indirectly perform data communication with the display device 6 as an external device (electronic device including a display unit). The display device 6 is a variety of electronic devices installed in the home and has a display screen with a touch panel (touch display screen). For example, a television receiver 6A, a projector 6B, a digital photo frame 6C, a personal computer (PC). 6D, in-vehicle terminal 6E, and the like, which are connected to a wireless LAN (private communication network) 7 constructed in the home. When the display device 6 is connected to the Internet 4 via a wireless LAN (private communication network) 7, various data such as image data can be downloaded from the information distribution device 5 and displayed. As described above, the display device 6 is connected to the information distribution device 5 so that the mobile phone 1 and the display device 6 are indirectly connected to each other via the information distribution device 5. In addition, the mobile phone 1 and the display device 6 are directly connected to each other via short-range communication, for example, Bluetooth (registered trademark).

  Here, the mobile phone 1 and the display device 6 are directly connected to each other by short-range communication when the mobile phone 1 functions itself as an operation device (pointing device: mouse) for the display device 6. That is, the mobile phone 1 uses a monitor link button, which will be described later, when an access operation is performed on an object to be processed, that is, in order to link the mobile phone 1 and the display device 6 via short-range communication. Is operated, it is searched whether or not there is a display device 6 capable of communication connection (short-distance communication) in the vicinity (for example, within a radius of 5 m) of the mobile phone 1, and the display device 6 is in the vicinity. If it exists, the display device 6 is specified as the output monitor of the mobile phone 1.

  Thus, when it is desired to link the mobile phone 1 and the display device 6 with the display device 6 as an output monitor for the mobile phone 1 and the mobile phone 1 as an operation device for the output monitor, the above-described monitor link The button is operated. Here, the object to be processed is link information, reception information, icons, files, image data, text data, and the like held on the mobile phone 1 side. For example, URL ( Uniform Resource Locator), data stored on the mobile phone 1 side, data such as icons and images displayed on the terminal screen.

  Then, when the access operation is performed on the object to be processed (when the monitor link button MB is operated), the mobile phone 1 identifies the object or the processing result as the output monitor. 6 is controlled. That is, the above-mentioned object to be processed, that is, an object accessible on the mobile phone 1 side (an object held on the mobile phone 1 side) is referred to as a terminal object, and the object to be displayed on the display device 6 specified as the output monitor or the object When the processing result is referred to as a monitor link object, the mobile phone 1 performs control for displaying the monitor link object on the display device 6 in accordance with an access operation to the terminal object.

  For example, if the terminal object is link information (for example, information having a storage location on the network or a file storage location on the server as a link destination), the mobile phone 1 processes the link information (received and acquired from the link destination). Data) is displayed on the display device 6 as a monitor link object, data such as a file stored at the link destination should be transmitted to the display device 6 to the information distribution device 5 Request communication (indirect communication) is requested. Further, if the terminal object is data other than link information, for example, an icon, the terminal object (icon) is displayed on the display device 6 as a monitor link object via the short distance communication. Direct communication to send is performed.

  After performing the control to display the monitor link object on the touch display screen of the display device 6 specified as the output monitor of the mobile phone 1 in this way, the mobile phone 1 operates the operation device mode (pointing device) for the display device 6. Mode: mouse mode). In this mouse mode, the cursor on the display device 6 is moved and displayed in accordance with the moving operation of the mobile phone 1, or the monitor link object on the display device 6 is pressed in accordance with the pressing operation of the monitor link button to be described later. Processing, that is, processing (to be described later) for the terminal object corresponding to the monitor link object is executed.

  On the other hand, when a touch action (described later) is performed on the touch display screen, the display device 6 transmits the touch position and action content to the mobile phone 1. When the mobile phone 1 receives a touch operation signal from the display device 6, the mobile phone 1 executes processing (described later) according to the touch position and the action content. As described above, the display device 6 can be used as an operation device between the mobile phone 1 and the display device 6 in addition to using the display device 6 as an output monitor of the mobile terminal device 1. That is, the touch operation on the display device 6 side is handled (recognized) as the operation on the mobile phone 1 side.

FIG. 2 is a block diagram showing basic components of the mobile phone 1.
The mobile phone 1 includes a voice call function, an e-mail function, an Internet connection function (Web access function), a short-range communication function, and the like. The audio signal processing unit 16, the operation unit 17, the touch display unit 18, the mouse sensor 19, the short-range communication unit 20, the wireless LAN module 21, the housing orientation detection unit 22, and the drawer detection unit 23 are configured.

  The CPU 11 is a central processing unit that operates by supplying power from a power supply unit 12 including a secondary battery (not shown) and controls the overall operation of the mobile phone 1 according to various programs in the storage unit 13. is there. The storage unit 13 stores programs and various applications for realizing the present embodiment in accordance with the operation procedures shown in FIGS. 7 to 9, and stores information necessary for the programs. Yes. The storage unit 13 may include a removable portable memory (recording medium) such as an SD card, or may include a storage area on a predetermined external server (not shown). . The memory 14 is a work area for temporarily storing various information necessary for the operation of the mobile phone 1 such as flag information, and is provided with a display position management table DT described later.

  The radio communication unit 15 demodulates the received signal into a received baseband signal during the operation of the call function, and then outputs the sound from the call speaker SP via the audio signal processing unit 16, and also receives the input sound from the call microphone MC. Data is taken from the audio signal processing unit 16 and encoded into a transmission baseband signal, and then transmitted and output from an antenna (not shown). The operation unit 17 includes a push button type power key, a side key SK such as a menu key, and a monitor link button MB. The CPU 11 performs predetermined processing according to an input operation signal from the operation unit 17. Execute. The monitor link button MB is used when the mobile phone 1 is caused to function as an operation device (mouse) for the display device 6 or is used when a monitor link object displayed on the display device 6 is operated. Key.

  The touch display unit 18 is disposed on the surface side of the casing constituting the mobile phone 1, and displays character data, a standby image, various objects, etc. on a high-definition liquid crystal display or an organic EL display, A touch key group such as a soft keyboard is displayed. It has a screen with different aspect ratios, and a transparent touch sensor (not shown) is stacked over the entire area of the screen to allow touch operation with a finger. And a touch screen for inputting data corresponding to the touch operation. The touch sensor uses a capacitive method that is lightweight, light transmissive, durable, etc., and detects key operations on soft keys (touch keys) or touch action operations (for example, tap operations). , Scroll operation, etc.).

  The mouse sensor 19 constitutes a light emitting part and a light receiving part of an optical mouse using infrared rays or laser light, and optically detects the moving direction and the moving amount of the mobile phone 1 in the mouse mode. The short-range communication unit 20 performs Bluetooth communication, and its communicable area has, for example, a substantially radius of 5 m, and performs serial / parallel conversion or the like on data to be communicated with Bluetooth. The wireless LAN module 21 constitutes a home wireless LAN communication function and is capable of high-speed wireless communication. The casing orientation detection unit 22 is an acceleration sensor that detects the orientation (posture) of the casing. The drawer detection unit 23 detects whether the mouse sensor 19 housed in the mobile phone 1 so as to be pulled out is in a state of being pulled out by any one of magnetic, mechanical, and electrical. The detection method is arbitrary.

FIG. 3 is a diagram for explaining the display position management table DT.
The display position management table DT is a table that manages the display position of the object displayed on the mobile phone 1 side in association with the display position of the object displayed on the display device 6 side. “Display position of object” and “Display position of monitor link object” are included. That is, the display position management table DT corresponds to “objects” “a”, “b”,..., “Monitor link object display position” on the display device 6, and the mobile phone 1. It is a table that stores and manages “terminal object display position” in association with each other.

FIG. 4 is a block diagram showing basic components of a television receiver 6A that is the display device 6. As shown in FIG.
The television receiver 6A has a CPU 61 as a core, in addition to basic components such as a power supply unit 62, a storage unit 63, a television broadcast receiving unit 64, an operation unit 65, a touch display unit 66, and a speaker 67, in this embodiment. Is configured to include an infrared communication unit 68, a short-range communication unit 69, and a wireless LAN module 70. The storage unit 63 stores a program and various applications for realizing the present embodiment in accordance with the operation procedure shown in FIGS. 10 and 11, and stores information necessary for the program. Yes.

  The touch display unit 66 constitutes a touch input function, and a transparent touch sensor (not shown) is laminated over the entire area of the screen so that a touch operation with a finger or the like is detected and the touch operation is performed. It is a touch screen for inputting corresponding data. In addition, as a touch sensor, an electrostatic capacitance method is adopted, and a key operation on a soft key (touch key) is detected or a touch action operation is detected. Here, the touch action operation includes a tap operation, a double tap operation, a hold (hold) operation, a pitch-in / pitch-out (enlargement / reduction) operation, a flick (pay = scroll) operation, a drag (move) operation, and the like. .

  The TV receiver 6A can perform infrared communication with a TV remote controller (not shown) via the infrared communication unit 68, and can perform short-range communication with the mobile phone 1 via the short-range communication unit 68. (Bluetooth communication) can be performed. Further, the television receiver 6 </ b> A can receive data distributed from the information distribution device 5 by being connected to the Internet 4 via the wireless LAN module 69.

  The mobile phone 1 and the television receiver 6A establish a communication link by Bluetooth communication (communication connection is set), whereby the short-range communication unit 20 on the mobile phone 1 side and the short-range communication unit on the TV receiver 6A side Data communication is possible between the mobile phone 1 and the television receiver 6 </ b> A via 68. In this case, when a monitor search operation (link operation: pairing operation) for specifying a connection partner is performed from the mobile phone 1, the mobile phone 1 has a display device 6 capable of communication connection nearby. After transmitting a monitor search signal for searching whether or not, a response signal is received from the television receiver 6A to establish a communication link between the mobile phone 1 and the television receiver 6A.

5 and 6 are schematic external views of the mobile phone 1, FIG. 5 (1) is a view of the mobile phone 1 as viewed from the front (front) direction, and FIG. 5 (2) is the back (back). 6 are views seen from the direction, and FIGS. 6A and 6B are views seen from the direction of one side surface of the mobile phone 1.
The mobile phone 1 has an overall rectangular parallelepiped shape, and the back surface of the case has a curved shape that is slightly raised in an arc as shown in FIG. On the surface of the cellular phone 1, touch display units 18 having different aspect ratios are disposed as shown in FIG. FIG. 5A is a front view of the mobile phone 1 showing a state in which the casing is vertically oriented so that the touch display unit 18 has a vertical screen, and one end side in the longitudinal direction of the vertical screen (upper side in the figure). ) Is provided with a call speaker SP, and a call microphone MC is provided on the other longitudinal end of the vertical screen (lower side in the figure). Furthermore, the drawer part 24 is accommodated in the longitudinal direction other end side (lower side in a figure) of a vertical screen so that extraction | drawer is possible. In addition, the above-described push button type monitor link button MB is disposed at a substantially central portion of the back surface of the housing.

  The drawer unit 24 is housed in a housing so that it can be pulled out (slidable). When the mobile phone 1 is used as a pointing device (mouse), as shown in FIG. The drawer portion 24 is pulled out from one end portion of the casing. FIG. 6A shows a state in which the drawer portion 24 is housed in the housing, and FIG. 6B shows a state in which the drawer portion 24 is pulled out from the housing. FIGS. 6A and 6B show a state where the housing is placed on a flat surface (for example, a desk surface) with the front side of the housing facing down in the drawing. In this state, when the drawer portion 24 is pulled out from the housing, the drawer portion 24 is used as a housing so that a space is formed by a step (gp) between the tip surface of the drawer portion 24 and the display surface of the touch display portion 18. It is attached. In this case, the lead-out portion 24 is drawn in an oblique direction (in the illustrated example, in an oblique direction of approximately 60 °) from the vicinity of one end portion on the surface side of the housing. As a result, the leading end portion of the lead-out portion 24 and the other end portion on the surface side of the casing are placed on the plane in a state of contact (two-point contact) on the plane (for example, the desk surface).

  In this way, by pulling out the drawer portion 24 in the oblique direction from the vicinity of one end of the surface of the housing, the one end side of the housing is raised, that is, the entire housing rises diagonally and protrudes upward in a curved manner. When the housing is placed on a flat surface such as a desk surface and the housing is sandwiched so that the back side of the housing is wrapped with a palm, the shape becomes an easy-to-use (easy to use) form as a mouse. . As described above, the back surface of the housing has a curved shape that is slightly raised in an arc shape, but the entire drawing portion 24 is also a slightly arcuate streamlined shape so that when the drawing portion 24 is pulled out, the drawing portion 24 is pulled out. The outer surface of the portion 24 and the outer surface of the end portion of the casing are curvedly continuous.

  Although not shown in the drawing, the drawing portion 24 is attached with a locking mechanism and a stopper mechanism. This locking mechanism maintains the state by elastically locking the drawer portion 24 in a state where the drawer portion 24 is completely pulled out from the housing and in a state where the drawer portion 24 is completely stored in the housing. Thus, when the drawer portion 24 is pulled out against the elastic force, the locking is released. Further, the stopper mechanism is configured to have a stopper that restricts the withdrawal of the drawer portion 24 at the longest position in order to suppress the withdrawal beyond the longest withdrawal amount shown in FIG. In addition, about the latching mechanism and the stopper mechanism, since the mechanism used generally is used, the illustration and detailed description shall be abbreviate | omitted.

  In addition, as shown in the partial enlarged view shown in FIG. 6 (2), a light emitting unit and a light receiving unit constituting an optical mouse sensor 19 using infrared rays or laser light are disposed in the drawer 24, and the housing. The reflected light is received by irradiating the desk surface on which the body is placed. When the above-described drawer detection unit 23 detects whether or not the drawer unit 24 (mouse sensor 19) has been pulled out, it is determined whether or not the drawer unit 24 has been pulled out to its longest drawer position. I try to detect it. Note that a power supply battery (for example, a button battery) 12a of the power supply unit 12 is stored in the drawer unit 24 so as to be freely inserted and removed (replaceable).

Next, the operation concept of the mobile phone 1 and the display device 6 (television receiver 6A) in the present embodiment will be described with reference to the flowcharts shown in FIGS. Here, each function described in these flowcharts is stored in the form of a readable program code, and operations according to the program code are sequentially executed. Further, it is possible to sequentially execute the operation according to the above-described program code transmitted via a transmission medium such as a network. In other words, in addition to the recording medium, an operation unique to the present embodiment can be executed using a program / data supplied externally via a transmission medium.
7 to 9 are flowcharts showing an outline of the operation of the characteristic part of the present embodiment in the entire operation of the mobile phone 1. When the flow of FIGS. Return to the main flow of operation (not shown).

  7 to 9 are flowcharts showing the operation of the mobile phone 1 which is started to be executed by the monitor link button operation (pairing operation). 10 and 11 are flowcharts showing the operation on the display device 6 (television receiver 6A) side. 12 and 13 are diagrams showing display examples in a state where the screen on the mobile phone 1 side and the screen on the display device 6 side are linked. 14 and 15 are diagrams showing other operation examples and display examples when the mobile phone 1 is used as an operation device for the display device 6 or the display device 6 is used as an operation device for the mobile phone 1. It is. Hereinafter, the flow of FIGS. 7 to 11 will be specifically described with reference to FIGS. 12 to 15.

  First, the CPU 11 is based on the detection results of the casing orientation detection unit 22 and the drawer detection unit 23, which is mouse style, that is, the drawer unit 24 is pulled out of the casing as shown in FIG. Thus, it is checked whether or not the back side of the casing is in a raised state (mouse style) (step A1 in FIG. 7). Now, the case orientation detection unit 22 detects that the case is sandwiched so that the back side of the case is wrapped with a palm, for example, the posture of the case (the touch display unit 18 or the mouse sensor 19) faces downward. If the drawer detection unit 23 detects that the drawer unit 24 is pulled out of the housing (mouse style) (YES in step A1), the touch display unit 18 displays a target of processing. It is checked whether the terminal object is being displayed (step A2).

  If the mouse style is not used (NO in step A1), or if the terminal object is not being displayed even in the mouse style (NO in step A2), the operation of the current monitor link button is invalidated. Therefore, the flow of FIGS. 7 to 9 is omitted, but if the terminal object is displayed on the mobile phone 1 side (YES in step A2), the display device 6 existing in the vicinity is searched. After the monitor search signal is transmitted from the short distance communication unit 20 (step A3), it is checked whether the response is received from the surrounding display device 6 by the short distance communication (step A4). If there is no response from the display device 6 (NO in step A4), a power-on signal is transmitted from the short-range communication unit 20 to the nearby display device 6 (step A5), and then the monitor search signal is again transmitted. Is transmitted (step A6), and it is checked whether a response is received from the surrounding display device 6 (step A7).

FIG. 10 is a flowchart of the start of the operation of the display device 6 (television receiver 6A) by receiving a power-on signal from the outside.
First, when receiving a power-on signal from the outside, the CPU 61 controls the power supply unit 62 to turn on the main power (step B1 in FIG. 10), and then checks whether the power-on signal is from the mobile phone 1 ( Step B2). Now, when a power-on signal is received from a TV remote control (not shown) via the infrared communication unit 68 (NO in step B2), the TV broadcast receiving unit 64 is activated to output a TV image from the touch display unit 66. The TV sound is output from the speaker 67 (step B3).

  When a TV operation is performed with a TV remote control or the like (YES in step B4), if it is not a power-off operation (NO in step B5), the processing corresponding to the operation includes, for example, channel switching, volume While performing processing such as adjustment, the television output processing is continued (step B6). And it returns to above-mentioned step B4, and will be in a television operation waiting state. Here, when a power-off operation is performed (YES in step B5), the process proceeds to a process of turning off the main power at this point. When a power-on signal is received from the mobile phone 1 by short-range communication (YES in step B2), a monitor search signal is received from the mobile phone 1 within a predetermined time (for example, within 3 seconds) after receiving the power-on signal. Is received (step B7), and if no monitor search signal is received (NO in step B7), the process proceeds to a process of turning off the main power supply at this time.

When a monitor search signal is received from the mobile phone 1 (YES in step B7), the flow proceeds to the flow in FIG. FIG. 11 is a flowchart started to be executed when a monitor search signal is received from the mobile phone 1 as an operation of the display device 6 (television receiver 6A).
First, the CPU 61 checks whether the display device 6 is operating, that is, whether a television image displaying a television image is being displayed on the screen (monitor screen) of the touch display unit 66 (step B11 in FIG. 11). If the television displaying video is being displayed (YES in step B11), the self-identification information (monitor ID) set in advance is read, and this monitor ID is displayed together with a display flag (not shown) in the mobile phone. 1 to respond to the monitor search signal (step B12). Then, it is checked whether or not the monitor link object is received from the mobile phone 1 by the short-range communication (step B13). If the monitor link object is not received (NO in step B13), the TV output process is continued and this is continued. At that time, the process exits the interrupt process of FIG.

  If the TV image is not being displayed and the TV image is not being displayed (NO in step B11), the mobile phone 1 displays its own identification information (monitor ID) together with a non-display flag (not shown). After transmitting and responding to the monitor search signal (step B23), it is checked whether the monitor link object is received from the mobile phone 1 (step B24). If the monitor link object is not received (NO in step B24), it is checked whether a predetermined time (for example, 3 seconds) has elapsed from the response and timed out (step B26). Returning to step B24, the system waits for reception, but when time-out occurs (YES in step B26), the process proceeds to a process of turning off the main power.

  On the other hand, when the mobile phone 1 receives a response from the surrounding display device 6 in response to the transmission of the monitor search signal as described above (YES in step A4 or step A7 in FIG. 7), a plurality of display devices 6 are displayed. If a response is received from a single display device 6 (NO in step A8), the display device 6 is specified as an output monitor and its reception ID (monitor) ID) is temporarily stored in the memory 14 (step A13). When responses from a plurality of display devices 6 are received (YES in step A8), referring to the display flag / non-display flag received at the time of response, It is checked whether there is a non-displaying display device 6 that has transmitted the non-displaying flag (step A9).

  As a result, if there is a non-displaying display device 6 (YES in step A9), that one display device 6 is specified as an output monitor, and its reception ID (monitor ID) is stored in the memory 14. Temporary storage is performed (step A10). When there are a plurality of display devices 6 that are not being displayed, the display device 6 that is closest in distance according to the reception status such as the radio wave intensity is compared with the preset screen size. A display device 6 having a large size is selected. If there is no display device 6 that is not being displayed (NO in step A9), the monitor ID of each responding display device 6 is received and displayed in a list (step A11), and a user operation is performed from the list. When an arbitrary monitor ID is selected and designated (step A12), the display device 6 corresponding to the selected monitor ID is specified as an output monitor, and the received ID (monitor ID) is temporarily stored in the memory 14 ( Step A13).

  After specifying the display device 6 in this manner, the flow proceeds to the flow of FIG. The display device 6 specified as the output monitor is simply referred to as an output monitor. First, referring to the display flag / non-display flag, the output monitor checks whether a television image is currently being displayed (step A14 in FIG. 8). If the output monitor is not displaying TV images (NO in step A14), the entire content of the screen (terminal screen) of the touch display unit 18, that is, the entire display content including the terminal object is displayed. The monitor link object is transmitted as it is to the output monitor (step A15). If the output monitor is being displayed (YES in step A14), it is checked whether the terminal object displayed in the terminal screen is link information such as a URL (step A16).

  If the terminal object is link information such as a URL (YES in step A16), it is checked whether the link information displayed in the terminal screen is one (YES in step A17). If there is one terminal object (link information) (YES in step A17), a request transmission is performed to the information distribution apparatus 5 via the Internet 4 (step A19). In other words, the distribution of information requesting that the data to be displayed is acquired from the data location (link destination) specified by the terminal object (link information) and sent to the output monitor as a monitor link object is distributed. This is performed for the device (server) 5.

  If each terminal object displayed in the terminal screen has a plurality of pieces of link information (NO in step A17), a process of directly transmitting the plurality of pieces of link information as monitor link objects to the output monitor by short-range communication. (Step A18). On the other hand, if the terminal object is not link information (NO in step A16), a terminal object such as an icon is directly transmitted to the output monitor as a monitor link object by short-range communication (step A20).

  On the other hand, when the output monitor receives the monitor link object directly from the mobile phone 1 or indirectly via the information distribution device 5 while the TV is not displayed (NO in step B11 in FIG. 11), YES in step B23. ) The monitor link object is displayed in the entire area of the monitor screen (step B24). When a monitor link object is received directly from the mobile phone 1 or indirectly via the information distribution device 5 (YES in step B13) while the television is being displayed (YES in step B11), a corner of the monitor screen is displayed. The monitor link object is displayed superimposed on the television image (step B14).

  FIG. 12 (1) shows a display example when the terminal object displayed on the terminal screen is link information and the output monitor is not displaying a TV image (NO in step A14 in FIG. 8). This is a case where the entire terminal screen including the terminal object (link information) is displayed on the entire output monitor by executing step A15 in FIG. In this case, the terminal object (link information) is displayed as a monitor link object on the entire screen, and a cursor is displayed in the vicinity of the monitor link object.

  FIG. 12 (2) shows a case where one link information is displayed as a terminal object on the terminal screen, and when the output monitor is displaying a television image (YES in step A14 in FIG. 8, step A16). And A17 (YES in A17), and image data (person) acquired through the information distribution apparatus 5 in response to a request transmission from the mobile phone 1 to the information distribution apparatus 5 by executing step A19 in FIG. Is displayed on the monitor screen in a corner (lower left in the figure) superimposed on the television image. In this case, image data (person's face image), which is the processing result of the terminal object, is displayed as a monitor link object at the corner of the screen, and a cursor is displayed in the vicinity thereof.

  FIG. 12 (3) shows a case where a plurality (three) of link information is displayed as a terminal object on the terminal screen, and when the output monitor is displaying a television image (in step A14 in FIG. 8). YES, YES in step A16, NO in step A17), and a plurality of link information is displayed on the TV image at the corner (lower left in the figure) of the monitor screen by executing step A18 in FIG. This is the case where they are displayed in a superimposed manner. In this case, a plurality of pieces of link information are displayed as monitor link objects at the corners of the screen, and a cursor is displayed in the vicinity thereof.

  FIG. 13 (1) shows a case where information (for example, a plurality of icons) other than link information is displayed as a terminal object on the terminal screen, and the output monitor is not displaying a TV image (FIG. 13). FIG. 8 illustrates a display example of “NO” in step A14, in which the entire terminal screen including the terminal object (a plurality of icons) is displayed on the entire output monitor by executing step A15 in FIG. In this case, a plurality of icons are displayed as a monitor link object, and a cursor is displayed in the vicinity of the monitor link object.

  FIG. 13 (2) shows a case where information (for example, a plurality of icons) other than link information is displayed as a terminal object on the terminal screen, and the output monitor is displaying a television image (FIG. 8). The display example of “YES in step A14 and NO in step A16” shows a display example. By executing step A20 in FIG. 8, a plurality of icons are superimposed on the TV image at the corner (lower left in the figure) of the monitor screen. Is displayed. In this case, a plurality of icons are displayed at the corners of the screen as monitor link objects, and a cursor is displayed in the vicinity thereof.

  Thus, the output monitor displays the monitor link object (step B14 or step B25 in FIG. 11), detects the display position of the monitor link object, and transmits it to the mobile phone 1 (step B15). . Whether or not a touch action operation is performed on the touch display unit 66 (step B16), whether a new monitor link object is received (step B18), or whether a link end instruction is received from the mobile phone 1 (Step B20). If any one of tap operation, double tap operation, hold operation, pitch-in / pitch-out operation, flick operation, drag operation, etc. is performed as a touch action operation (YES in step B16), the touch position and action content are displayed. After detecting and acquiring (step B17) and transmitting it as a touch operation signal to the mobile phone 1 (step B18), the process returns to the above-described step B16.

  When the output monitor receives a new monitor link object (YES in step B19), the output monitor checks whether the TV is being displayed (step B20). The display returns to the alignment display (step B14), but if the television is not being displayed (NO in step B20), the process returns to the above-described full display (step B25), and the above-described operation is repeated. When a link end instruction is received from the mobile phone 1 (YES in step B21), it is checked whether the television is being displayed (step B22). If the television is being displayed (YES in step B22), the flow of FIG. The process exits from the interrupt process, but if the television is not being displayed (NO in step B22), the process proceeds to a process of turning off the main power.

  When the mobile phone 1 receives the display position of the monitor link object from the output monitor (step A21 in FIG. 8), it detects the display position of the terminal object corresponding to the monitor link object, The display position management table DT is stored in association with the display position of the monitor link object (step A22). Then, after shifting to the operation device mode (mouse mode) (step A23), the flow proceeds to the flow of FIG. 9 and processing related to the monitor link object (processing for the terminal object corresponding to the monitor link object) is executed.

  That is, prior to the start of processing related to the monitor link object (processing for the terminal object corresponding to the monitor link object), the presence / absence of reception from the output monitor is checked (step A24 in FIG. 9), or the mobile phone 1 is used as a mouse. It is checked whether or not the moving operation to be moved or the monitor link button MB is operated (step A35), or the mouse style is released, that is, the drawer 24 is pulled out from the housing as shown in FIG. 6 (2). Then, it is checked whether the mouse sensor 19 is facing downward (step A37). If the presence of reception from the output monitor is detected (YES in step A24), a touch operation signal (touch position and action content) from the output monitor is received and acquired (step A25), and the touch position is the monitor link object. Is checked (step A26). If the touch position matches the display position of the monitor link object (YES in step A26), a command corresponding to the action content is generated (step A27).

  For example, if the action content is one of tap, double tap, hold, pitch-in / pitch-out, flick, and drag, a command corresponding to that is generated. Then, the display position management table DT is referred to, and processing corresponding to the above-described generation command is executed for the terminal object corresponding to the monitor link / object (step A28). For example, an object selection process, an object movement process, an object enlargement / reduction process, and the like are executed. Then, the processing result is transmitted to the output monitor as a new monitor link object (step A29). Then, after receiving and acquiring the display position of the new monitor link object from the output monitor (step A30), the display position management table DT associates the display position of the monitor link object with the display position of the terminal object. (Step A31). Thereafter, the process returns to step A24 described above.

  In FIG. 14, a plurality of (three) icons are displayed as terminal objects on the terminal screen, and the plurality of icons are displayed as monitor link objects on the output monitor (in the example shown, the contents of the terminal screen are fully displayed). It is the figure which showed the example of a display when the icon position desired as a touch action on a monitor screen is touched in the state which is being held. In the illustrated example, when the position of the icon “a” is touched, the output monitor transmits the touch position and action content (tap) to the mobile phone 1, and the mobile phone 1 supports the action content (tap). In addition to displaying the processing result on the terminal screen and sending it as a new monitor link object to the output monitor, a new monitor link object is displayed on the monitor screen (in the example shown in the figure). (Full display).

  Further, when the mobile phone 1 receives the touch position and the action content (step A25), if the touch position does not match the display position of the monitor link object (NO in step A26), the mobile phone 1 sets the action content. After the corresponding command is generated (step A32), for example, scroll processing, enlargement / reduction processing, and the like are executed as processing for controlling the entire terminal screen according to the generation command (step A33). Then, the contents of the terminal screen updated according to the processing result of the screen control are transmitted to the output monitor as a new monitor link object (step A34). Then, after receiving and acquiring the display position of the new monitor link object from the output monitor (step A30), the display position management table DT associates the display position of the monitor link object with the display position of the terminal object. (Step A31). Thereafter, the process returns to step A24 described above.

  FIG. 15 shows a monitor screen in a state in which an image is displayed as a terminal object on the terminal screen and this image is also displayed as a monitor link object on the output monitor (in the illustrated example, the contents of the terminal screen are fully displayed). It is the figure which showed the example of a display when the pitch-out touch operation which instruct | indicates the expansion of an image is performed as an upper touch action. In the illustrated example, when a pitch-out touch operation is performed, the output monitor transmits the touch position and action content (pitch out) to the mobile phone 1, and the mobile phone 1 supports the action content (pitch out). In addition to displaying the processing result on the terminal screen and sending it to the output monitor as a new monitor link object, a new monitor link object is displayed on the monitor screen (in the example shown in the figure). (Full display).

  If the moving operation for moving the mobile phone 1 as a mouse or the monitor link button MB is operated (YES in step A35), the processing corresponding to the operation is performed (step A36), and then the above-described step A37. Move on. In this case, when a moving operation is performed, a cursor movement signal is transmitted to the output monitor, and then processing for receiving the moved cursor position from the output monitor is executed. When the monitor link button MB is operated, if the display position of the cursor matches the display position of the monitor link object on the monitor screen of the output monitor, processing related to the monitor link object (corresponding to the monitor link object) Process for the terminal object). When the mouse style is released (YES in step A37), a link end signal is transmitted to the output monitor to release the pointing device mode (mouse mode) (step A38), and then the operations shown in FIGS. Get out of the flow.

  As described above, in the present embodiment, when an access operation is performed on a processing target terminal object, the mobile phone 1 displays the terminal object or the processing result on the display device 6 as a monitor link object. And a process corresponding to the touch operation performed on the display device 6 side is executed for the terminal object corresponding to the monitor link / object. When the display device 6 is used as an output monitor, the touch operation on the display device 6 performed while the object is displayed on the display device 6 is recognized as the operation on the mobile phone 1 and processing related to the object is executed. It is possible to combine operability and convenience.

  Therefore, the user can perform an operation of touching the screen while viewing the monitor link object displayed on the display device 6 under the control of the mobile phone 1. In other words, the mobile phone 1 can be used as an operation device for the display device 6, and the display device 6 can be used as an operation device for the mobile phone 1, so that they can be used as operation devices. It becomes possible to combine operability and convenience.

  When an access operation is performed on a terminal object to be processed, the mobile phone 1 specifies the display device 6 as an output monitor if there is a display device 6 capable of communication connection in the vicinity. Since the process corresponding to the touch operation is executed, if there is a display device 6 capable of communication connection in the vicinity of the user who owns the mobile phone 1, the display device 6 It is possible to execute processing corresponding to the touch operation without being conscious of the communication connection or switching to the operation device mode.

  If the terminal object is link information, the mobile phone 1 notifies the information distribution device 5 of the identification information (monitor ID) and link information of the display device 6 specified as the output monitor. Since the data specified by the link information is transmitted to the display device 6 so as to control the display device 6 to display the data as a monitor link object, the specification of the display device 6 is When high-resolution display or display processing with high load (such as high-resolution moving image processing) is possible, it is possible to realize display that makes full use of the specifications.

  The mobile phone 1 performs control for displaying the terminal object on the display device 6 as a monitor link object by transmitting the terminal object to the display device 6 specified as the output monitor by short-range communication. Therefore, it is possible to perform transmission without charge, which is economical and effective in terms of security since the communicable range is narrow.

  When an access operation is performed on the mobile phone 1 terminal object, if any display device 6 does not respond to a search signal transmitted by short-range communication, a power-on signal is transmitted. Since it is determined whether or not there is a response from one of the display devices in response to the transmission of the search signal again after that, even if the nearby display device 6 is in the power-off state, a special response is made in advance. It can be used as an output monitor without any operation.

  When the mobile phone 1 performs control for displaying the monitor link object on the display device 6 specified as the output monitor, if other display data is already displayed on the display device 6, the other display data is displayed. In addition, since the monitor link object is displayed together with the monitor link object, other display data (for example, a television image) already displayed can be viewed even if the monitor link object is displayed.

  The mobile phone 1 displays the entire display content including the terminal object on the display device 8 as it is as a monitor link object if no other display data has already been displayed on the display device 6. The object becomes easier to see.

  Since the operation device mode (mouse mode) is canceled based on the attitude (orientation) of the casing detected by the casing orientation detection unit 22, the attitude of the casing (without the special key operation) Mouse mode can be canceled simply by changing the orientation.

  In the above-described embodiment, Bluetooth communication is used as short-range communication between the mobile phone 1 and the display device 6, but infrared communication or the like may be used. Further, although the information distribution apparatus 5 is connected via the Internet 4, it may be connected via a local communication network.

  When the mobile phone 1 is used as a pointing device (mouse), as shown in FIG. 6 (2), the case is placed with the front side of the casing facing down, that is, the touch display unit 18 facing a plane (for example, a desk surface). However, the plane is not limited to a horizontal plane, and may be a vertical plane (for example, a wall surface). In the above-described embodiment, the mobile phone 1 is used as a mouse function. However, the mobile phone 1 may be used as a touch pad function (pointing device) or a motion sensor function (space movement operation device). Also good.

  In the above-described embodiment, if other display data is already displayed on the display device 6, the monitor link object is superimposed and displayed together with the other display data. The display data and the monitor link object may be displayed in parallel.

  Moreover, although the case where it applied to a mobile telephone was illustrated as a portable terminal device in embodiment mentioned above, not only this but PDA, a digital camera, a music player, etc. may be sufficient.

  Further, the “machine” and “unit” shown in the above-described embodiment may be separated into a plurality of cases according to function, and are not limited to a single case. In addition, each step described in the above-described flowchart is not limited to time-series processing, and a plurality of steps may be processed in parallel or separately.

The embodiment of the present invention has been described above. However, the present invention is not limited to this, and includes the invention described in the claims and the equivalent scope thereof.
Hereinafter, the invention described in the claims of the present application will be appended.
(Appendix)
(Claim 1)
The invention described in claim 1
A mobile terminal device that processes an object according to an operation on an object to be processed,
A communication means for performing data communication directly or indirectly with a display device as an external device;
When the object to be processed is a terminal object and an access operation is performed on the terminal object, the terminal object or the processing result is displayed as a monitor link object on the display device via the communication unit. Display control means for performing control to
Management means for storing and managing the display position of the monitor link object on the display device side received and acquired via the communication means in association with the terminal object;
When the display operation signal is received and acquired via the communication means in response to an operation on the display content on the display device side, the display operation signal indicates the terminal object corresponding to the monitor link object. Processing means for executing processing according to the operation content;
It is a portable terminal device provided with.
(Claim 2)
The invention according to claim 2 is the portable terminal device according to claim 1,
Determining whether or not the display device capable of communication connection exists in the vicinity of the terminal device when the object to be processed is a terminal object and an access operation is performed on the terminal object Means,
A specifying means for specifying the display device as an output monitor when the determining device determines that the display device exists;
Further comprising
The display control means performs control for displaying the monitor link object on the display device specified as the output monitor via the communication means.
The mobile terminal device according to claim 1, wherein the mobile terminal device is configured as described above.
(Claim 3)
The invention according to claim 3 is the portable terminal device according to claim 1 or 2,
The communication means is a network communication means for indirectly communicating with the display device via an information distribution device via a network,
The terminal object is link information that specifies the location of data to be displayed,
The display control means acquires the identification information of the display device and notifies the information distribution device of the identification information and the link information, so that the data specified by the link information from the information distribution device is obtained. By transmitting to the display device specified by the identification information, control to display the data as the monitor link object on the display device is performed.
This is a portable terminal device that is characterized as described above.
(Claim 4)
The invention according to claim 4 is the portable terminal device according to claim 1 or 2,
The communication means is a short-range communication means for performing short-range communication directly with a display device existing in the vicinity,
The display control means controls the display apparatus to display the terminal object as the monitor link object by causing the display apparatus to transmit the terminal object via the short-range communication means. Do,
This is a portable terminal device that is characterized as described above.
(Claim 5)
The invention according to claim 5 is the portable terminal device according to any one of claims 1 to 4,
The communication means is a short-range communication means for performing short-range communication directly with a display device existing in the vicinity,
The determination unit is configured to determine whether or not there is a response from any of the display devices to a search signal transmitted from the communication unit when an access operation is performed on the terminal object. It is determined whether there is a display device capable of communication connection in the vicinity of the terminal device, and when there is no response from any of the display devices, the communication means instructs the power-on of the display device There is a display device capable of communication connection in the vicinity of the terminal device depending on whether there is a response from one of the display devices in response to the transmission of the search signal again after transmitting the signal. To determine whether or not
This is a portable terminal device that is characterized as described above.
(Claim 6)
The invention according to claim 6 is the portable terminal device according to any one of claims 1 to 5,
The display control means determines whether other display data is already displayed on the display device when performing control for displaying the monitor link object on the display device specified as the output monitor. If other display data is already displayed, control is performed to display the monitor link object together with the other display data.
This is a portable terminal device that is characterized as described above.
(Claim 7)
The invention according to claim 7 is the portable terminal device according to claim 6,
In a state where the terminal object is displayed on the terminal screen of the terminal device, the display control unit determines that no other display data is displayed on the display device specified as the output monitor. Control to display the entire display content including the terminal object displayed on the screen as it is as the monitor link object on the display device;
This is a portable terminal device that is characterized as described above.
(Claim 8)
The invention according to claim 8 is the portable terminal device according to any one of claims 1 to 7,
Mode control means for shifting to an operation device mode for causing the terminal device to function as an operation device for the display device;
And a posture detecting means for detecting the posture of the housing,
The mode control means cancels the operation device mode based on the detection result of the posture detection means.
This is a portable terminal device that is characterized as described above.
(Claim 9)
The invention according to claim 9 is the portable terminal device according to any one of claims 1 to 8,
The operation for the display content on the display device side is a touch action on a touch screen,
The processing means executes processing according to the action content indicated by the touch operation signal received and acquired via the communication means.
The portable terminal device according to the present invention is characterized in that it is configured as described above.
(Claim 10)
The invention according to claim 10 is:
Against the computer,
A function of performing data communication directly or indirectly with a display device as an external device;
When an object to be processed is a terminal object and an access operation is performed on the terminal object, the terminal object or a processing result thereof is displayed as a monitor link object on the display device via the data communication. A function to perform control for
A function of storing and managing the display position of the monitor link object on the display device side acquired through the data communication in association with the terminal object;
When the display operation signal is received and acquired via the data communication in response to an operation on the display content on the display device side, the display operation signal indicates the terminal object corresponding to the monitor link object. A function to execute processing according to the operation content;
It is a program for realizing.

DESCRIPTION OF SYMBOLS 1 Mobile phone 3 Wireless communication network 4 Internet 5 Information distribution apparatus 6 Display apparatus 6A Television receiver 11 CPU
DESCRIPTION OF SYMBOLS 13 Memory | storage part 15 Wireless communication part 17 Operation part 18 Touch display part 19 Mouse sensor 22 Case direction detection part 23 Drawer detection part DT Display position management table

Claims (3)

A mobile terminal device that processes an object in response to an operation on the object to be processed,
A communication means for performing data communication directly or indirectly with a display device as an external device;
When the object to be processed is a terminal object and an access operation is performed on the terminal object, the terminal object or the processing result is displayed as a monitor link object on the display device via the communication unit. Display control means for performing control to
Management means for storing and managing the display position of the monitor link object on the display device side received and acquired via the communication means in association with the terminal object;
When the display operation signal is received and acquired via the communication means in response to an operation on the display content on the display device side, the display operation signal indicates the terminal object corresponding to the monitor link object. Processing means for executing processing according to the operation content;
With
The communication means is a network communication means for indirectly communicating with the display device via an information distribution device via a network,
The terminal object is link information that specifies the location of data to be displayed,
The display control means acquires the identification information of the display device and notifies the information distribution device of the identification information and the link information, so that the data specified by the link information from the information distribution device is obtained. By transmitting to the display device specified by the identification information, control to display the data as the monitor link object on the display device is performed.
The portable terminal device characterized by the above-mentioned. Determining whether or not the display device capable of communication connection exists in the vicinity of the terminal device when the object to be processed is a terminal object and an access operation is performed on the terminal object Means,
A specifying means for specifying the display device as an output monitor when the determining device determines that the display device exists;
Further comprising
The display control means performs control for displaying the monitor link object on the display device specified as the output monitor via the communication means.
The mobile terminal device according to claim 1, which is configured as described above. A program for controlling a computer of a mobile terminal device that processes an object according to an operation on the object to be processed,
The computer,
A communication means for performing data communication directly or indirectly with a display device as an external device;
When the object to be processed is a terminal object and an access operation is performed on the terminal object, the terminal object or the processing result is displayed as a monitor link object on the display device via the communication unit. Display control means for performing control for
Management means for storing and managing the display position of the monitor link object on the display device side received and acquired via the communication means in association with the terminal object;
When the display operation signal is received and acquired via the communication means in response to an operation on the display content on the display device side, the display operation signal indicates the terminal object corresponding to the monitor link object. Processing means for executing processing according to the operation content;
Function as
The communication means is a network communication means for indirectly communicating with the display device via an information distribution device via a network,
The terminal object is link information that specifies the location of data to be displayed,
The display control means acquires the identification information of the display device and notifies the information distribution device of the identification information and the link information, so that the data specified by the link information from the information distribution device is obtained. A computer-readable program for performing control for causing the display device to display the data as the monitor link object by transmitting the data to the display device specified by the identification information.

Images