JP6577731B2 - Terminal device, display control method, and program - Google Patents

Description

  The present invention relates to a technique for displaying a screen of content such as a web page on a user terminal such as a smartphone.

  Today, mobile terminals such as smartphones are widespread. For example, it is possible to display and browse various content screens on a network using a mobile terminal using a browser generally installed in the mobile terminal.

  In addition to the browser, various applications can be mounted on the mobile terminal. The user can use various services of the application by performing operations on the screen while viewing the screen of the application displayed on the touch panel of the mobile terminal.

JP 2004266668 A

  As described above, the user can browse the content displayed on the mobile terminal and can operate the application on the screen displayed on the mobile terminal.

  However, the size of the screen of the mobile terminal is smaller than the size of the screen of a PC or the like. For example, when content is displayed by a browser on a mobile terminal, a great amount of information is displayed on a limited screen. Therefore, characters, icons, diagrams, etc. are too small to be seen easily and cannot be operated easily. In such a case, if the screen is enlarged, characters and the like are easy to see and operate, but on the other hand, it is impossible to list the entire screen, and the display content confirmation and operation are inefficient. Become.

  The present invention has been made in view of the above points, and provides a technique that enables a user to efficiently browse and operate display information even when a terminal having a small screen such as a portable terminal is used. The purpose is to do.

According to the embodiment of the present invention, connection means for connecting to the display device by wire or wirelessly,
Display means for performing screen display;
Acquisition means for acquiring content display screen information;
Based on the display screen information, first screen information corresponding to a first screen displayed on the display means and second screen information corresponding to a second screen displayed on the display device are created. , Displaying the first screen on the display means using the first screen information, and simultaneously transmitting the second screen information to the display device, thereby displaying the second screen on the display device. A display control means for displaying the terminal device ,
The display control means includes
Screen information for causing the display device to display a screen of the content is created as the second screen information, and designation in the content displayed by the second screen information as the first screen information Screen information for displaying the displayed portion on the display means is created.
A terminal device is provided.

Further, according to the embodiment of the present invention, there is provided a display control method executed by a terminal device including a connection unit that connects to a display device by wire or wirelessly, and a display unit that performs screen display.
An acquisition step of acquiring content display screen information;
Based on the display screen information, first screen information corresponding to a first screen displayed on the display means and second screen information corresponding to a second screen displayed on the display device are created. , Displaying the first screen on the display means using the first screen information, and simultaneously transmitting the second screen information to the display device, thereby displaying the second screen on the display device. a display control method and a display control step of displaying,
In the display control step, the terminal device
Screen information for causing the display device to display a screen of the content is created as the second screen information, and designation in the content displayed by the second screen information as the first screen information Screen information for displaying the displayed portion on the display means is created.
A display control method is provided.

  The embodiment of the present invention provides a technique that allows a user to efficiently browse and operate display information even when a terminal having a small screen such as a portable terminal is used.

It is a block diagram of the communication system which concerns on embodiment of this invention. 2 is a configuration diagram of a user terminal 10. FIG. It is a figure which shows the example of the information stored in the setting information storage part. It is a figure for demonstrating a display position and a display size. It is a figure for demonstrating the flow of the process for a screen display. It is a figure for demonstrating the example 1 of a display (user terminal mode). It is a figure for demonstrating the example 2 of a display (display apparatus mode). It is a figure for demonstrating the example 3 of a display (display apparatus mode).

  Embodiments of the present invention will be described below with reference to the drawings. The embodiment described below is only an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

  For example, in the following embodiment, it is assumed that a smartphone is used as a user terminal (terminal device) and a television or the like is used as a display device. However, the user terminal is not limited to a smartphone, and the display device is a television. Not limited. In the following embodiments, it is assumed that an application that performs screen display is a browser, but an application to which the present invention can be applied is not limited to a browser.

(System configuration)
FIG. 1 shows a configuration diagram of a communication system according to an embodiment of the present invention. As shown in FIG. 1, the communication system includes a user terminal 10 and a display device 20. The user terminal 10 is, for example, a smartphone, and includes a function of wirelessly connecting to the network 40, acquiring content from the content providing apparatus 30 such as a Web server via the network 40, and displaying a content screen.

  In the present embodiment, a display device 20 is provided and connected to the user terminal 10. The connection may be wireless or wired. The display device 20 is, for example, a general television, but may be any device as long as it can display a screen having a size larger than the screen size that can be displayed by the user terminal 10.

  In the present embodiment, a screen is displayed on the user terminal 10 based on content display screen information created (or acquired) on the user terminal 10, and display screen information is sent from the user terminal 10 to the display device 20. As a result, the screen of the content is also displayed on the display device 20.

  The display screen information transmitted from the user terminal 10 to the display device 20 may be bit information for each pixel (that is, image data itself), and is interpreted when screen display is performed on the display device 20. It may be a (drawing command) or other type of information. In the case of using a method of transmitting display screen information as a drawing command, it is assumed that the display device 20 includes a CPU that interprets the drawing command, a graphic processor, and the like.

(Configuration example of user terminal 10)
FIG. 2 shows a configuration diagram of user terminal 10 according to the present embodiment. As shown in FIG. 2, the user terminal 10 in the present embodiment includes an operation / display unit 11, a display screen information creation unit 12, a display control unit 13, and a control information storage unit 14. Note that FIG. 2 shows only functional units that are particularly related to the embodiment of the present invention in the user terminal 10. For example, if the user terminal 10 is a smartphone, a function (not shown) for operating as a smartphone It also has. The functional configuration shown in FIG. 2 is only an example. As long as the operation of the user terminal 10 according to the present embodiment can be executed, the function division and the name of the function unit may be anything.

  The operation / display unit 11 is, for example, a touch panel, and is a functional unit that displays a screen (image) to a user and enables an operation on the display screen. The display screen information creation unit 12 corresponds to an application that performs screen display, such as a browser, and creates display screen information that is the basis of the display screen displayed on the operation / display unit 11. The display screen information is passed to the display control unit 13.

  For example, when the display screen information creation unit 12 is a browser, the display screen information creation unit 12 receives content (HTML data) from the content providing apparatus 30 via the network 40, and analyzes the HTML data to thereby generate the HTML. Display screen information that is the basis of the content display screen corresponding to the data is created. For example, when the display screen information is image data (RGB data), the display screen information is given to the operation / display unit 11 from the display control unit 13 to display the content on the screen. For example, when the display screen information is information on the previous stage of image data such as a drawing command, image data is created from the drawing command and displayed by the image processing function of the operation / display unit 11, for example.

  Further, the display screen information creation unit 12 receives information on the operation performed on the operation / display unit 11 on which the screen is displayed, and the display screen information after the change by the operation (eg, enlargement, reduction, movement) , Input character display, etc.) are created and passed to the display control unit 13.

  The display control unit 13 includes a buffer that temporarily holds the display screen information received from the display screen information creation unit 12. The display control unit 13 is based on the setting information stored in the setting information storage unit 14, and from the display screen information, the user terminal screen information used for screen display on the user terminal 10 and the display device used for screen display on the display device 20. Screen information is created, and the user terminal screen information is passed to the operation / display unit 11 for display, and the display device screen information is transmitted to the display device 20 for display.

  Note that when the assumed application is a browser, the display control unit 13 may be configured to include all or part of the functions of the browser. In that case, the display screen information creation unit 12 passes the content (HTML data) received (acquired) from the network 40 to the display control unit 13 as display screen information, and the display control unit 13 analyzes the content and the user terminal screen information. / Create display device screen information.

  FIG. 3 shows an example of setting information stored in the setting information storage unit 14. The example of the setting information illustrated in FIG. 3A is the display position and the display size of the display device 20 and the user terminal 10, respectively. FIG. 4 is a diagram for explaining the display position and the display size. In the present embodiment, the entire content screen related to the display screen information created by the display screen information creation unit 12 has a size (eg, horizontal X pixel, vertical Y pixel) specified in HTML data, for example. The display position is information indicating which part of the entire screen is to be displayed on the corresponding terminal / device. For example, the display position is represented by pixel coordinates from a predetermined position (eg, upper left corner) of the entire content screen. Can do. The display size indicates which size portion of the content is displayed on the corresponding terminal / device.

  The display position / display size information is changed based on a user operation, or a predetermined value is held. Details will be described later.

  In the present embodiment, there are a user terminal mode in which an operation is performed on a screen displayed on the user terminal 10 and a display device mode in which an operation is performed on a screen displayed on the display device 20. The user can switch modes by, for example, an operation on the user terminal 10. The operation is, for example, pressing (touching) a predetermined button. As shown in FIG. 3B, the current mode is held as setting information.

  In each mode, various operations are possible depending on the setting, and the contents of the setting are stored in the setting information storage unit 14 as shown in FIG. The setting contents are set in advance.

  The setting information shown in FIGS. 3A to 3C is only an example. For example, in the example shown in FIG. 3C, the setting content is held for each mode, but one setting content used in common in all modes may be held.

  The user terminal 10 according to the present embodiment is described in the present embodiment, for example, in a computer (including a communication device such as a smartphone) provided with an application such as a browser and a means for connecting to the display device 20 by wire or wireless. This can be realized by executing a program describing the processing contents of the display control unit 13 to be executed. In other words, the function of the display control unit 13 of the user terminal 10 is a program corresponding to processing executed in the user terminal 10 using hardware resources such as a CPU (processor), memory, and hard disk built in the computer. It is possible to implement by executing. The above-mentioned program can be recorded on a computer-readable recording medium (portable memory or the like), stored, or distributed. It is also possible to provide the program through a network such as the Internet or electronic mail.

(Processing flow for screen display)
An example of the flow of processing for screen display will be described with reference to FIG. FIG. 5 shows an example in which the user terminal 10 includes a browser as the display screen information creation unit 12 and displays content received from the network 40.

  When the display screen information creation unit 12 receives content from the network 40, data processing such as analysis of HTML data and display screen information creation is performed (step S101), and the display screen information is stored in a buffer of the display control unit 13 (step S101). S102).

  The display control unit 13 displays the user terminal screen information for display on the user terminal 10 and the display device for display on the display device 20 based on the designated mode (user terminal mode / display device mode). By creating screen information, transmitting user terminal screen information to the operation / display unit 11, and transmitting display device screen information to the display device 20, a content screen is displayed on each of the user terminal 10 and the display device 20. The In addition, displaying the content screen on each of the user terminal 10 and the display device 20 is an example, and the content screen may not be displayed on the user terminal 10 depending on the mode.

(Display example 1)
FIG. 6 shows an example of screen display in the user terminal mode. Basically, in the user terminal mode, the display device 20 displays the full content of the display screen so that the entire screen can be viewed, for example, by scrolling in the vertical direction. A part of the content is displayed on the user terminal 10. In the example of FIG. 6, it is shown that a portion indicated by a dotted line in the content is displayed on the user terminal 10.

  With regard to the processing operation of this example, for example, as the setting contents shown in FIG. 3C, “user terminal: position A, enlarged display at X times, display device: position B, entire display” is set for the user terminal mode. Has been. These values are initial values. Further, “entire display” is an example, and a display size that is a part of the content may be set. In that case, the part of the content indicated by the display size is displayed first.

  According to the above setting, information corresponding to “position B, entire display” is stored in the “display device” field of FIG. 3A, and the display control unit 13 Display screen information for displaying the content screen at position B is transmitted.

  Further, in accordance with the above setting, the position A is stored in the display position in the “user terminal” field of FIG. As for the display size, for example, when the entire width of the content screen is displayed in the full width of the screen of the user terminal, the magnification is set to 1 and the displayed content screen is X times (vertically and horizontally each X times). The size (number of vertical and horizontal pixels) on the content screen displayed on the user terminal is stored.

  The display control unit 13 refers to the above information, creates user terminal screen information corresponding to the content portion to be displayed on the user terminal 10, and performs display via the operation / display unit 11.

  The display shown in FIG. 6 is realized by the processing as described above. Also, by referring to the display position / display size for the user terminal stored in FIG. 3A, the display control unit 13 determines where the display screen on the user terminal 10 corresponds to the whole. Display device screen information including information to be displayed (eg, dotted line frame) may be created. Thereby, the dotted line frame shown in FIG. 6 can be displayed on the display device 20.

  In the user terminal mode, the user enlarges, reduces, or moves the screen (display content) displayed on the user terminal 10 by performing an operation on the screen while watching the display screen of the user terminal 10, for example. be able to. For example, an operation of enlarging the screen display is performed by “pinch out” on the screen. The operation content is transmitted from the operation / display unit 11 to the display control unit 13, and the display position / display size of the content to be displayed on the user terminal 10 is determined according to the operation content (operation amount or the like). (A) is stored as updated setting information. In addition, since a small part in the content is displayed larger by enlargement, the “display size” is a small value. The content portion of this display size is displayed on the full screen of the user terminal 10.

  Based on the setting information updated as described above, user terminal screen information is created and displayed. Further, when a dotted line frame is displayed as shown in FIG. 6, display device screen information including the updated dotted line frame is created based on the updated setting information, transmitted to the display device 20, and displayed. Note that the dotted frame is reduced by the enlargement operation.

  The same processing as described above is performed for reduction, movement, etc., and screen display is performed. In the user terminal mode, by scrolling up and down on the user terminal 10, for example, when displaying a portion of content that is not displayed on the display device 10, the display control unit 13 performs the scroll operation. The movement amount on the corresponding content is acquired, and the “display position” in the display device in FIG. 3A is changed by the movement amount using the movement amount, and the display device screen corresponding to the changed display position By creating information and transmitting it to the display device 20, the scrolled content is displayed on the display device 20.

  In the user terminal mode, information can be input on the content screen displayed on the user terminal 10. For example, on the user terminal 10, when an input field in the content is enlarged and displayed, and information is input to the input field, the information is transferred from the operation / display unit 11 to the display screen information creating unit 12, and the display screen information is displayed. Is created and passed to the display control unit 13. The display control unit 13 uses the display screen information, for example, to change only the changed image information for each of the currently displayed user terminal screen information and display device screen information, and to change the user terminal Display is performed using screen information and display device screen information. Thereby, the content of the input operation is displayed on each of the user terminal 10 and the display device 20.

  In response to the input, the input information is transmitted from the display screen information creation unit 12 to the content providing apparatus 30 via the network 40. Then, the response content is returned to the display screen information creation unit 12. After that, display on the user terminal 10 / display device 20 can be performed as in the case of the initial content screen display.

(Display Example 2, Display Example 3)
Next, display examples 2 and 3 will be described. In display examples 2 and 3, display in the display device mode will be described.

  Regarding the processing operation of this example, for example, as the preset setting contents shown in FIG. 3C, “user terminal: position A, overall display” and “display device: position A, overall display” are set for the display device mode. "Is set. These indicate initial values. As in the case described above, “entire display” is an example, and a display size that is a part of the content may be set. In that case, the part of the content indicated by the display size is displayed first.

  According to the above setting, information corresponding to “position A, entire display” is stored in the “display device” field of FIG. 3A, and “position A, entire display” is also stored in the “user terminal” field. Is stored.

  The display control unit 13 refers to the above information, and displays user terminal screen information for displaying the entire content screen (for example, the entire horizontal direction) on the user terminal 10 and the entire content screen (for example, the entire horizontal direction). ) Is displayed on the display device 20, and the information is displayed on the user terminal / display device.

  As a result, for example, the display shown in FIG. 7 is performed. As shown in FIG. 7, the entire content screen is displayed on both the display device 20 and the user terminal 10. However, since the screen of the user terminal 10 is small, it becomes a reduced screen.

  In the display device mode, the user mainly enlarges (eg, pinch out) or reduces (pinch in) the screen (display content) displayed on the user terminal 10 while viewing the display screen of the display device 20, for example. ), Moving (flicking, dragging, etc.) and scrolling operations, the display screen of the display device 20 can be enlarged, reduced, moved, and scrolled. At this time, for example, the same screen as the screen displayed on the display device 20 is displayed on the user terminal 10.

  As an example, it is assumed that the user performs an operation of enlarging the screen by “pinch out” on the user terminal 10 while mainly viewing the screen of the display device 20. The operation content is transmitted from the operation / display unit 11 to the display control unit 13, and the display position / display size of the content displayed on the display device 20 / user terminal 10 is determined according to the operation content (operation amount, etc.). This is stored as the updated setting information in FIG. In addition, since a small part in the content is displayed larger by enlargement, the display size is smaller than the original display size (content size).

  Based on the setting information updated as described above, display screen information and user terminal screen information are created and displayed. As a result, the updated content portion of the display size is displayed on the screen of each of the display device 20 and the user terminal 10. The same processing as described above is performed for reduction, movement, scrolling, etc., and screen display is performed.

  In the above example, in the display device mode, the same screen as the screen displayed on the display device 20 is displayed on the user terminal 10 and the operation is performed on the screen of the user terminal 10, but this is only an example. Absent.

  For example, without displaying the content screen on the user terminal 10, the user terminal 10 may be used as a “remote control” for performing an operation on the screen displayed on the display device 20.

  In this case, for example, “user terminal: remote control” and “display device: position A, entire display” are set for the display device mode as preset setting contents shown in FIG.

  The display control unit 13 refers to the above information, creates display device screen information for displaying the content screen at the position A on the display device 20, and thereby displays the display device 20 on the display device 20. Further, for example, the display control unit 13 creates and displays user terminal screen information for displaying a remote control screen on the user terminal 10.

  In an example in which the user terminal 10 is used as a remote control, the user enlarges, reduces, moves, or scrolls the remote control screen (display content) displayed on the user terminal 10 while viewing the display screen of the display device 20, for example. By performing this operation, the display screen of the display device 20 can be enlarged, reduced, moved, and scrolled.

  As an example, it is assumed that the user performs a remote control operation for enlarging the screen while viewing the screen of the display device 20. The operation content is transmitted from the operation / display unit 11 to the display control unit 13, and the display position / display size of the content to be displayed on the display device 20 is determined according to the operation content, and this is updated in FIG. Stored as later setting information.

  Display device screen information is created and displayed based on the setting information updated as described above. Thereby, the updated content portion of the display size is displayed on the display device 20. The same processing as described above is performed for reduction, movement, etc., and screen display is performed.

  Further, in the display device mode, information can be input on the content screen displayed on the display device 20 by operating the user terminal 10. For example, when the display shown in FIG. 7 is performed, the user touches the input field in the content displayed on the display device 20 on the user terminal 10 to enter the input mode. Accordingly, for example, as illustrated in FIG. 8, the input interface screen is displayed on the user terminal 10 based on the control by the display control unit 13. Information input from the screen is transferred from the operation / display unit 11 to the display screen information creation unit 12, and display screen information is created and passed to the display control unit 13. The display control unit 13 uses the display screen information to change only the image information that has been changed, for example, the currently displayed display device screen information, and performs display using the changed display device screen information. . Thereby, the content of the input operation is displayed on the display device 20.

  In response to the input, the input information is transmitted from the display screen information creation unit 12 to the content providing apparatus 30 via the network 40. Then, the response content is returned to the display screen information creation unit 12. After that, display on the user terminal 10 / display device 20 can be performed as in the case of the initial content screen display.

(Summary of embodiment)
As described above, according to the embodiment of the present invention, connection means for connecting to a display device by wire or wireless, display means for performing screen display, acquisition means for acquiring display screen information of content, and the display Based on the screen information, the first screen information corresponding to the first screen to be displayed on the display means and the second screen information corresponding to the second screen to be displayed on the display device are created, Display control for displaying the first screen on the display unit using the first screen information, and displaying the second screen on the display device by transmitting the second screen information to the display device. A terminal device is provided.

  The display control means creates screen information for displaying the content screen on the display device as the second screen information, and is displayed by the second screen information as the first screen information. Screen information for displaying a part of the content to be displayed on the display means may be created.

  The display control unit may display information indicating a part of the content displayed on the display unit on the display device.

  In the case where the first mode is designated, the display control means displays the screen after being changed by the operation when the operation is performed on the first screen displayed by the display means. You may comprise so that it may be displayed on a means.

  In the case where the second mode is designated, the display control means is changed by the operation when the terminal device is operated to change the second screen displayed by the display device. It is good also as displaying the screen after being done on the said display apparatus.

(Effects of the embodiment, etc.)
According to the present embodiment, a display device 20 is connected to the user terminal 10, and a screen obtained by cutting out a part of the content screen displayed on the display device 20 while displaying the content screen on the display device 20 is displayed on the user terminal 10. Can be operated on the user terminal 10, so that detailed operations can be performed while grasping a wide range. Note that the screen displayed by the user terminal 10 is not limited to a part of the screen displayed on the display device 20, and a screen that is completely different from the screen displayed on the display device 20 may be displayed. Moreover, since it can also operate on the display apparatus 20 using the user terminal 10, a detailed operation can be performed also by this, grasping | ascertaining a wide range. That is, the user can efficiently browse and operate the display information.

(Section 1)
Connection means for connecting to the display device by wire or wirelessly;
Display means for performing screen display;
Acquisition means for acquiring content display screen information;
Based on the display screen information, first screen information corresponding to a first screen displayed on the display means and second screen information corresponding to a second screen displayed on the display device are created. The first screen information is used to display the first screen on the display means, and the second screen information is transmitted to the display device to display the second screen on the display device. Display control means and
A terminal device comprising:
(Section 2)
The display control means includes
Screen information for displaying the content screen on the display device is created as the second screen information, and a part of the content displayed by the second screen information is created as the first screen information. Create screen information to be displayed on the display means
The terminal device according to item 1.
(Section 3)
The display control means causes the display device to display information indicating a part of the content displayed on the display means.
The terminal device according to item 2.
(Section 4)
The display control means includes
In the case where the first mode is designated, when an operation is performed on the first screen displayed by the display means, the screen after the change by the operation is displayed on the display means.
The terminal device according to any one of Items 1 to 3.
(Section 5)
The display control means includes
When the second mode is designated and the terminal device is operated to change the second screen displayed by the display device, the screen changed by the operation is displayed. Display on the display device
The terminal device according to any one of Items 1 to 4.
(Section 6)
A display control method executed by a terminal device including a connection unit that connects to a display device by wire or wirelessly and a display unit that performs screen display,
An acquisition step of acquiring content display screen information;
Based on the display screen information, first screen information corresponding to a first screen displayed on the display means and second screen information corresponding to a second screen displayed on the display device are created. The first screen information is used to display the first screen on the display means, and the second screen information is transmitted to the display device to display the second screen on the display device. Display control step and
A display control method comprising:
(Section 7)
A program for causing a computer to function as display control means in the terminal device according to any one of items 1 to 5.
The present invention is not limited to the above-described embodiments, and various modifications and applications are possible within the scope of the claims.

DESCRIPTION OF SYMBOLS 10 User terminal 11 Operation / display part 12 Display screen information creation part 13 Display control part 14 Control information storage part 20 Display apparatus 30 Content provision apparatus 40 Network

Claims (6)

Connection means for connecting to the display device by wire or wirelessly;
Display means for performing screen display;
Acquisition means for acquiring content display screen information;
Based on the display screen information, first screen information corresponding to a first screen displayed on the display means and second screen information corresponding to a second screen displayed on the display device are created. , Displaying the first screen on the display means using the first screen information, and simultaneously transmitting the second screen information to the display device, thereby displaying the second screen on the display device. A display control means for displaying the terminal device ,
The display control means includes
Screen information for causing the display device to display a screen of the content is created as the second screen information, and designation in the content displayed by the second screen information as the first screen information Screen information for displaying the displayed portion on the display means is created.
Terminal device. The terminal device according to claim 1 , wherein the display control unit causes the display device to display information indicating a portion of the content displayed on the display unit. The display control means includes
When the first mode is designated, when an operation is performed on the first screen displayed by the display unit, the screen after the change by the operation is displayed on the display unit. The terminal device according to 1 or 2 . The display control means includes
When the second mode is designated and the terminal device is operated to change the second screen displayed by the display device, the screen changed by the operation is displayed. the display terminal device according to any one of the apparatus claims 1 to be displayed on 3. A display control method executed by a terminal device including a connection unit that connects to a display device by wire or wirelessly and a display unit that performs screen display,
An acquisition step of acquiring content display screen information;
Based on the display screen information, first screen information corresponding to a first screen displayed on the display means and second screen information corresponding to a second screen displayed on the display device are created. , Displaying the first screen on the display means using the first screen information, and simultaneously transmitting the second screen information to the display device, thereby displaying the second screen on the display device. a display control method and a display control step of displaying,
In the display control step, the terminal device
Screen information for causing the display device to display a screen of the content is created as the second screen information, and designation in the content displayed by the second screen information as the first screen information Screen information for displaying the displayed portion on the display means is created.
Display control method. The program for functioning a computer as a display control means in the terminal device of any one of Claims 1 thru | or 4 .

Images