JP2019175460A - Display controller, display control method, and control program - Google Patents

Abstract

To provide a display controller that allows a GUI image to be displayed in a state suitable for manipulation.SOLUTION: A display controller 100 comprises: an acquisition unit 110 for acquiring display information that includes structure information that defines a plurality of display elements to be displayed on a screen for GUI and indicates an inclusion relationship between the display elements, and screen information that indicates the form of a target screen on which a GUI image is displayed; a processing unit 130 for determining the arrangement and display mode of each of the display elements of the GUI image on the basis of information for arrangement determination in which the arrangement of the display elements of the GUI image is defined, size constraint information in which size requirements for the display elements of the GUI image for the assurance of operability are defined, and the display information, so that the display elements of the GUI image fit within the target screen in according with the form of the target screen indicated by the screen information and requirements for the display elements for which requirements are defined in the size constraint information are satisfied; and an output unit 140 for performing output corresponding to the determination result of the processing unit 130.SELECTED DRAWING: Figure 2

Description

  The present disclosure relates to a technique for controlling display of an image related to a GUI (Graphical User Interface).

  Conventionally, when a label or the like as a GUI component cannot be displayed on a display having a small screen size, the label or the like is displayed by display control using a display control rule such as scrolling the label or reducing the font of the label. An information processing apparatus that displays on the display is known (see, for example, Patent Document 1).

JP 2007-12037 A

  In the conventional display control technique, when displaying a GUI image (that is, an image as a GUI) that is a user's operation target when a relatively small display is a display target, the image is always displayed in a state suitable for the operation. Not necessarily.

  The present disclosure provides a display control device that displays a GUI image in a state suitable for operation. The present disclosure also provides a display control method used in the display control device and a control program that is a computer program for display control.

  The display control device according to the present disclosure includes display information that defines a plurality of display elements to be displayed on a screen used as a GUI (Graphical User Interface) and includes structure information that indicates an inclusion relationship between the plurality of display elements. An acquisition unit that acquires display information and screen information indicating a form of a target screen that is a screen for displaying the plurality of display elements, and a predetermined arrangement of one or more display elements among the plurality of display elements. Information for determining the arrangement, size constraint information in which requirements regarding the size of each of the display elements of the plurality of display elements in order to ensure operability are predetermined, and the display information The plurality of display elements fits in the target screen according to the form of the target screen indicated by the screen information, and the size of the plurality of display elements For a display element for which the requirement is defined by about information, an arrangement determination process for determining the arrangement of each of the plurality of display elements and a display mode for determining the display mode of each of the plurality of display elements so that the requirement is satisfied. A first processing unit that executes a determination process; and an output unit that performs output according to the results of the placement determination process and the display mode determination process performed by the first processing unit.

  The display control method according to the present disclosure is a display control method used in a display control device, and is display information that defines a plurality of display elements to be displayed on a screen used as a GUI (Graphical User Interface). An acquisition step for acquiring the display information including structural information indicating an inclusion relationship between a plurality of display elements, and screen information indicating a form of a target screen that is a screen for displaying the plurality of display elements, and the plurality of displays Predetermined information for determining the arrangement of one or more display elements among the elements, and a requirement regarding the size of each of the display elements of the one or more display elements in order to ensure operability. Based on the predetermined size constraint information and the display information, the target screen includes the target screen according to the form of the target screen. The arrangement of each of the plurality of display elements is determined so that the number of display elements can be accommodated and the requirement is satisfied for the display elements for which the requirement is defined by the size constraint information among the plurality of display elements. A processing step for executing a layout determination process and a display mode determination process for determining the display mode of each of the plurality of display elements, and an output corresponding to the results of the layout determination process and the display mode determination process in the processing step Output step to be performed.

  The control program in the present disclosure is a control program for causing a display control device including a microprocessor to execute display control processing, and the display control processing is displayed on a screen used as a GUI (Graphical User Interface). Display information that is defined for a plurality of display elements to be performed and includes display information including structure information indicating an inclusion relationship between the plurality of display elements, and a form of a target screen that is a screen for displaying the plurality of display elements An acquisition step for acquiring screen information to be displayed; information for determining a predetermined arrangement for the arrangement of one or more display elements among the plurality of display elements; and a plurality of display elements for ensuring operability. Based on the size constraint information in which the requirements regarding the size in the display mode of each of the one or more display elements are predetermined, and the display information Regarding the display element in which the plurality of display elements are accommodated in the target screen according to the form of the target screen indicated by the screen information, and the requirement is defined by the size constraint information among the plurality of display elements. A processing step for executing an arrangement determining process for determining an arrangement of each of the plurality of display elements and a display mode determining process for determining a display mode of each of the plurality of display elements so that the requirement is satisfied; And an output step of performing output in accordance with the result of the arrangement determination process and the display mode determination process.

  According to the display control device, the display control method, or the control program in the present disclosure, the GUI image is displayed in a state suitable for a user operation (that is, a state in which user operability is ensured).

1 is a diagram illustrating a configuration example of a system including a display control device according to a first embodiment. 3 is a block diagram illustrating a functional configuration of the display control apparatus according to Embodiment 1. FIG. It is a figure which shows an example of the display information which a display control apparatus acquires. It is a figure which shows an example of the screen information which a display control apparatus acquires. It is a figure which shows an example of the information for arrangement | positioning determination which a display control apparatus memorize | stores. It is a figure which shows another example of the information for arrangement | positioning determination which a display control apparatus memorize | stores. It is a figure which shows an example of the size constraint information which a display control apparatus memorize | stores. It is a flowchart which shows an example of the display control process in a display control apparatus. It is a flowchart which shows an example of the display determination process as a part of display control process. It is a flowchart which shows an example (operation example 1) of the arrangement | positioning determination process as a part of display determination process. It is a flowchart which shows an example (operation example 2) of the arrangement | positioning determination process as a part of display determination process. It is a figure which shows the example of arrangement | positioning of the several display element which comprises the GUI image before the completion of a display determination process. It is a figure which shows the example of arrangement | positioning of the some display element which comprises a GUI image after the completion of a display determination process. 6 is a diagram illustrating a configuration example of a system including a display control apparatus according to Embodiment 2. FIG. It is a block diagram of the display control apparatus which concerns on this Embodiment. It is a flowchart which shows an example of the display control process as a display control method in a display control apparatus. It is a figure which shows the example of arrangement | positioning of the several display element which comprises the GUI image in FIG.

  In order to display a GUI image in a state suitable for a user operation according to the size of a display to be displayed, a display control device according to the present disclosure has a plurality of items to be displayed on a screen used as a GUI (Graphical User Interface). Display information stipulated with respect to the display elements, including the display information including structure information indicating an inclusion relationship between the plurality of display elements, and screen information indicating a form of a target screen that is a screen for displaying the plurality of display elements; An acquisition unit for acquiring the information, predetermined arrangement determination information about the arrangement of one or more display elements among the plurality of display elements, and one or more of the plurality of display elements to ensure operability The form of the target screen indicated by the screen information based on the size constraint information in which the requirements regarding the size in the display mode of each display element are predetermined and the display information Accordingly, the plurality of display elements are accommodated in the target screen, and the plurality of display elements are set such that the requirements are satisfied for the display elements for which the requirements are defined by the size constraint information. A first processing unit that executes a layout determination process for determining a layout of each display element and a display mode determination process for determining a display mode of each of the plurality of display elements; the layout determination process by the first processing unit; and An output unit that performs output according to the result of the display mode determination process.

  As a result, even if the size of the screen used as the GUI is relatively small, an appropriate arrangement or the like of each display element constituting the GUI image is determined by referring to the screen information, size constraint information, and the like. Therefore, the GUI image is displayed in a state suitable for the user's operation, that is, in a state where the user's operability is ensured.

  Here, for example, the structure information is expressed by associating each of the plurality of display elements with each node in a tree structure of a plurality of hierarchies, and associating the display element having the largest display area with the highest root node, It is information in which nodes corresponding to one or more display elements to be included in a display area of a display element corresponding to a relatively higher node are arranged under the higher node, and the first processing unit is As a result of executing the provisional arrangement process for temporarily determining the arrangement for the plurality of display elements so as to satisfy the requirement defined in the size constraint information based on the display information and executing the display mode determination process. For one or more display elements of the plurality of display elements, if the display element does not fit within the target screen, the display element fits within the target screen. As described above, when the display element does not fit in the display area related to the node higher than the node corresponding to the display element in the tree structure, the placement determination process is performed so that the display element fits in the display area. It is good also as performing.

  As a result, it becomes possible to cope with the case where each display element constituting the GUI image does not fit in the area to be accommodated depending on the existing arrangement technology, etc., and each display of the GUI image so as to ensure the operability of the user. The arrangement of elements can be appropriately determined.

  In addition, for example, the first processing unit once determines the arrangement of the plurality of display elements and executes the display mode determination process, so that one display element of the plurality of display elements is the target screen. If it does not fit within the display area, or if it does not fit within the display area of the node corresponding to the one display element in the tree structure of the structure information, the one display element It is also possible to change the relative arrangement between the display elements corresponding to the nodes immediately below the node that is one level higher than the node in the tree structure corresponding to.

  As a result, the relative arrangement of the plurality of display elements that should be accommodated in a certain display area is adjusted, so that an appropriate GUI image can be formed.

  Further, for example, the placement determination information includes first placement information indicating the placement of each of the plurality of display elements to be determined in the provisional placement processing, and immediately below one node in the tree structure of the structure information. Each of the display elements corresponding to each node includes second arrangement information indicating a different arrangement from the arrangement indicated by the first arrangement information, and the first processing unit executes the provisional arrangement processing according to the first arrangement information, The arrangement determination process may be executed according to the second arrangement information.

  As a result, the first arrangement information prepared in advance by a designer or the like and the second arrangement information for replacement thereof are used based on the assumption that there are various sizes of screens used as a GUI. It is possible to provide a GUI image composed of a plurality of display elements arranged appropriately.

  In addition, for example, the arrangement determination information includes arrangement constraint information in which requirements regarding relative arrangement of all or some of the plurality of display elements are determined in advance, and the first processing unit includes the arrangement constraint information The provisional arrangement process and the arrangement determination process may be executed so as to satisfy the requirement defined by the information.

  As a result, when there is an appropriate relative arrangement (that is, relative position) for improving operability among a plurality of display elements (for example, a plurality of buttons that are functionally closely related such as an execution button and a stop button) Since the arrangement constraint information related to the relative arrangement and the like is determined and used in the case where the objects should be arranged close to each other, it is possible to provide a GUI image that ensures operability even if the size of the target screen is small.

  In addition, for example, the first processing unit once determines the arrangement of the plurality of display elements and executes the display mode determination process, so that one display element of the plurality of display elements is the target screen. The tree structure corresponding to the one display element if it does not fit within the display area of the node corresponding to the one display element in the tree structure of the structure information. The width and height of the display area in which the display elements corresponding to all the nodes immediately below the node one level higher in the node are to be stored are identified, and the display elements to be stored in the display area are determined based on the specific result. It is good also as performing the said arrangement | positioning determination process so that it may accommodate.

  This makes it possible to use the information on the width and height of the area that should be clear when it is determined that the display element does not fit in the area to be contained, and to determine the arrangement of the display element again. It is possible to determine an appropriate arrangement.

  In addition, for example, the first processing unit once determines the arrangement of the plurality of display elements and executes the display mode determination process, so that one display element of the plurality of display elements is the target screen. In the tree structure corresponding to the one display element in the case where it does not fit in the display area related to the node higher than the node corresponding to the one display element in the tree structure of the structure information When each display element corresponding to two or more nodes immediately below a node one level higher than the node is arranged in a matrix, the width of the display area in which each display element arranged in the matrix is to be stored; The height is specified, and the number of rows and the number of columns in the matrix-like arrangement is changed so that the display elements to be stored in the display area are stored based on the specific result. It is, may execute the layout determination processing.

  Thus, by appropriately changing the number of rows and the number of columns for the display elements arranged in a matrix, it is possible to appropriately maintain the operability and realize an appropriate arrangement of the display elements.

  Further, for example, the size constraint information is information that defines a minimum size of each display element as the requirement regarding the size in the display mode of each of one or more display elements among the plurality of display elements. The processing unit executes the display mode determination process at least after execution of the arrangement determination process, and the first processing unit executes the plurality of displays in the display mode determination process executed after execution of the arrangement determination process. For display elements whose minimum size is defined as the requirement in the size constraint information among the elements, the display element is included in a display area related to a node above the node corresponding to the display element in the tree structure of the structure information. If it can be displayed in a size larger than the minimum size within the range that fits, the display mode of the display element is determined to be the large size If the display element cannot be displayed in a size larger than the minimum size within the display area related to the node above the node corresponding to the display element in the tree structure of the structure information, the minimum size is set. In this way, the display mode of the display element may be determined.

  Thereby, after changing the arrangement of the display elements constituting the GUI image, the size is adjusted so that the size can be easily operated by the arrangement, so that a GUI image with high operability can be provided.

  In addition, for example, in the display mode determination process executed after execution of the arrangement determination process, the first processing unit displays a minimum size defined as the requirement in the size constraint information among the plurality of display elements. For the element, if the display element can be displayed in a size larger than the minimum size within the display area related to the display area related to the node higher than the node corresponding to the display element in the tree structure of the structure information, It is good also as determining the display mode of the said display element so that it may become the maximum size.

  In general, the GUI button is easier to operate when it is larger than the smaller one, so that it is possible to provide a GUI image with sufficiently high operability.

  Further, for example, the display information includes display mode information that specifies a display mode of each of one or more display elements among the plurality of display elements, and the first processing unit includes the display information among the plurality of display elements. For the display element whose display mode is specified by the mode information, the provisional arrangement process may be executed and the display mode determination process may be executed so as to comply with the specification.

  Thereby, in the case where the display mode for the display elements constituting the GUI image is designated, the placement according to the designation is first tried, and if that is not sufficient, the placement change is performed again. It is possible to efficiently determine the arrangement according to the necessity of the arrangement change based on the size of the target screen.

  Further, for example, as a result of executing the provisional arrangement process and the display mode determination process, the first processing unit is configured such that the display element is the target for one or more display elements of the plurality of display elements. If the display element does not fit in the screen, or if the display element does not fit in the display area related to the node higher than the node corresponding to the display element in the tree structure, the output unit outputs information on the display element. Further, after the output of the information about the display element, input information related to the arrangement of the display element may be acquired and the arrangement determination process may be executed based on the input information.

  Accordingly, the display control apparatus can accept an input related to an arrangement instruction by a human, and a GUI image configured with display elements having an appropriate arrangement and display mode in which operability is ensured according to the instruction. Will be displayed.

  Further, for example, the output unit displays each of the plurality of display elements on the target screen in an arrangement and a display mode according to the result of the arrangement determination process and the display mode determination process by the first processing unit. Then, the output may be performed.

  As a result, a GUI image composed of display elements having an appropriate arrangement and display mode ensuring operability is displayed on the target screen.

  In addition, for example, the output unit displays each of the plurality of display elements according to a result of the arrangement determination process and the display mode determination process by the first processing unit in order to display each of the plurality of display elements on the target screen. The output may be performed by transmitting display control information that defines the arrangement and display mode to the outside of the display control device.

  Thereby, the display control device outputs display control information to be provided to an external device (for example, a terminal device operated by the user) that displays a GUI image on the target screen by transmission. For this reason, the external device can display an appropriate GUI image ensuring operability on the target screen by referring to the display control information.

  In addition, the display control apparatus according to the present disclosure further analyzes the plurality of display elements defined in the display information acquired by the acquisition unit before executing the arrangement determination process and the display mode determination process. The feature of each of the plurality of display elements is specified, and the enlargement limit size for expanding the size of the plurality of display elements according to each specified feature and the policy for expanding the size are determined, and the determined enlargement A second processing unit that enlarges the size of each of the plurality of display elements based on the limit size and the policy may be provided.

  For example, when a GUI image is enlarged and displayed, simply enlarging the GUI image only enlarges the GUI image as a whole, and the GUI image appears to be extended. Here, among a plurality of display elements constituting a GUI image, there are a display element that needs to be greatly enlarged and a display element that does not need to be enlarged so much. Since the GUI image is extended and cannot be seen, it is sufficient to determine the display mode for each display element in advance. However, the display mode must be determined for each display element, and the processing is complicated. become. In addition, when the GUI image is unexpectedly resized, it may not be able to respond appropriately.

  Therefore, in the display control device according to the present disclosure, by specifying the characteristics of a plurality of display elements, the size of the display elements can be automatically enlarged according to the characteristics of each display element. For this reason, a GUI image satisfying a predetermined display standard or the like can be displayed on a display different in size (width and height of the image) and aspect ratio from the GUI image.

  In addition, for example, the enlargement limit size is equal to or larger than each size corresponding to each of the plurality of display elements with respect to each size before enlarging the size of each of the plurality of display elements, and the screen information It is good also as being below the size of the above-mentioned object screen which shows.

  As a result, some display elements of the plurality of display elements are prevented from becoming too large, or some display elements of the plurality of display elements from becoming too small due to the display elements that have become too large. Is done. For this reason, it becomes possible to display on the target screen an appropriate GUI image that ensures a certain visibility and operability of the GUI image for the user.

  Further, for example, in the policy, the size of the plurality of display elements is enlarged while maintaining the aspect ratio of the display mode size of each of the plurality of display elements, and the size of the display mode of each of the plurality of display elements. Expanding the size of the plurality of display elements without maintaining the aspect ratio of the display element, and expanding each display mode only in a specific direction without maintaining the aspect ratio of the size of the display mode of each of the plurality of display elements. It is good also as at least any of doing is included.

  Thereby, the GUI image in which the display mode is enlarged is displayed on the target screen in accordance with the determined policy, that is, the display mode of the display element.

  In addition, for example, the second processing unit analyzes the plurality of display elements according to whether or not the plurality of display elements have a function to be operated, and determines the characteristics of the plurality of display elements. It may be specified.

  Thereby, since the analysis for specifying a display element becomes easy, the 2nd process part can specify the characteristic of a display element easily.

  Further, for example, the second processing unit further analyzes the plurality of display elements based on the size and shape of the plurality of display elements, and identifies the characteristics of the plurality of display elements. It is good as well.

  Thereby, the 2nd processing part can specify a display element according to modes, such as a size in a display element, and a shape.

  The display control method according to the present disclosure is a display control method used in a display control device, and is display information that defines a plurality of display elements to be displayed on a screen used as a GUI (Graphical User Interface). An acquisition step for acquiring the display information including structural information indicating an inclusion relationship between a plurality of display elements, and screen information indicating a form of a target screen that is a screen for displaying the plurality of display elements, and the plurality of displays Predetermined information for determining the arrangement of one or more display elements among the elements, and a requirement regarding the size of each of the display elements of the one or more display elements in order to ensure operability. Based on the predetermined size constraint information and the display information, the target screen includes the target screen according to the form of the target screen. The arrangement of each of the plurality of display elements is determined so that the number of display elements can be accommodated and the requirement is satisfied for the display elements for which the requirement is defined by the size constraint information among the plurality of display elements. A processing step for executing a layout determination process and a display mode determination process for determining the display mode of each of the plurality of display elements, and an output corresponding to the results of the layout determination process and the display mode determination process in the processing step Output step to be performed.

  As a result, even if the size of the screen used as the GUI is relatively small, an appropriate arrangement or the like of each display element constituting the GUI image is determined by referring to the screen information, size constraint information, and the like. Therefore, the GUI image is displayed in a state suitable for the user's operation, that is, in a state where the user's operability is ensured.

  The control program in the present disclosure is a control program for causing a display control device including a microprocessor to execute display control processing, and the display control processing is displayed on a screen used as a GUI (Graphical User Interface). Display information that is defined for a plurality of display elements to be performed and includes display information including structure information indicating an inclusion relationship between the plurality of display elements, and a form of a target screen that is a screen for displaying the plurality of display elements An acquisition step for acquiring screen information to be displayed; information for determining a predetermined arrangement for the arrangement of one or more display elements among the plurality of display elements; and a plurality of display elements for ensuring operability. Based on the size constraint information in which the requirements regarding the size in the display mode of each of the one or more display elements are predetermined, and the display information Regarding the display element in which the plurality of display elements are accommodated in the target screen according to the form of the target screen indicated by the screen information, and the requirement is defined by the size constraint information among the plurality of display elements. A processing step for executing an arrangement determining process for determining an arrangement of each of the plurality of display elements and a display mode determining process for determining a display mode of each of the plurality of display elements so that the requirement is satisfied; And an output step of performing output in accordance with the result of the arrangement determination process and the display mode determination process.

  If this control program is installed in a display control device (computer or the like) including a processor (microprocessor), the display control device may execute a process for displaying a GUI with secured operability on the target screen. Could be possible. Therefore, this control program can provide a GUI suitable for the user's operation.

  These comprehensive or specific various aspects include one or more combinations of an apparatus, a system, a method, an integrated circuit, a computer program, a computer-readable recording medium, and the like.

  Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed descriptions of already well-known matters and repeated descriptions for substantially the same configuration may be omitted. This is to avoid the following description from becoming unnecessarily redundant and to facilitate understanding by those skilled in the art.

  The inventor provides drawings and the following description for those skilled in the art to fully understand the present disclosure, and is not intended to limit the subject matter described in the claims. . That is, the numerical values, shapes, constituent elements, arrangement and connection forms of constituent elements, and steps or order of steps, etc. shown in the description using the following specific examples are examples, and It is not intended to limit the subject matter described in the scope. Among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims can be arbitrarily added. Each figure is a mimetic diagram and is not necessarily illustrated strictly.

(Embodiment 1)
In this embodiment, as an example of a system including a display control device, a display control system including the display control device 100 and the server device 200 will be described with reference to the drawings.

[1-1. Configuration of display control system]
FIG. 1 is a configuration diagram of a display control system. The display control system includes a display control device 100 and a server device 200 that communicate via a communication path 201. The communication path 201 is a communication network or the like for wired or wireless communication. In addition, although the one server apparatus 200 is shown in the figure, the server apparatus 200 used as the communication other party of the display control apparatus 100 may be spaced apart from each other.

  The display control apparatus 100 has a GUI image display function for displaying a GUI image (that is, an image as a GUI) on a display based on display information acquired from the server apparatus 200, and accepts a user's operation and performs various operations according to the operation. The information processing apparatus (for example, search processing, display content switching processing, calculation processing, etc.) is executed. The display control apparatus 100 acquires new display information from the server apparatus 200 by, for example, transmitting information according to a user operation to the server apparatus 200, and displays a new GUI image based on the display information. Also good. The display information is information defining a GUI image, that is, information defining a plurality of display elements to be displayed on a screen used as a GUI. The GUI image may be a web page. The display information will be described in detail later. The GUI image to be displayed on the display surface includes a plurality of display elements. The plurality of display elements include various GUI parts distinguished for each role as a GUI and an area where the GUI parts can be arranged. GUI parts include, for example, labels, input fields, radio buttons, check boxes, list boxes, combo boxes, scroll bars, scroll boxes, tables, lists, images, sentences, paragraphs, areas for storing other GUI parts, etc. There is. Note that the image may be, for example, a still image or a moving image, and may include characters, figures, and the like. Further, the display element may be any visual information such as a photograph, a character, and a graphic.

  As illustrated in FIG. 1, the display control apparatus 100 is configured by a computer including a processor (microprocessor) 101, a memory 102, a communication I / F (Interface) 103, and a user I / F 104. The display control device 100 may be a stationary device such as a broadcast receiver such as a television broadcast, a desktop computer, or an ATM (Automated Teller Machine), but may be a portable device such as a tablet or a smartphone. It may be.

  The memory 102 is a ROM (Read Only Memory), a RAM (Random Access Memory), or the like, and various programs executed by the processor 101 (for example, a computer program such as a control program for display control processing) and various information (that is, various types). Data) can be stored. The display control apparatus 100 implements a display control method by causing the processor 101 to execute a control program for display control processing stored in the memory 102. The control program may be, for example, an application (application program) or a part of an OS (Operating System). For example, when the processor 101 operates in accordance with the control program, the display control apparatus 100 realizes a GUI image display function using a display control method related to GUI image display. The computer program is configured by combining a plurality of instruction codes indicating instructions for the processor in order to achieve a predetermined function. In addition to the memory 102, the display control apparatus 100 may include a recording medium such as a hard disk capable of recording various programs and various data in the same manner as the memory 102.

  The communication I / F 103 is a communication circuit for communicating with an external apparatus such as the server apparatus 200 via the communication path 201 in accordance with a predetermined communication protocol. The processor 101 can acquire display information transmitted from the server device 200 via the communication I / F 103.

  The user I / F 104 is a device that presents information to the user under the control of the processor 101 or a device that receives a user operation and transmits it to the processor 101, and is a touch panel that can display a GUI image, for example. The touch panel includes a display capable of displaying a GUI image and a touch pad superimposed on the display surface. The display in the user I / F 104 may be of any kind, such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The user I / F 104 is not necessarily limited to the touch panel, and may be configured by an image display device such as a display or a projector and an input reception device such as a keyboard or a pointing device such as a mouse. Good. The GUI image is displayed on a screen (for example, a screen such as a display surface) in an image display device provided in the display control device 100. Here, for convenience of explanation, the display control apparatus 100 is described as including a display (for example, a display as a part of a touch panel) and displaying a GUI image on the display. The user's operation on the screen on which the GUI image is displayed is performed using a pointing device or other input device even if the operation is a touch operation with a finger, a pen, or the like on the surface of the touch panel, for example. It may be an operation.

  The server device 200 is a computer having a display information transmission function (for example, a web server function) that is information related to a GUI image to be displayed. The server apparatus 200 transmits display information to the display control apparatus 100 via the communication path 201. Note that the communication partner of the server apparatus 200 is not limited to one display control apparatus 100, and a GUI image display similar to that of the display control apparatus 100 except that a plurality of display control apparatuses (for example, the screen size of the display may be different from each other). A plurality of display control devices having a function). Note that the display information is not dedicated information specific to only the display form (for example, screen size) of one display control apparatus 100, but for example, to each of a plurality of displays having different forms (screen size, etc.). It may be information defining display elements to be displayed. The display information may be dedicated information specialized for a display device such as a display in an information processing device other than the display control device 100.

[1-2. Functional configuration of display control device]
FIG. 2 is a block diagram illustrating a functional configuration of the display control apparatus 100.

  As illustrated in FIG. 2, the display control apparatus 100 includes an acquisition unit 110, a storage unit 120, a processing unit 130 (first processing unit), and an output unit 140.

  The acquisition unit 110 has a function of acquiring display information and screen information indicating a form of a screen (referred to as a “target screen”) that is a display target of the display element in the display control apparatus 100 and transmitting the screen information to the processing unit 130. Have. The acquisition unit 110 stores the acquired display information and screen information in the storage unit 120 so that the processing unit 130 can refer to the display information and the screen information. Good. The acquisition unit 110 is realized by, for example, the processor 101 that executes a control program, the communication I / F 103, and the like. The acquisition unit 110 may acquire display information or screen information by receiving it from the server device 200 via the communication path 201, or may be a recording medium (for example, a memory 102, a hard disk, or other recording medium) provided in the display control device 100. Display information or screen information may be acquired by reading from the medium. Note that the screen information may be acquired by the acquisition unit 110 from the server device 200 when the server device 200 on the network with which the display control device 100 can communicate is a display driver provider, or display control may be performed. The acquisition unit 110 may read and acquire from the display or the display driver in the apparatus 100.

  An example of the display information acquired by the acquisition unit 110 is shown in FIG. As shown in the figure, the display information is information defining a plurality of display elements to be displayed on a screen used as a GUI, and includes structure information. In the example of FIG. 3, each display element is represented by using an identification name such as “A”, “AA”, “AB”, “AAA”, “AAB”, and the like. This identification name may be, for example, a name that can distinguish the type of GUI component, or any code, data value, or the like.

  The structure information is information indicating an inclusion relationship between a plurality of display elements. In the example of FIG. 3, the structure information is information expressed by associating each of a plurality of display elements with each node in a tree structure of a plurality of hierarchies. In a tree structure, when a node other than the lowermost layer is referred to as an upper node, a node under the upper node is a node at any level below the upper node, and a node immediately below the upper node is a node of the upper node. It is a node one level below. The display area of the display element corresponding to the upper node includes the display area of the display element corresponding to each node under the upper node. In the structure information in the example of FIG. 3, the display element “A” having the largest display area is associated with the highest root node, and is included in the display area of the display element corresponding to the relatively higher node. Nodes corresponding to one or a plurality of display elements are arranged under the upper nodes. As a premise, the display area of the display element corresponding to the root node fits in a target screen (screen such as a display) on which the GUI image is to be displayed. In the tree structure of the example of FIG. 3, the node corresponding to the display element “AA” and the display element “AB” are directly below the root node corresponding to the display element “A” (that is, subordinate and one layer below). The node is located. In addition, nodes corresponding to each of the five display elements “AAA”, “AAB”, “AAC”, “AAD”, and “AAE” are located one level below the node corresponding to the display element “AA”. Yes. Further, nodes corresponding to the three display elements “ABA”, “ABB”, and “ABC” are located immediately below the node corresponding to the display element “AB”. In addition, nodes corresponding to the four display elements “ABCA”, “ABCB”, “ABCC”, and “ABCD” are located immediately below the node corresponding to the display element “ABC”. For example, the structure information includes four display elements “ABCA”, “ABCB”, “ABCC”, “ABCD” (hereinafter, these four display elements are represented by GUI in the display area of the display element “ABC”. Is described as being a button.) Should be included. The display elements “ABCA” and “ABCB” corresponding to the lowest leaf node of the tree structure do not include other display elements within the display area of the display element. In addition, as a structure information, when a plurality of nodes are located immediately below one node, information related to a certain order for display may be included between a plurality of display elements corresponding to the plurality of nodes. For example, the display elements corresponding to the nodes shown in the upper part of FIG.

  In addition to the structure information described above, the display information may include display mode information that designates the display mode of each of one or more display elements among the plurality of display elements. FIG. Including examples. For example, if the display element is a character string, the display mode of the display element is, for example, the contents of the character string, the character font type, the character font size, the character color, the background color, and the like. The display mode may include distinction such as bold and italic, or distinction such as right alignment and center alignment. If the display element is a button or icon as a GUI component, the display mode is, for example, display content, button size (width, height, etc.), and the like. A display mode of a certain display element may include a relative position with respect to another display element.

  The format of the display information is not limited to the format shown in FIG. 3, and the display information can be any data or program having any structure as long as it can identify structural information indicating the inclusion relationship between the plurality of display elements. May be. For example, the display information may be information specified by a document described in, for example, XML (eXtensible Markup Language) or HTML (HyperText Markup Language). In addition, the display information or the display mode information may include information specified by a predetermined script (for example, a script that defines processing related to a display element) attached to a document described in HTML, for example. . The display mode information may be information specified by, for example, a part of a document described in HTML, or CSS (Cascading Style Sheets) that defines the display mode of display elements.

  The display control apparatus 100 designates information (for example, a network address of a website) such as a URL (Uniform Resource Locator) that designates information of a web page to be displayed from the network via the user I / F 104, for example. Display information about a web page as a GUI image to be displayed may be acquired by receiving a user operation, accessing the server device 200 having a web server function, receiving and analyzing an HTML document, and the like. .

  An example of the screen information acquired by the acquisition unit 110 is shown in FIG. The screen information indicates a form of a target screen that is a display target of display elements by the display control device 100. As shown in FIG. 4, the screen information includes, for example, the screen size (that is, the width of the target screen), the screen size (that is, the height of the target screen), and other screens (that is, display elements should be displayed). Range) information. In FIG. 4, the screen width and the screen height are represented by the number of pixels in pixel units, but the unit may be any unit, such as meters (m). In the case where a plurality of shapes such as a rectangle, an ellipse, and the like are assumed as the shapes of screens that can be displayed in the operation of the display control apparatus 100, for example, as shown in FIG. Information indicating the shape of the target screen. Further, the screen information may include information indicating a range of the notch when a so-called notch (that is, a recess such as a notch) exists in a part of the target screen. The display control apparatus 100 determines a displayable range of display elements based on the screen information, and the display element indicated as the root node in the display information structure information is included in the displayable range. As described above, the arrangement, display mode, and the like of each display element constituting the GUI image are determined.

  The storage unit 120 is an area of a recording medium such as a memory in which display information and screen information acquired by the acquisition unit 110 can be stored. The storage unit 120 also stores arrangement determination information and size constraint information. The arrangement determination information is information that defines the arrangement of one or more display elements among a plurality of display elements to be displayed on a screen used as a GUI indicated by the display information. The arrangement determination information may be information included in a part of the display information acquired by the acquisition unit 110 (for example, a part of the display mode information illustrated in FIG. 3). This arrangement determination information is information used when the processing unit 130 of the display control apparatus 100 determines the arrangement of display elements.

  An example of the arrangement determination information stored in the storage unit 120 is shown in FIGS. 5A and 5B. The example of FIGS. 5A and 5B is an example of the information for determining the arrangement determined in correspondence with the display information shown in FIG. 3, but the information for determining the arrangement is, for example, the type of the GUI component as the display element. It may be information on the arrangement determined accordingly. The arrangement determination information includes, for example, a default value related to normal arrangement similar to information used when an existing web browser arranges display elements according to an HTML document, and further includes, for example, alternative arrangement and arrangement restrictions. Information relating to an algorithm for automatically changing the arrangement may be included. In the example of FIG. 5A, the display elements identified by “ABA”, “ABB”, and “ABC” are arranged in the upper, middle, and lower positions in this order. In addition, in the arrangement determination information in FIG. 5A, each display element (that is, each button) identified by “ABCA”, “ABCB”, “ABCC”, and “ABCD” is arranged in this order from left to right. 1st arrangement information which shows that. In addition, in the arrangement determination information in FIG. 5A, the display element identified by “ABCB” is arranged to the right of the display element identified by “ABCA” and is identified by “ABCC” as information relating to the alternative arrangement. Second arrangement information indicating that the display element is arranged below the display element identified by “ABCB” and the display element identified by “ABCD” is arranged to the right of the display element identified by “ABCC”. Is included. In the example of FIG. 5A, assuming the structure information in the display information of FIG. 3, each display element “ABCA” corresponding to each node immediately below the node corresponding to the display element identified by “ABC” in the tree structure. , “ABCB”, “ABCC”, and “ABCD”, the second arrangement information indicates an arrangement different from the arrangement indicated by the first arrangement information. FIG. 5B shows an example in which the placement determination information includes placement constraint information in which requirements regarding the relative placement of all or some of the plurality of display elements to be displayed are predetermined. In the example of FIG. 5B, the position of the display element identified by “ABCB” is the display identified by “ABCA” as a restriction on the display element to be included in the area of the display element identified by “ABC”. There is a layout requirement that is limited to the right of the element. This arrangement constraint information can be used when it is useful to arrange a plurality of display elements in a specific positional relationship, for example, in order to facilitate operation. For example, the arrangement constraint information can be used to arrange a stop button side by side next to a button for reproducing moving images, music, etc. (that is, a button of a GUI component). Further, in the example of FIG. 5B, as a restriction on the display elements to be included in the display element area identified by “AA”, an arrangement requirement that the display elements are arranged in a vertical row is defined. Assuming the structure information in FIG. 3, this is “AAA”, “AAB”, “AAC”, “AAD”, “AAD” corresponding to the nodes under the node corresponding to the display element identified by “AA”. The requirement that each display element identified by “AAE” should be arranged in a list in a vertical row is defined.

  An example of the size constraint information stored in the storage unit 120 is shown in FIG. The size constraint information is information that defines requirements regarding the size in the display mode of each of one or more display elements among a plurality of display elements to be displayed as a GUI in order to ensure operability necessary as a GUI. The operability here is a concept that also includes the visibility of display elements that are necessary as a premise of the operation. The size constraint information indicates that the display element can be visually recognized even when the target screen used as the GUI is small, and the size requirement for operating the buttons and the like as the GUI (for example, one or more display elements respectively) Minimum size). The minimum size of the display element as a requirement relating to the above-described size is determined based on the size of a general user's finger, for example, and is, for example, the minimum size that can be pressed with a finger. The minimum size (for example, the minimum font size) of the display element that is a character string is, for example, the minimum size that can be read with the naked eye by a general user. The processing unit 130 of the display control apparatus 100 determines the display mode of the display elements so as to satisfy the requirements regarding the size indicated by the size constraint information. In the example of FIG. 6, 8 points (pt) are defined for the font size as the minimum size of the display element of a character string such as a label identified by “ABB”. This indicates that the character string of the display element identified by “ABB” should be displayed with 8 points or more. In the example of FIG. 6, the minimum size of each display element (that is, each button) identified by “ABCA”, “ABCB”, “ABCC”, and “ABCD” is 150 pixels (px) in width and high 150 pixels (px) are defined. This indicates that the four buttons identified by “ABCA” or the like should be displayed with a width of 150 pixels or more and a height of 150 pixels or more.

  The processing unit 130 is configured to configure a GUI image in the target screen according to the form of the target screen indicated by the screen information based on the arrangement determination information, the size constraint information, and the display information stored in the storage unit 120. A function that executes arrangement determination processing and display mode determination processing so that a plurality of display elements can be accommodated and the requirements are satisfied for the display elements of the plurality of display elements whose requirements are determined by the size constraint information Have The processing unit 130 is realized by, for example, the processor 101 that executes a control program. The processing unit 130 includes a layout determination unit 131 that executes a layout determination process that determines the layout of each of the plurality of display elements according to the GUI, and a display mode determination process that determines the display mode of each of the plurality of display elements. And a display mode determination unit 132 that executes The processing unit 130 executes the layout determination process and the display mode determination process according to the screen information based on the display information, the layout determination information, and the size constraint information, so that the target screen form such as size and shape is obtained. Accordingly, the arrangement and display mode of each display element are determined, and it is possible to display a display element that ensures operability as a GUI even when the target screen is large or small. For example, when the display mode determination process is executed by the display mode determination unit 132 based on the size constraint information that defines the above-described minimum size (see FIG. 6) as the size requirement, the display mode determination unit 132 The display mode of the display element for which the requirements relating to the minimum size are determined so as to be the minimum size or a larger size is determined. Thereby, operability is sufficiently ensured for a user who performs an operation via the display element related to the GUI.

  In order to determine the arrangement and display mode of a plurality of display elements for GUI (that is, a plurality of display elements constituting a GUI image), the processing unit 130 is first determined by size constraint information based on display information. Temporary placement processing for temporarily determining the placement of a plurality of display elements so as to satisfy the above requirements is executed and display mode determination processing is executed. The provisional arrangement process can be executed by, for example, the arrangement determining unit 131 in the processing unit 130. In the provisional arrangement process, for example, the arrangement of each display element can be determined by a technique similar to that of an existing web browser or the like. This provisional arrangement process is performed by grasping the inclusion relationship of display elements such as GUI parts based on the structure information in the display information, and calculating the size of each display element based on the display mode information and the like. However, in the provisional arrangement process, the display elements that do not satisfy the requirements defined by the size constraint information are adjusted so as to satisfy the requirements defined by the size constraint information. As a result of executing the temporary arrangement process and the display mode determination process once performed, the processing unit 130, for one or more display elements of a plurality of display elements, when the display elements do not fit within the target screen. Displays the display area related to the upper node of the node corresponding to the display element in the tree structure of the structural information in the display information (that is, corresponds to the upper node so that the display element fits in the target screen) If the display element does not fit within the display area), the arrangement determination process is executed so that the display element fits within the display area. Note that the processing unit 130 may further perform the display mode determination process when the arrangement determination process is performed.

  When the processing unit 130 once determines the arrangement of the plurality of display elements and executes the display mode determination process, one display element of the plurality of display elements for the GUI does not fit in the target screen. Or, if it does not fit in the display area related to the node higher than the node corresponding to the one display element in the tree structure of the structure information in the display information, the tree corresponding to the one display element is used as the arrangement determination process. The relative arrangement of the display elements corresponding to the nodes immediately below the node one level higher than the node in the structure can be changed. As a method for changing the arrangement, there are a method for determining the arrangement based on information for alternative arrangement (for example, the first arrangement information and the second arrangement information in FIG. 5A), a method for automatically determining the arrangement, and the like. In the method of automatically determining the arrangement, for example, it may be performed so as to satisfy the requirements defined in the arrangement constraint information (see FIG. 5B). When it does not fit within the target range (for example, the display area within the target screen or the upper node), a method of repeating the random placement again until each display element falls within the target range may be used.

  The output unit 140 is realized by, for example, a user I / F 104 such as a display, a processor 101 that executes a control program, and the like. The output unit 140 performs output according to the results of the layout determination process and the display mode determination process performed by the processing unit 130. As a specific example, the output unit 140 outputs each display element for the GUI by displaying it on a display as a target screen in an arrangement and display manner according to the results of the placement determination processing and the display manner determination processing. I do.

[1-3. Operation of display control device]
Hereinafter, an operation related to the display control method by the display control apparatus 100 having the above-described configuration will be described.

[1-3-1. Display control processing]
FIG. 7 is a flowchart illustrating an example of display control processing as a display control method in the display control apparatus 100. Hereinafter, the operation of the display control apparatus 100 will be described with reference to FIG.

  The display control process is started by, for example, a predetermined user operation for displaying a GUI image (for example, a power-on operation of the display control device 100, a start operation of a specific application, and other operations). Here, the display surface in the user I / F 104 of the display control apparatus 100 will be described as the target screen.

  First, the display control apparatus 100 acquires screen information related to a target screen on which a plurality of display elements constituting a GUI image are to be displayed (step S1).

  Next, the display control apparatus 100 acquires display information defined for a plurality of display elements constituting the GUI image, for example, from the server apparatus 200 (step S2).

  Next, the display control apparatus 100 displays information with reference to the placement determination information (see FIGS. 5A and 5B), size constraint information (see FIG. 6), display information (see FIG. 3), and screen information (see FIG. 4). Determination processing (that is, provisional arrangement processing, display mode determination processing, and arrangement determination processing) is executed (step S3).

  And the display control apparatus 100 performs the output according to the result of the arrangement | positioning determination process finally performed by step S3, and the display mode determination process (step S4). With this output, for example, each display element constituting the GUI image is displayed on the display in the user I / F 104. Note that if the user performs an operation corresponding to the displayed GUI image to display a new GUI image, the display control apparatus 100 can execute the display control process again. In this case, acquisition of already acquired screen information (step S1) may be omitted. In this case, in step S2, display information corresponding to the new GUI image is acquired, for example.

[1-3-2. Display decision processing]
Hereinafter, the display determination process (step S3) in the display control process described above will be described in detail with reference to the flowchart illustrated in FIG. In the display determination process of this example, the processing unit 130 executes the provisional layout process according to the first layout information of the layout determination information (see FIG. 5A), and executes the layout determination process according to the second layout information.

  First, the arrangement determining unit 131 of the processing unit 130 executes provisional arrangement processing for temporarily determining the arrangement of a plurality of display elements constituting the GUI image based on the display information (Step S11). In this provisional arrangement process, for example, the display elements of the buttons identified by “ABCA”, “ABCB”, “ABCC”, and “ABCD” in the display information of FIG. 1 Arranged according to the arrangement information. The other display elements are arranged in the provisional arrangement process according to, for example, arrangement determination information. In the provisional arrangement process, the arrangement may be determined so as not to overlap when each display element is displayed with a size satisfying the requirement with reference to the size constraint information. Note that the processing unit 130 executes provisional arrangement processing so that the display elements whose display mode is specified in the display mode information (see FIG. 3) among the plurality of display elements constituting the GUI image are in accordance with the specification, And it is good also as performing a display mode determination process.

  Next, the display mode determination unit 132 of the processing unit 130 executes the display mode determination process so that each display element constituting the GUI image has a size that satisfies the requirements related to the size constraint information (step S12).

  Next, the processing unit 130 displays the upper display area (that is, the display area of the display element corresponding to the upper screen of the target screen and the node corresponding to the display element) among the display elements constituting the GUI image. It is determined whether or not there are display elements that do not fit (step S13).

  If it is determined affirmative in step S13, that is, if any display element constituting the GUI image is determined not to be within the upper display area, the processing unit 130 determines the minimum indicated by the size constraint information. It is determined whether or not there are display elements that are sized but do not fit in the upper display area (step S14).

  If the determination in step S14 is affirmative, the processing unit 130 executes an arrangement determination process for changing the arrangement in the arrangement determination unit 131 for each display element that does not fit in the upper display area ( Step S15).

  After the placement determination process in step S15 or when a negative determination is made in step S14, the processing unit 130 determines the display mode for each display element that does not fit in the upper display area by the display mode determination unit 132. Processing is executed (step S16). In step S <b> 16, the display mode determination unit 132 determines the display mode so as to be equal to or larger than the minimum size defined as the requirement related to the size constraint information. Here, for example, the display mode determination unit 132 may use a method of determining the display mode so that the display size is equal to or larger than the minimum size defined as the requirement relating to the size constraint information and the maximum size can be accommodated in the upper display area. It is good also as the method of determining a display mode so that it may become a size larger than the minimum size even if it is not the maximum size. In addition, the display mode determination part 132 will determine the display mode of a display element so that it may become the minimum size, when it cannot display with a size larger than minimum size. This method is useful when the operability can be further improved if the display element has a larger size. That is, it is useful for the processing unit 130 to execute the display mode determination process at least after the execution of the layout determination process to find out whether or not the display element can be displayed larger, because this can lead to an improvement in operability.

  After the display mode determination process in step S16, the processing unit 130 returns again to step S13 in which it is determined whether or not there is still a display element that does not fit in the upper display area among the display elements constituting the GUI image. .

  If the determination is negative in step S13, that is, if it is determined that all the display elements constituting the GUI image are within the upper display area, the processing unit 130 ends the display determination process. As a result, each display element constituting the GUI image is displayed on the display in the user I / F 104 in the arrangement and display manner determined last.

  In addition, it is possible to prevent the positive determination in step S13 from being repeated indefinitely by designing display information and the like in consideration of the possibility that the target screen size is sufficiently small. When the affirmative determination is repeated a predetermined number of times or more in step S13, for example, all or a part of display elements constituting the GUI image may be displayed so as to be scrollable according to the operation.

[1-3-3. Operation example 1 of arrangement determination processing]
Hereinafter, an operation example 1 of the arrangement determination process will be described with reference to the flowchart of FIG. 9A showing an example of the arrangement determination process (step S15) in the display determination process described above.

  As a premise, description will be made assuming that arrangement determination information indicating a plurality of arrangements for one display element is defined as in the first arrangement information and the second arrangement information in FIG. 5A.

  First, the arrangement determining unit 131 of the processing unit 130 focuses on one display element that should be accommodated in the upper display area for a display element that does not fit in the upper display area (step S21). Here, the display information shown in FIG. 3 is identified by “ABC” corresponding to the node immediately below the tree-structured node corresponding to the display element identified by “AB” with the left half of the target screen as the display area. The display area of the display element to be described will be described using an example in which the display area is ¼ of the area of the target screen and located on the lower left side of the target screen. Of the four buttons (that is, the display elements identified by “ABCA”, “ABCB”, “ABCC”, and “ABCD”) to be stored in the display area of the display element identified by “ABC”, “ABCD ”Is not included in the display area related to“ ABC ”. In this case, in step S21, the arrangement determining unit 131 identifies each of “ABCA”, “ABCB”, “ABCC”, and “ABCD” as one display element to be stored in the display area related to “ABC”. Attention is paid to any one of the displayed elements.

  Next, the arrangement determining unit 131 determines the arrangement of the focused display element using the arrangement information next to the arrangement information used last (step S22). In this step S22, for example, the arrangement is determined by using the second arrangement information subsequent to the first arrangement information in the arrangement determining information used in the provisional arrangement process. FIG. 5A is merely an example, and information indicating three or more arrangements for one display element may be defined in the arrangement determination information.

  Next, the arrangement determining unit 131 determines whether there is another display element that should be stored in the upper display area of the display element of interest (in this example, the display area of the display element identified by “ABC”). However, if it exists, paying attention to the other one display element (step S24), the process returns to the determination of the arrangement in step S22.

  Then, when the arrangement for each display element to be stored in the upper display area is determined, it is determined in step S23 that there is no other display element, and the arrangement determination process ends. By such an arrangement determination process, information for one or more alternative arrangements is used, and each display element constituting the GUI image is suitable for the target screen under a size constraint that ensures operability. The selected arrangement will be selected.

[1-3-4. Operation example 2 of arrangement determination processing]
Hereinafter, an operation example 2 of the arrangement determination process will be described with reference to the flowchart of FIG. 9B showing an example of the arrangement determination process (step S15) in the display determination process described above.

  As a result of the processing unit 130 deciding the arrangement for the plurality of display elements constituting the GUI image by the display determination process and executing the display mode determination process, one display element of the plurality of display elements is included in the target screen. Or may not fit within the display area related to the node above the node corresponding to the one display element in the tree structure of the structure information (see FIG. 3). In this operation example 2, the width and height of the display area in which the display elements corresponding to all the nodes immediately below the node one level higher than the node in the tree structure corresponding to the one display element are to be stored are specified. Based on a specific result, an arrangement determination process is executed so that display elements to be stored in the display area are stored. Specifically, each display element corresponding to two or more nodes immediately below a node one layer higher in the tree structure corresponding to the one display element that does not fit in the display area is arranged in a matrix. The width and height of the display area in which each display element arranged in the matrix is to be stored, and the matrix so that each display element to be stored in the display area is stored based on the specific result. The arrangement determination process is executed by changing the number of rows and the number of columns in the shape arrangement. The matrix arrangement is an arrangement of m rows and n columns, and both m, which is the number of rows, and n, which is the number of columns, are natural numbers, and m and n may be the same value or different values.

  As a premise of this operation example 2, the processing unit 130 performs provisional placement processing (step S11 in FIG. 8) so as to satisfy the requirements defined by the placement constraint information in the placement determination information. In the placement determination process, placement is determined so as to satisfy the requirements defined by the placement constraint information.

  Here, it is assumed that the placement constraint information as placement requirements for one or more display elements is determined in the placement decision information as shown in FIG. 5B. An example of determining the arrangement of display elements will be described. In addition, the display area of the display element identified by “ABC” in the display information shown in FIG. 3 is located on the lower left side of the target screen with a size equal to or smaller than ¼ of the area of the target screen. Four buttons (that is, display elements identified by “ABCA”, “ABCB”, “ABCC”, and “ABCD”) to be stored in the display area) are arranged in one horizontal row (that is, one row and four columns), A description will be given assuming that the button identified by “ABCD” does not fit in the display area related to “ABC”.

  An example of this state is shown in FIG. FIG. 10 shows an arrangement example of a plurality of display elements constituting a GUI image before the display determination process is completed. The arranged display elements 401 to 405 indicate display elements identified by “AAA”, “AAB”, “AAC”, “AAD”, and “AAE” in the display information of FIG. 3, for example. The display element 410 indicates a display element identified by “ABA”, and the display element 420 indicates a display element identified by “ABB”. The display elements 431 to 434 are identified by the respective display elements to be included in the display area of the display element identified by “ABC” (that is, “ABCA”, “ABCB”, “ABCC”, and “ABCD”). Display element).

  The arrangement determining unit 131 first determines whether or not a plurality of display elements are stored in a matrix in the upper display area of the display elements that do not fit in the upper display area (step S31).

  If the determination in step S31 is affirmative (that is, some display elements are not accommodated in the upper display area but a plurality of display elements are accommodated in a matrix), the arrangement determining unit 131 The width and height of the upper display area are specified (step S32). Here, based on the screen information shown in FIG. 4, the display area related to the display element identified by “ABC” as a size equal to or less than ¼ of the area of the target screen having a width of 1280 pixels and a height of 720 pixels The description will be continued assuming that the width is 620 pixels and the height is 340 pixels, for example.

  The arrangement determination unit 131 follows a restriction of the arrangement restriction information so as to be within the width and height specified in step S32, and then displays a plurality of display elements (that is, “ABCA”, “ABCB”, “ABCC”, “ The number of rows and the number of columns in the matrix arrangement of display elements identified by ABCD is changed (step S33). Thereby, the number of rows and the number of columns of the button of 1 row and 4 columns is changed so as to be 2 rows and 2 columns, for example. Another example of changing the number of rows and the number of columns includes an example in which 1 row and 6 columns are changed to 2 rows and 3 columns, 3 rows and 2 columns, or 6 rows and 1 column. When the arrangement determining unit 131 determines a matrix arrangement for a plurality of buttons according to the changed number of rows and columns, for example, the button identified by “ABCB” is determined according to the arrangement constraint information illustrated in FIG. 5B. , The arrangement is determined so as to be arranged to the right of the button identified by “ABCA”.

  FIG. 11 shows an arrangement example of a plurality of display elements constituting the GUI image after the display determination process is completed by completing the arrangement determination process. The display elements 431 to 434 are a group of buttons arranged in a matrix of 1 row and 4 columns in FIG. 10 and are arranged in a matrix of 2 rows and 2 columns in FIG. Here, the minimum width and height of each button which is the display elements 431 to 434 are assumed to be 150 pixels (see FIG. 6). After the arrangement determining process, the display mode determining process shown in step S16 of FIG. 8 makes the size of each button larger, for example, a height of 170 pixels, a width of 310 pixels, etc., and the operation becomes easier. As such, it may be determined.

  If the determination in step S31 is negative, the arrangement determining unit 131 changes the arrangement of the display elements that did not fit in the upper display area according to the restrictions of the arrangement restriction information (step S34), and the arrangement is determined. Finish the process.

  In this operation example 2, an example in which a plurality of display elements are arranged in a matrix in step S33 is shown, but the processing unit 130 does not necessarily have to arrange each display element in a matrix.

[1-4. Effect]
According to the display control apparatus 100 described above, an arrangement that ensures the user's operability and fits the display elements constituting the GUI image on the target screen can be determined by referring to the size constraint information and the like. Thereby, for example, a terminal device such as a smartphone or a tablet having an input / output device such as a touch panel including a relatively small display can be used as an easy-to-operate GUI.

(Embodiment 2)
In the first embodiment described above, the example in which the display control apparatus 100 includes the user I / F 104 such as a display for displaying a GUI image has been described. In contrast, in the present embodiment, a display control system in which a display control apparatus 100a that does not include a display or the like performs control for displaying a GUI image on a terminal apparatus 300 that includes a display or the like is described with reference to the drawings. explain.

[2-1. Configuration of display control system]
FIG. 12 is a configuration diagram of the display control system according to the present embodiment. This display control system includes a display control device 100a and a server device 200 that communicate with each other via a communication path 201, and a terminal device 300 that can communicate with the display control device 100a in a wired or wireless manner. In FIG. 12, the same components as those shown in FIG. 1 are denoted by the same reference numerals, and description thereof will be omitted as appropriate. The display control device 100a is a partial modification of the display control device 100 shown in the first embodiment, and is the same as the display control device 100 except that it is not particularly described here. Although only one terminal device 300 is shown in FIG. 12, a plurality of terminals may be present, and for example, the display of each terminal device may be different. Further, display information, screen information, arrangement determination information, and size constraint information handled by the display control apparatus 100a are the same as those described in the first embodiment. Note that the display information is not dedicated information specific to the display form (for example, screen size) of one terminal device 300, but is displayed on each of a plurality of displays having different forms (screen size, etc.), for example. It may be information defining display elements to be performed. The display information may be dedicated information specialized for a display device such as a display in an information processing device other than the terminal device 300.

  The display control device 100a arranges and displays a plurality of display elements constituting the GUI image in order to display the GUI image on the target screen that is the display of the terminal device 300 based on the display information acquired from the server device 200. This is an information processing apparatus having a GUI image display control function for transmitting display control information that defines the above to the terminal device 300.

  As shown in FIG. 12, the display control apparatus 100a is configured by a computer including a processor 101, a memory 102, and a communication I / F 103a.

  The memory 102 is a ROM, a RAM, or the like, and can store various programs executed by the processor 101 (for example, a control program for display control processing) and various information. The display control apparatus 100 a implements a display control method by causing the processor 101 to execute a control program for display control processing stored in the memory 102. For example, when the processor 101 operates in accordance with the control program, the display control apparatus 100a realizes a GUI image display control function. In addition to the memory 102, the display control apparatus 100a may include a recording medium such as a hard disk capable of recording various programs and various data in the same manner as the memory 102.

  The communication I / F 103a is a communication circuit for communicating with an external device such as the server device 200, the terminal device 300, or the like according to a predetermined communication protocol. The processor 101 can acquire display information transmitted from the server device 200 via the communication I / F 103a. Further, the processor 101 can transmit display control information to the terminal device 300 via the communication I / F 103a.

  The functional configuration of the display control apparatus 100a is the same as the functional configuration of the display control apparatus 100 shown in the first embodiment (see FIG. 2). However, the output unit 140 of the display control apparatus 100a displays the arrangement determination process and the display mode by the processing unit 130 in order to display each of the plurality of display elements constituting the GUI image on the target screen (that is, the display of the terminal device 300). The display control information that defines the arrangement and display mode of each display element according to the result of the determination process is output by transmitting it to the outside of the device itself (specifically, the terminal device 300).

  As illustrated in FIG. 12, the terminal device 300 includes a computer including a processor 301, a memory 302, a communication I / F 303, and a user I / F 304. The terminal device 300 may be a stationary device such as a desktop computer or ATM, or may be a portable device such as a tablet or a smartphone.

  The memory 302 is a ROM, a RAM, or the like, and can store various programs executed by the processor 301 and various information. The terminal device 300 implements various types of information processing by executing a program stored in the memory 302 by the processor 301.

  The communication I / F 303 is a communication circuit or the like for communicating with the display control apparatus 100a according to a predetermined communication protocol. The processor 301 can acquire display control information for displaying a GUI image from the display control apparatus 100a via the communication I / F 303. The processor 301 that executes the program determines the arrangement and display mode of each display element constituting the GUI image based on the acquired display control information, and displays it on a display or the like in the user I / F 304.

  The user I / F 304 is a device that presents information to the user under the control of the processor 301 or a device that receives a user operation and transmits it to the processor 301, and is a touch panel including a display capable of displaying a GUI image, for example. The user I / F 304 is not necessarily limited to the touch panel, and may be configured by an image display device such as a display or a projector, and an input reception device such as a keyboard or a pointing device such as a mouse. Good. The GUI image is displayed on a screen (for example, a screen such as a display surface) in an image display device provided in the terminal device 300. Here, for convenience of explanation, the terminal device 300 will be described as including a display (for example, a display as a part of a touch panel) and displaying a GUI image on the display. For example, the terminal device 300 transmits information corresponding to the user's operation to the display control device 100a, and the display control device 100a relays the information to the server device 200, thereby acquiring new display information from the server device 200. Then, display control information for displaying a new GUI image based on the display information may be created and returned to the terminal device 300.

[2-2. Operation of display control device]
The display control device 100a performs display control processing in the same manner as the display control device 100 shown in the first embodiment (see FIGS. 7, 8, 9A, and 9B).

  However, the screen information acquired in step S1 is screen information related to the display of the terminal device 300 that is a display target of the GUI image.

  In addition, output corresponding to the results of the layout determination process and the display mode determination process in step S4 is transmitted to the terminal device 300 as display control information indicating the layout and display mode determined for each display element constituting the GUI image. It is realized by doing.

[2-3. Effect]
According to the display control apparatus 100a described above, the size restriction information is changed from the display information defined for each display element of the GUI image in accordance with the screen information related to the form of the display or the like of the terminal device 300 on which the GUI image is to be displayed. Display control information is generated by determining the arrangement and display mode of each display element so as to satisfy such requirements. Since the display control device 100a provides the display control information to the terminal device 300, the terminal device 300 can display a GUI image in which each display element is appropriately arranged while ensuring user operability. Become. That is, the display control device 100a allows the terminal device 300 to provide an easy-to-operate GUI even when it has a relatively small display.

(Embodiment 3)
In the above-described second embodiment, the example in which the processing unit 130 of the display control apparatus 100 executes the arrangement determination process and the display mode determination process has been described. On the other hand, in the present embodiment, before the determination processing unit 133 executes the arrangement determination process and the display mode determination process, the expansion limit size for expanding the sizes of the plurality of display elements and the policy for expanding the size The display control apparatus 100b that performs the process of determining the image will be described with reference to the drawings.

[3-1. Configuration of display control apparatus]
FIG. 13 is a configuration diagram of the display control apparatus 100b according to the present embodiment. In FIG. 13, the same components as those shown in FIG. 1 and subsequent figures are denoted by the same reference numerals, and description thereof will be omitted as appropriate. The display control device 100b is a partial modification of the display control device 100 described in the first embodiment and the like, and is not particularly described here, and is the same as the display control device 100. The display information, screen information, arrangement determination information, and size constraint information handled by the display control apparatus 100b are the same as those described in the first embodiment. Note that the display information is not dedicated information specific to the display form (for example, screen size) of one terminal device 300, but is displayed on each of a plurality of displays having different forms (screen size, etc.), for example. It may be information defining display elements to be performed. The display information may be dedicated information specialized for a display device such as a display in an information processing device other than the terminal device 300.

  As illustrated in FIG. 13, the display control apparatus 100 b includes a determination processing unit 133 (second processing unit) in addition to the acquisition unit 110, the storage unit 120, the processing unit 130, and the output unit 140.

  The determination processing unit 133 analyzes the plurality of display elements defined in the display information acquired by the acquisition unit 110 and identifies the characteristics of each of the plurality of display elements before executing the arrangement determination process and the display mode determination process. . Further, the determination processing unit 133 analyzes the plurality of display elements and identifies the characteristics of each of the plurality of display elements depending on at least whether or not the plurality of display elements have a function to be operated. That is, when the display element has a function to be operated, it can be said that the display element is a button or the like having a function to be operated, so the determination processing unit 133 specifies the characteristics of the display element. can do. Further, the determination processing unit 133 further analyzes the plurality of display elements based on the size and shape of the plurality of display elements, and identifies the characteristics of each of the plurality of display elements. For example, the determination processing unit 133 specifies that the display element is any one of an image, a character string, a ruled line such as a frame line, and the like based on the size and shape of the display element.

  Further, the determination processing unit 133 determines an enlargement limit size for enlarging the size of the plurality of display elements and a policy for enlarging the size of the plurality of display elements according to the characteristics of each of the specified plurality of display elements. To do. The determination processing unit 133 has a function of expanding the size of each of the plurality of display elements based on the determined expansion limit size and the policy. Such a determination processing unit 133 is realized by, for example, a processor that executes a control program.

  Here, the enlargement limit size is equal to or larger than each size corresponding to each of the plurality of display elements and is indicated by the screen information on the basis of each size that is a size before the size of each of the plurality of display elements is enlarged. It is less than the target screen size. That is, the enlargement limit size is equal to or larger than the reference size of the display element and is equal to or smaller than the size of the target screen (less than the screen width and the screen height) when the original display element size is used as a reference.

  The policy that the determination processing unit 133 enlarges the size of the plurality of display elements is to enlarge the size of the plurality of display elements while maintaining the aspect ratio of the display mode size of each of the plurality of display elements. Enlarging the size of a plurality of display elements without maintaining the aspect ratio of the display mode size of each display element, and displaying each of the display modes without maintaining the aspect ratio of the size of each display mode Is expanded at least in a specific direction.

  For example, when enlarging the size of a plurality of display elements while maintaining the aspect ratio of the display mode size of each of the plurality of display elements, images and characters that require the display element to maintain the aspect ratio of the display mode size This is the case of a row. In this case, the determination processing unit 133 enlarges the size of the display element while maintaining the image aspect ratio, the character string width, the character string height ratio, and the like.

  For example, when expanding the size of a plurality of display elements without maintaining the aspect ratio of the display mode size of each of the plurality of display elements, the display element does not need to maintain the aspect ratio of the size of the display mode The display element is a button or the like having a function to be operated. In this case, the determination processing unit 133 enlarges the button regardless of the aspect ratio of the button or the like.

  For example, if each display mode is enlarged only in a specific direction without maintaining the aspect ratio of the display mode size of each of the plurality of display elements, the display element needs to maintain the aspect ratio of the display mode size. This is the case where there are no display elements, and when the display elements are ruled lines such as frame lines. In this case, the determination processing unit 133 enlarges a ruled line or the like (extends in a predetermined direction) only in a specific direction.

  The determination processing unit 133 is communicably connected to the processing unit 130, and outputs a GUI image indicating a plurality of enlarged display elements to the processing unit 130. The determination processing unit 133 may be mounted in the processing unit 130.

[3-2. Operation of display control device]
Hereinafter, an operation related to a display control method by the display control apparatus 100b having the above-described configuration will be described.

  FIG. 14 is a flowchart illustrating an example of display control processing as a display control method in the display control apparatus 100b. The display control apparatus 100b performs the process of FIG. 14 when displaying a GUI image on a target screen having a size different from the size of the GUI image. The process of FIG. 14 is performed before executing the arrangement determination process and the display mode determination process. Hereinafter, the operation of the display control apparatus 100b will be described with reference to FIG.

  The determination processing unit 133 analyzes the plurality of display elements defined in the display information acquired by the acquisition unit 110 and identifies the characteristics of each of the plurality of display elements (S101). The determination processing unit 133 analyzes whether the display element has a function to be operated, or analyzes a plurality of display elements based on the size and shape of the display element, The characteristics of each of the plurality of display elements are specified. For example, if the display element has a function to be operated, the determination processing unit 133 specifies that the button has a function to be operated. In addition, the determination processing unit 133 determines that the display element is an image if the size of the display element is large based on the size and shape of the display element and the shape of the display element is a square or a rectangle close to a square. Is identified. When the shape of the display element is long in a specific direction, the determination processing unit 133 specifies that it is a ruled line such as a frame line.

  Next, the determination processing unit 133 performs processing for enlarging the sizes of the plurality of display elements in accordance with the characteristics of the specified plurality of display elements. The determination processing unit 133 starts a loop for enlarging the size of each display element, and executes the following steps S102 to S105 for each process (S102).

  Next, the determination processing unit 133 determines an enlargement limit size for enlarging the size of the display element according to the characteristics of the specified display element (S103).

  The enlargement limit size is determined by the size of the target screen indicated by the screen information and the first size based on each size before the display element size is enlarged. The determination processing unit 133 determines the size of the display element according to the characteristics of the display element based on the enlargement limit size. For example, when the display element is a button having a function to be operated, it is only necessary to secure a size that can be operated by the user. Therefore, for example, the determination processing unit 133 increases the size of the button from 1 to about 2 times. Decide between. In the present embodiment, the button size is preferably about 1.2 times. In addition, when the display element is an image, a ruled line, and the like, it should be as large as possible so that the user can easily recognize it. It is determined to be an arbitrary multiple up to a multiple of the same size as the screen. When the display element is a character string, it is only necessary to ensure a size that can be visually recognized by the user. Therefore, the determination processing unit 133 determines the size of the character string between 1 and 2 times. Further, the ruled lines need only be enlarged in a specific direction, and for example, it is not necessary to enlarge (thicken) the thickness.

  Next, the determination processing unit 133 determines a policy for enlarging the size of the display element according to the characteristic of the specified display element (S104).

  For example, when the display element is a button having a function to be operated, the determination processing unit 133 enlarges the size of the display element without maintaining the aspect ratio of the button size. When the display element is an image, a character string, or the like, the determination processing unit 133 enlarges the size of the display element while maintaining the aspect ratio of the size of the image, the character string, or the like. When the display element is a ruled line or the like, the determination processing unit 133 enlarges the size of the display element without maintaining the aspect ratio of the size of the ruled line or the like.

  This is because if the aspect ratio of the image, character string, etc. is not maintained, the appearance will deteriorate, and it will be difficult for the user to recognize the information shown in the image, character string, etc. For this reason, it is necessary to maintain the aspect ratio of images, character strings, and the like.

  Next, the determination processing unit 133 enlarges the size of the display element according to the enlargement limit size for enlarging the size of the display element and the policy for enlarging the size of the display element determined in Steps S103 and S104. (S105). That is, the determination processing unit 133 enlarges the GUI image. In addition, when enlarging the GUI image, the determination processing unit 133 may not only increase the size of the display element but also enlarge the blank portion of each display element in the same manner. Specifically, the blank is composed of Margin indicating a blank portion outside the display element and Padding indicating a blank portion inside the border of the display element. In the present embodiment, the size of the blank portion may be increased similarly.

  The determination processing unit 133 performs processing for enlarging the sizes of the plurality of display elements for all the display elements. When the process of enlarging the display element size is performed on all display elements, the determination processing unit 133 ends the loop of enlarging the size of each display element. The determination processing unit 133 outputs to the processing unit 130 a GUI image showing a plurality of display elements whose sizes have been enlarged in step S105.

  The processing unit 130 adjusts the sizes of the plurality of display elements so that the plurality of display elements enlarged in step S105 fit within the target screen (S106). That is, based on the enlargement limit size determined according to the characteristics of the display element in step S103, the display control apparatus 100b enlarges the size of each of the plurality of display elements by performing the processing shown in FIG. The sizes of the plurality of display elements are adjusted so that the display elements do not become smaller than the respective previous sizes (original sizes of the display elements) and the plurality of enlarged display elements fit within the target screen. Then, the display control device 100b ends the display control process.

  FIG. 15 is a diagram showing an arrangement example of a plurality of display elements constituting the GUI image in FIG. In FIG. 15A, the original GUI image is displayed on the target screen 400a. In FIG. 15A, for example, four display elements (lists) 401 to 404 are displayed in the GUI image.

  B of FIG. 15 is a result of performing the processing of steps S102 to S105 of FIG. 14 when displaying the original GUI image on the target screen 400b having a size different from that of the target screen 400a. In FIG. 15B, the GUI image is enlarged. At this time, the determination processing unit 133 enlarges the display element 410 that is an image to the limit of the size of the target screen 400. The determination processing unit 133 similarly enlarges the space between each display element (indicated by a white arrow). 15B, a display element 405 is further added to FIG. 15A, and five display elements 401 to 405 are displayed on the GUI image. When the display processing unit 410 enlarges the display element 410, the determination processing unit 133 may increase the number of display elements by several or may decrease by about several. When the determination processing unit 133 increases or decreases the number of display elements, for example, the number of display elements may be increased or decreased for each type of display element such as an image, a character string, or a ruled line such as a frame line. It is preferable that the number after the display elements are increased or decreased is about five. This is because, from the viewpoint of user experience, it is generally preferable that the number of items (number of options) in the list displayed on the same screen is 7 ± 2. Taking this embodiment as an example, when the display element is enlarged, the determination processing unit 133 displays four display elements 401 to 404 displayed on the target screen 400a and five display elements 401 to 401 on the target screen 400b. The number of display elements may be increased so that 405 is displayed. Note that the number of display elements to be displayed when the display elements are enlarged may be arbitrarily changed by the display control device.

  15C, the sizes of the plurality of display elements are adjusted so that the GUI image of FIG. 15B fits within the target screen. In this way, the arrangement of the plurality of display elements that make up the GUI image is changed.

[3-3. Effect]
According to the above-described display control apparatus 100b, by specifying the characteristics of each of the plurality of display elements, the enlargement limit size and the size for enlarging the size of the plurality of display elements according to each specified characteristic are increased. A policy is determined. Thereby, the size of the display element can be automatically enlarged based on the determined policy. For this reason, a GUI image satisfying a predetermined display standard or the like can be displayed on a display that has a different size (width and height of the image) and aspect ratio from the GUI image.

(Other embodiments, etc.)
As described above, Embodiments 1 and 2 have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to this, and can also be applied to embodiments in which changes, replacements, additions, omissions, and the like are appropriately performed.

  In the above-described embodiment, an example in which the acquisition unit 110 of the display control device 100 acquires display information from the server device 200 has been described, but the acquisition unit 110 may acquire the display information from anywhere. Further, the acquisition unit 110 executes arithmetic processing (for example, analysis processing of the data) based on data recorded on the recording medium of the display control device 100 or data received from an external device, and displays the contents of the display information. It is good also as acquiring the display information by specifying.

  In the first embodiment described above, an example in which the output by the output unit 140 of the display control device 100 is to display a plurality of display elements on the target screen, and in the second embodiment, the output unit of the display control device 100a is shown. An example in which the output by 140 is to transmit display control information indicating the arrangement and display mode of each display element to be used to display a plurality of display elements on the target screen to the terminal device 300 is shown. In addition, the output unit 140 may output information indicating the arrangement and display mode of each display element constituting the GUI image, or the GUI image, using audio, light, or any other medium, and is detachable. It may be output by recording on a simple recording medium (for example, a memory card).

  In the above-described embodiment, the example in which the size constraint information indicates the requirements such as the minimum size for the specific display element is shown. However, the size constraint information indicates the minimum size of the character font in all the display elements that display characters. Or a minimum size requirement for display elements that are all GUI buttons that can be pressed. Note that the user may be able to change the requirements regarding the size indicated by the size constraint information. Thereby, the user can appropriately set the size constraint information according to his / her own eyesight, finger size, and the like.

  In the above-described embodiment, the example in which the arrangement determining unit 131 of the processing unit 130 performs the determination regarding the presence of a display element that does not fit in the upper display area in steps S13 and S31 in the arrangement determining process has been described. In relation to this determination, when there is a display element that does not fit in the upper display area, the arrangement determining unit 131 may output information regarding the display element via the output unit 140. Here, the upper display area may be the entire area of the target screen. The output by the output unit 140 can be realized by, for example, displaying a display element that does not fit in the upper display area, or displaying a parameter related to a display mode such as a size and a position related to the display element. By this output, a person (for example, a user of the display control apparatus 100) who has recognized information related to a display element that does not fit in the upper display area may determine the arrangement of the display element. The person can input information related to the determined arrangement of the display elements (for example, information such as the position of the display elements) to the display control apparatus 100. Then, the arrangement determining unit 131 in the display control apparatus 100 performs steps S15 and S34 in the arrangement determining process based on the input information (that is, information related to the input arrangement of the display elements) (for example, according to the arrangement instructed by a human). It is good also as changing the arrangement | positioning of the display element in the above.

  Moreover, the function sharing (refer FIG. 2) of each part in the display control apparatuses 100 and 100a mentioned above is only an example, can be changed arbitrarily, and each part may be integrated. In addition, the display control devices 100 and 100a may share a part of processing in the processing unit 130 with devices outside the display control devices 100 and 100a (for example, the server device 200, the terminal device 300, and the like).

  Further, the execution order of various processes (for example, the procedures shown in FIGS. 7, 8, 9A, or 9B) in the display control devices 100 and 100a described above is not necessarily limited to the order described above. Instead, for example, the execution order can be changed, a plurality of procedures can be performed in parallel, or a part of the procedures can be omitted. In addition, all or part of various processes (for example, the procedure shown in FIGS. 7, 8, 9A, or 9B) in the display control devices 100 and 100a described above are realized by hardware such as a dedicated electronic circuit. Alternatively, it may be realized using a processor and software. Note that the processing by software is realized by a processor included in the display control device 100 or 100a executing a program stored in the memory (for example, a control program for GUI image display control). Further, the program may be recorded and distributed on a non-transitory recording medium such as a computer-readable ROM, optical disk, or hard disk. For example, the distributed control program for display control is installed in the display control device 100 or the display control device 100a and is executed by the processor of the display control device at the installation destination, thereby causing the display control device to perform various processes (for example, 7, 8, 9 </ b> A, or 9 </ b> B or the like can be performed. The control program for display control of the GUI image may be stored in advance in a recording medium (for example, the storage unit 120) of the display control devices 100 and 100a, or is transferred to the recording medium via a wide area communication network including the Internet. It may be supplied. The control program for display control includes steps (procedures) for executing display control processing. This display control process includes, for example, display information that is defined for a plurality of display elements to be displayed on a screen used as a GUI and includes structure information indicating an inclusion relationship between the plurality of display elements, An acquisition step (for example, steps S1 and S2) for acquiring screen information indicating the form of a target screen, which is a screen for displaying a plurality of display elements, and an arrangement of one or more display elements among the plurality of display elements in advance Information for determining the arrangement, the size constraint information in which the requirements regarding the size of each of the display elements of the plurality of display elements in order to ensure operability are predetermined, and the display information Based on the form of the target screen indicated by the screen information, the plurality of display elements can be accommodated in the target screen, and the size constraint among the plurality of display elements As for the display elements whose requirements are defined in the report, the layout determination process for determining the layout of each of the plurality of display elements and the display mode determination process for determining the display mode of each of the plurality of display elements so that the requirements are satisfied And a processing step (for example, step S3) and an output step (for example, step S4) for performing output in accordance with the results of the layout determining process and the display mode determining process in that processing step.

In addition, all or a part of the functional blocks as the functional components in the display control devices 100 and 100a described above are individually made into one chip by a semiconductor device such as an IC (Integrated Circuit) and an LSI (Large Scale Integration). Alternatively, it may be integrated into one chip so as to include a part or all of it. The LSI integrated into one chip may be a system LSI that includes a processor, a ROM, a RAM, and the like and functions as a computer. Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. FPGA (Field Programmable Gate Array) that can be programmed after LSI manufacturing
Alternatively, a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used. Furthermore, if integrated circuit technology that replaces LSI appears as a result of progress in semiconductor technology or other derived technology, functional blocks may be integrated using this technology. Biotechnology can be applied as a possibility.

  In addition, forms realized by arbitrarily combining the components and functions described in the above-described embodiments are also included in the scope of the present disclosure.

  The present disclosure is applicable to a display control device, a display control system, and the like for displaying a GUI image.

100, 100a Display control device 101, 301 Processor (microprocessor)
102, 302 Memory 110 Acquisition unit 120 Storage unit 130 Processing unit (first processing unit)
131 Arrangement Determining Unit 132 Display Mode Determining Unit 133 Determination Processing Unit (Second Processing Unit)
140 output unit 200 server apparatus 201 communication path 300 terminal apparatus 400, 400a, 400b target screen 401-405, 410, 420, 431-434 display element

Claims (20)

Display information that is defined for a plurality of display elements to be displayed on a screen used as a GUI (Graphical User Interface) and includes structure information indicating an inclusion relationship between the plurality of display elements, and the plurality of displays An acquisition unit for acquiring screen information indicating a form of a target screen that is a screen for displaying an element;
In the display mode of each of the one or more display elements of the plurality of display elements, in order to ensure operability, and predetermined arrangement determination information about the arrangement of one or more display elements of the plurality of display elements Based on the size constraint information in which requirements regarding size are determined in advance and the display information, the plurality of display elements are accommodated in the target screen according to the form of the target screen indicated by the screen information, and the Among the plurality of display elements, an arrangement determining process for deciding the arrangement of each of the plurality of display elements so that the requirement is satisfied for the display elements for which the requirement is determined by the size constraint information, and each of the plurality of display elements A first processing unit that executes a display mode determination process for determining a display mode of
A display control apparatus comprising: an output unit that performs output according to a result of the arrangement determination process and the display mode determination process by the first processing unit. The structure information is expressed by associating each of the plurality of display elements with each node in a tree structure of a plurality of hierarchies, associating the display element having the largest display area with the highest root node, It is information in which nodes corresponding to one or more display elements to be included in the display area of the display element corresponding to the node are arranged under the upper node,
The first processing unit executes temporary arrangement processing that temporarily determines the arrangement of the plurality of display elements so as to satisfy the requirement defined by the size constraint information based on the display information and the display mode determination processing As a result of executing, for one or more display elements of the plurality of display elements, if the display elements do not fit in the target screen, so that the display elements fit in the target screen, If the display element does not fit in the display area related to the node higher than the node corresponding to the display element in the tree structure, the placement determination process is executed so that the display element fits in the display area The display control apparatus according to claim 1. The first processing unit temporarily determines the arrangement of the plurality of display elements and, as a result of executing the display mode determination process, one display element of the plurality of display elements does not fit in the target screen. In the case where the arrangement information does not fit within the display area related to the node higher than the node corresponding to the one display element in the tree structure of the structure information, The display control apparatus according to claim 2, wherein a relative arrangement of display elements corresponding to each node immediately below a node one level higher than a node in the tree structure is changed. The placement determination information corresponds to first placement information indicating the placement of each of the plurality of display elements to be determined in the provisional placement processing, and each node immediately below one node in the tree structure of the structure information Each display element includes second arrangement information indicating a different arrangement from the arrangement indicated by the first arrangement information,
The display control apparatus according to claim 3, wherein the first processing unit executes the provisional arrangement process according to the first arrangement information, and executes the arrangement determination process according to the second arrangement information. The placement determination information includes placement constraint information in which requirements regarding relative placement for all or part of the plurality of display elements are predetermined,
The display control apparatus according to claim 3, wherein the first processing unit executes the provisional arrangement process and the arrangement determination process so as to satisfy the requirement defined by the arrangement constraint information. The first processing unit temporarily determines the arrangement of the plurality of display elements and, as a result of executing the display mode determination process, one display element of the plurality of display elements does not fit in the target screen. If it does not fit within the display area related to the node higher than the node corresponding to the one display element in the tree structure of the structure information, 1 of the node in the tree structure corresponding to the one display element Specify the width and height of the display area that should contain the display elements corresponding to all the nodes directly below the upper node of the hierarchy, and so as to contain the display element that should be contained in the display area based on the specific result, The display control apparatus according to any one of claims 3 to 5, wherein the arrangement determining process is executed. The first processing unit temporarily determines the arrangement of the plurality of display elements and, as a result of executing the display mode determination process, one display element of the plurality of display elements does not fit in the target screen. Or in the case where the structure information does not fit in the display area related to the node higher than the node corresponding to the one display element in the tree structure, one level of the node in the tree structure corresponding to the one display element When display elements corresponding to two or more nodes immediately below the upper node are arranged in a matrix, specify the width and height of the display area in which each display element arranged in the matrix is to be stored And by changing the number of rows and the number of columns in the matrix arrangement so as to accommodate the display elements that should be accommodated in the display area based on the specific result, The display control device according to any one of claims 3-5 to perform location determination process. The size constraint information is information that defines the minimum size of each display element as the requirement regarding the size in the display mode of each of one or more display elements among the plurality of display elements,
The first processing unit executes the display mode determination process at least after execution of the arrangement determination process,
In the display mode determination process executed after execution of the arrangement determination process, the first processing unit is configured to display a display element whose minimum size is defined as the requirement in the size constraint information among the plurality of display elements. If the display element can be displayed in a size larger than the minimum size within the display area of the upper node of the node corresponding to the display element in the tree structure of the structure information, the size becomes the large size. The display mode of the display element is determined as described above, and a size larger than the minimum size in a range in which the display element is contained in a display area related to a node higher than the node corresponding to the display element in the tree structure of the structure information The display mode of the display element is determined so as to be the minimum size when it cannot be displayed by the method according to any one of claims 2 to 7. The display control apparatus described. In the display mode determination process executed after execution of the arrangement determination process, the first processing unit is configured to display a display element whose minimum size is defined as the requirement in the size constraint information among the plurality of display elements. If the display element can be displayed in a size larger than the minimum size within the display area related to the upper node of the node corresponding to the display element in the tree structure of the structure information, the maximum size and The display control apparatus according to claim 8, wherein the display mode of the display element is determined so as to be. The display information includes display mode information that specifies a display mode of each of one or more display elements among the plurality of display elements,
The first processing unit executes the provisional arrangement process and the display mode determination process so that a display element whose display mode is specified by the display mode information among the plurality of display elements is subjected to the specification. The display control apparatus according to any one of claims 2 to 9. The first processing unit, as a result of executing the provisional arrangement process and the display mode determination process, for one or more display elements of the plurality of display elements, the display elements are within the target screen. If there is no display element, or if the display element does not fit within the display area related to the node higher than the node corresponding to the display element in the tree structure, the display unit outputs information on the display element, and the display The display control apparatus according to any one of claims 2 to 10, wherein after the output of information related to an element, input information relating to the arrangement of the display element is acquired and the arrangement determination process is executed based on the input information. The output unit displays each of the plurality of display elements on the target screen in an arrangement and a display mode according to the result of the arrangement determination process and the display mode determination process by the first processing unit, and thereby the output The display control apparatus according to any one of claims 1 to 11. The output unit displays and displays each of the plurality of display elements on the target screen according to a result of the arrangement determination process and the display mode determination process by the first processing unit. The display control device according to any one of claims 1 to 11, wherein the output is performed by transmitting display control information defining a mode to the outside of the display control device. Further, before executing the arrangement determination process and the display mode determination process, the display elements specified in the display information acquired by the acquisition unit are analyzed to identify the characteristics of the display elements. Then, an enlargement limit size for enlarging the size of the plurality of display elements and a policy for enlarging the size are determined according to the specified characteristics, and the plurality of the display elements are determined based on the determined enlargement limit size and the policy The display control apparatus as described in any one of Claims 1-13 provided with the 2nd process part which expands the size of each display element. The enlargement limit size is equal to or larger than each size corresponding to each of the plurality of display elements on the basis of each size before enlarging the size of each of the plurality of display elements, and the target indicated by the screen information The display control apparatus according to claim 14, wherein the display control apparatus is smaller than a screen size. The policy includes expanding the size of the plurality of display elements while maintaining the aspect ratio of the display mode size of each of the plurality of display elements, and setting the aspect ratio of the display mode size of each of the plurality of display elements. Enlarging the size of the plurality of display elements without maintaining, at least enlarging each display mode only in a specific direction without maintaining the aspect ratio of the size of the display mode of each of the plurality of display elements The display control apparatus according to claim 15, wherein any one of them is included. The second processing unit analyzes the plurality of display elements and specifies the characteristics of each of the plurality of display elements depending on at least whether or not the plurality of display elements have a function to be operated. The display control apparatus according to any one of 14 to 16. The second processing unit further analyzes the plurality of display elements based on the size and shape of the plurality of display elements, and identifies the characteristics of each of the plurality of display elements. Display controller. A display control method used in a display control device,
Display information that is defined for a plurality of display elements to be displayed on a screen used as a GUI (Graphical User Interface) and includes structure information indicating an inclusion relationship between the plurality of display elements, and the plurality of displays An acquisition step for acquiring screen information indicating a form of a target screen that is a screen for displaying an element;
In the display mode of each of the one or more display elements of the plurality of display elements, in order to ensure operability, and predetermined arrangement determination information about the arrangement of one or more display elements of the plurality of display elements Based on the size constraint information in which requirements regarding size are determined in advance and the display information, the plurality of display elements are accommodated in the target screen according to the form of the target screen indicated by the screen information, and the Among the plurality of display elements, an arrangement determining process for deciding the arrangement of each of the plurality of display elements so that the requirement is satisfied for the display elements for which the requirement is determined by the size constraint information, and each of the plurality of display elements Processing steps for executing a display mode determination process for determining the display mode of
A display control method comprising: an output step of performing output in accordance with the result of the arrangement determining process and the display mode determining process in the processing step. A control program for causing a display control device including a microprocessor to execute display control processing,
The display control process includes:
Display information that is defined for a plurality of display elements to be displayed on a screen used as a GUI (Graphical User Interface) and includes structure information indicating an inclusion relationship between the plurality of display elements, and the plurality of displays An acquisition step for acquiring screen information indicating a form of a target screen that is a screen for displaying an element;
In the display mode of each of the one or more display elements of the plurality of display elements, in order to ensure operability, and predetermined arrangement determination information about the arrangement of one or more display elements of the plurality of display elements Based on the size constraint information in which requirements regarding size are determined in advance and the display information, the plurality of display elements are accommodated in the target screen according to the form of the target screen indicated by the screen information, and the Among the plurality of display elements, an arrangement determining process for deciding the arrangement of each of the plurality of display elements so that the requirement is satisfied for the display elements for which the requirement is determined by the size constraint information, and each of the plurality of display elements Processing steps for executing display mode determination processing for determining the display mode of
A control program comprising: an output step for performing output in accordance with the result of the arrangement determining process and the display mode determining process in the processing step.

Images