JP2010092082A - Content information processing method, content information processing program, and content information processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To optimize a behavior of bi-directional digital content created in supposing a key input to that which is suitable for input by a pointing device such as a touch panel. <P>SOLUTION: A content information processing method includes steps of: generating a first document tree by performing the syntax analysis of a markup document; detecting an element in which a predetermined type of event handler as a substance is set from the first document tree; generating a second document tree based on the markup document; determining the element in which the event handler is set as a substance when the display of other elements is changed by executing the predetermined type of event handler, and determining the element whose display has been changed as a virtual body associated with the substance; and performing processing for focusing the substance when the virtual body associated with the substance receives a predetermined pointer operation. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a content information processing method, a content information processing program, and a content information processing apparatus.

  In recent years, with the development of digital terrestrial broadcasting networks, not only home-use fixed digital television receivers but also mobile terminals such as mobile phones equipped with a reception function for one-segment broadcasting, which is a form of terrestrial digital broadcasting, have spread rapidly. I am doing. The main feature of digital broadcasting is that multimedia contents are provided by data broadcast signals in addition to providing conventional broadcast programs by video / audio signals. Since the format of multimedia content in digital broadcasting is defined based on Web technology, it is possible to provide a service that seamlessly links broadcasting and the Internet.

  Data broadcasting contents in Japan are described in BML (Broadcast Markup Language), which is an XML (Extensible Markup Language) based markup language. The specification of BML is defined in ARIB STD-B24 established by the Association of Radio Industries and Businesses (ARIB). This specification is based on the premise of user input using direction keys such as the cross key on fixed digital TV remote controllers and many conventional mobile terminals. It is produced assuming user input by.

  Recently, PCs (Personal Computers) equipped with digital TV tuners are increasing, and an effective technique for browsing BML contents using a pointing device such as a mouse is being studied (Patent Document 1). In addition, mobile terminals equipped with a pointing device such as a touch panel are increasing, and there are increasing opportunities to browse data broadcast contents using the pointing device.

  By the way, a normal browser has a mechanism for clearly indicating that the element is focused by changing the display such as enclosing the focused element with a broken line or a thick line or applying a shadow. However, the mode of display change when an element is focused varies depending on the type and setting of the browser, which is one factor that impairs the uniformity of content design. In addition, since a pointer is not displayed on a screen in a normal BML browser in which a user input by a pointing device such as a mouse, that is, a pointer operation is not assumed, a setting is made so that a focused element is surrounded by a thin dotted line. In such a case, there is a problem that it is difficult to understand which element is focused.

  For this reason, many data broadcasting content creators produce BML documents with a special structure as shown in FIG. 4 in order to improve the design and visibility when elements are focused. That is, the actual focus targets S1 to S4 such as the object element are displayed in an inconspicuous manner (or the display is made invisible), while the focus states of the focus targets S1 to S4 (whether or not they are focused). A mechanism for displaying the images V1 to V4 that change in conjunction with each other in a conspicuous manner is incorporated in the BML document. In the following description, the actual focus target displayed in such a manner that the user cannot visually recognize (or cannot visually recognize) such as S1 to S4 is linked to the “entity” and the focus state of the entity such as V1 to V4. An element whose display changes is called a “virtual body”. A content including an entity and a virtual body is referred to as “entity / virtual body content”.

In FIG. 4, S1 is focused, and the image V1 corresponding to S1 is a highlighted image. Further, the images V2 to V4 corresponding to S2 to S4 which are not focused are normal display images. Here, when the user presses the cross key downward, the focus shifts to S2 arranged immediately below S1, the image V1 corresponding to S1 is the normal display image, and the image V2 corresponding to S2 is highlighted. The display is switched (FIG. 5). In this way, the user recognizes that the focus has shifted from “today's recommendation” (V1) to “notable new program” (V2) because the images V1 and V2 have changed in conjunction with the operation of the cross key. That is, the user does not notice the presence of the actual focus objects S1 to S4 because of the inconspicuous (or invisible) display mode, and the display of the images V1 to V4 has changed in conjunction with the operation of the cross key. Therefore, the illusion that the images V1 to V4 are focus targets is obtained. With such a content configuration, an excellent visual effect at the time of focusing independent of the model is realized.
JP 2002-232371 A

  Even if the user has the illusion that the virtual object is the true focus target as the content creator thinks, the user intends as long as the user input is made using the direction keys as the content creator assumes. There is no problem because the focus is controlled on the street. However, this illusion causes operational inconvenience when browsing the real / virtual content using a pointing device. In other words, since the virtual body is not a focus target but merely a display, it cannot be selected even if the virtual body is selected by a pointer operation using a pointing device such as a mouse or a touch panel. Therefore, there is a problem that a desired user input cannot be performed. Further, there is a problem that an unexpected event occurs because the pointing device accidentally overlaps with an entity that the user is not aware of.

  The present invention has been made in view of the above circumstances, and the object of the present invention is to make the behavior of digital content created assuming directional key input suitable for input by a pointing device such as a touch panel or a mouse. It is to provide a method, a program, and an apparatus that are optimized.

  In order to solve the above problems, according to the present invention, in an information processing apparatus capable of accepting an operation by a pointing device, a method for performing processing for interactively displaying content on a display based on a document described in a markup language Is provided. The content information processing method provided by the present invention includes a step of parsing a document to generate a first document tree and a predetermined entity that can change a display of a virtual body that is another element when focused. Detecting an element for which an event handler of this type is set from the first document tree, generating a second document tree based on the document, and an entity detected for the second document tree For each of the possible elements, the step of executing each of the predetermined types of event handlers set, and when the display of other elements changes due to the execution of the predetermined types of event handlers, the event handlers are set. And the element whose display has changed is determined as a virtual body associated with the entity. A step, hypotheses associated with the entity and a step of performing processing to focus the entity upon receiving a predetermined pointer operation.

  By optimizing data for interactively displaying content by such a method, the above-described actual / virtual content can be appropriately operated using a pointing device such as a mouse or a touch panel. Here, “displaying the content interactively” means displaying the content in such a manner that a user operation using a pointing date or the like can be performed on an element included in the displayed content.

  The second document tree may be generated by parsing the original markup document, or may be generated by duplicating the first document tree. If the first document tree is duplicated and generated, there is no need to perform syntax analysis again, so the burden on the CPU is reduced and the processing speed is improved.

  The step of performing processing for focusing an entity may include the step of setting an event handler in the virtual body that executes a method for focusing the entity when the pointer is positioned on the virtual body associated with the entity. desirable. There are several possible mechanisms for changing the state of an entity in response to a pointer operation on a virtual body, but one of the simplest methods is to set an event handler on the virtual body. For example, when adding a mechanism in which an entity is focused in response to a pointer operation on a virtual object, when the pointer is positioned on the virtual object associated with the entity (in other words, the pointer is displayed on the display of the virtual object). It is conceivable to set an event handler in the virtual body that executes the method that focuses the entity when the display overlaps). At this time, the event handler set for the virtual body includes, for example, an onmouseover event handler, and the method for focusing an entity includes, for example, the focus method. When setting such an event handler, the setting of the event handler can be held and functioned as an internal process of software such as a browser. However, intermediate data including set event handlers (data corresponding to a document tree, element tree, layout tree, etc., created in an intermediate process from the generation of a markup document to a drawing object) may be generated. In other words, the step of performing the process for focusing the entity may include the step of generating intermediate data including the event handler to be set. With this configuration, the present invention can be implemented relatively easily without significantly changing the structure of software such as a browser.

  Similarly, the content information processing method described above may further include a step of performing processing for clicking on an entity when a virtual body associated with the entity receives a predetermined pointer operation. In this case, the step of performing processing for clicking the entity may include a step of setting an event handler for executing a method for clicking the entity in the virtual body when the virtual body associated with the entity is clicked. desirable. At this time, the event handler set for the virtual body in the step of performing the process for clicking the entity includes, for example, an onclick event handler, and the method for clicking the entity includes, for example, the click method. According to such a configuration, when a pointer operation corresponding to a click on the virtual body is performed, the effect of clicking on the entity is easily realized.

  The above method may further include a step of determining whether or not each of the plurality of virtual bodies is in an inclusive relationship when there are a plurality of virtual bodies associated with one entity. In this case, it is preferable that a virtual body included in another virtual body is configured not to perform a process for focusing an entity when the virtual body receives a predetermined pointer operation. With such a configuration, it is possible to reduce the amount of processing necessary to execute the above method, and to improve the response of content display.

  Further, it is preferable that the step of generating the second document tree is executed when an element in which a predetermined type of event handler is set is detected from the first document tree. Various processes according to the present invention, which are performed using the second document tree, are for validating a user operation on a virtual body. Therefore, when an element for which a predetermined type of event handler that can be a virtual body is set is not detected from the first document tree, various processes according to the present invention need not be performed in the first place. Therefore, with such a configuration, unnecessary processing is not performed by the CPC, and the processing speed can be improved.

  The present invention also provides a program for causing a computer to execute each step of the above method.

The present invention also provides an information processing apparatus that performs processing for interactively displaying content on a display based on a document described in a markup language. An information processing apparatus provided by the present invention includes a pointing device input receiving unit that receives an input operation by a pointing device, a unit that acquires a document, and a first document tree that parses the document and generates a first document tree. Generation means, and entity candidate detection means for detecting an element set with a predetermined type of event handler that can be an entity that changes the display of a virtual body, which is another element when focused, from the first document tree A second document tree generating means for generating a second document tree based on the document, and the predetermined type event set for each of the elements that can be detected entities with respect to the second document tree. Event handler execution means for executing each of the handlers, and a predetermined type When the display of another element is changed by execution of the event handler, the element for which the event handler is set is determined to be an entity, and the element for which the display has been changed is determined to be a virtual body associated with the entity / Virtual body relation determining means, and a focus processing means for performing processing for focusing the entity when the virtual body associated with the entity receives a predetermined pointer operation.

  The second document tree generation means is preferably configured to generate the second document tree by duplicating the first document tree.

  The focus processing means includes first event handler setting means for setting, in the virtual body, an event handler that executes a method for focusing the entity when the pointer is positioned on the virtual body associated with the entity. It is preferable.

  The focus processing means may include intermediate data generating means for generating intermediate data including an event handler set by the first event handler setting means. The event handler set by the first event handler setting means includes, for example, an onmouseover event handler, and the method for focusing an entity includes, for example, the focus method.

  The information processing apparatus may further include a click processing unit that performs processing for clicking an entity when a virtual body associated with the entity receives a predetermined pointer operation.

  The click processing means preferably includes second event handler setting means for setting an event handler that executes a method for clicking an entity when the virtual object associated with the entity is clicked. For example, an onclick event handler is included in the event handler set by the second event handler setting means, and a click method is included in the method for clicking the entity.

  The content information processing apparatus may further include means for determining whether or not each of the plurality of virtual bodies is in an inclusive relationship when there are a plurality of virtual bodies associated with one entity. In this case, when there is an inclusion relationship among a plurality of virtual bodies associated with one entity, the focus processing means receives a predetermined pointer operation for a virtual body included in another virtual body. It is preferable that the processing for focusing the entity is not performed when the camera is used.

  According to the present invention, a content information processing method for optimizing the behavior of content created assuming use of a user input means other than a pointing device such as a cross key to be suitable for input using a pointing device, A content information processing program and a content information processing apparatus are provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an external view of a terminal device 10 according to an embodiment of the present invention. FIG. 1A shows the upper surface, and FIG. 1B shows the lower surface. FIG. 2 is a block diagram illustrating a schematic configuration of the terminal device 10. In this embodiment, the terminal device 10 is a mobile phone terminal. For example, a PDA (Personal Digital Assistants), a PHS (Personal Handy phone System), a portable game machine, a home appliance, a desktop PC, a laptop PC, and a digital TV receiver. It is also possible to replace the terminal with another type of terminal equipped with various digital content browsers such as a web browser and a digital data broadcasting browser.

  As illustrated in FIG. 2, the terminal device 10 includes a CPU (Central Processing Unit) 103 that performs overall control of the device. Each element constituting the terminal device 10 is mutually connected to the CPU 103 via the system bus 119. Each element constituting the terminal device 10 includes a ROM (Read Only Memory) 105, a RAM (Random Access Memory) 107, a wireless communication unit 108, a touch panel sensor 109, a display 110, a speaker 111, a microphone 112, and a VRAM (Video Random Access). Memory) 115, a power supply unit 116, and the like. The system bus 119 includes a power supply line in addition to the data line. The touch panel sensor 109 is formed integrally with the substrate on the transparent substrate that constitutes the display screen of the display 110 and constitutes a touch panel display together with the display 110. Each element of the terminal device 10 including the CPU 103 is supplied with power from a power supply unit 116 including a battery via a power supply line. Power supply from the power supply unit 116 is controlled by a power supply On / Off key 117.

  The ROM 105 is a non-volatile semiconductor memory such as a flash memory, and is an auxiliary storage device that stores various application programs such as a browser 1051 and various setting data in addition to an OS (Operating System) 1050 for a portable terminal. is there. The browser 1051 is an application program that enables browsing of data broadcast contents described in BML (Broadcast Markup Language), etc., in addition to Web contents described in HTML (Hypertext Transfer Protocol), XHTML (eXtensible HyperText Markup Language), etc. It is. Note that BML is an XML (Extensible Markup Language) -based data broadcasting description language formulated by the Japan Radio Industry Association.

  The CPU 103 accesses necessary hardware via the system bus 119 after the terminal device 10 is powered on. For example, immediately after powering on the terminal device 10, the CPU 103 accesses the ROM 105, reads an OS (Operating System) 1050, loads it into the RAM 107, and starts it. As a result, each element performs a cooperative operation under the management of resources and processes by the OS 1050 to execute various processes.

  For example, when an operation for starting the browser 1051 by a finger touch on the touch panel sensor 109 is performed, the OS 1050 loads the browser 1051 into the RAM 107 and starts it in order to operate the browser 1051 on the OS 1050. A browser 1051 activated by the OS 1050 performs processing in cooperation with hardware such as the CPU 103 in accordance with a user's access to Web content, a data broadcast reception operation, or the like. As a result of such processing, the user can view Web content, view data broadcasts, and the like through the display 110.

  The wireless communication unit 108 extracts a signal of a predetermined frequency from the radio wave received by the antenna 108a and converts it into a baseband signal, AD conversion of the baseband signal, OFDM (Orthogonal Frequency Division Multiplexing) demodulation and error correction And a demodulator that outputs an MPEG-2 TS (MPEG-2 Transport Stream) stream transmitted from a broadcasting station, and separates each signal (ES: Elementary Stream) of video, audio, and data from the MPEG-2 TS stream It has a decoding unit for decoding and outputting. The browser 1051 displays various broadcasts on the display 110 using the video, audio, and data ES supplied from the wireless communication unit 108 and outputs audio signals from the speaker 111.

  The display 110 has a capacitive touch panel formed on the surface of an organic EL (Electro-Luminescence) display. When the user touches the display on the touch panel display 110 (109) with a fingertip or a stylus, the position on the touch panel display where the change in capacitance is caused is detected, and this input position information is output from the touch panel sensor 109. . The OS 1050 (CPU 103) interprets the user input based on the display position information of each element displayed on the display 110, the input position information detected by the touch panel sensor 109, and its time change, and processing based on the interpretation result I do. For example, it is assumed that when an icon corresponding to the data broadcast reception function is displayed on the standby screen, the user touches and releases the icon for a moment. The touch panel sensor 109 sends the position where the capacitance is reduced, that is, the position information of the user input, to the CPU 103 only while the capacitance of the touch panel is reduced due to the finger touch of the user. The OS 1050 determines an element (data broadcast reception function icon) displayed at the display position of the display 110 corresponding to the input position of the touch panel sensor 109 based on the received input position information. Further, the OS 1050 determines that the user input corresponds to “click” because the user input position does not change and is completed in a short time. Accordingly, the OS 1050 interprets that the user input is a command to start data broadcast reception, and instructs the browser 1051 to start reception of data broadcast. In this manner, the user touches an element displayed on the display 110 (109) with a touch panel function with a fingertip or a stylus, thereby selecting an element, inputting an instruction associated with the element, inputting a handwritten character, inputting a graphic, etc. User input can be performed intuitively.

  Note that the display device method of the display 110 is not limited to the organic EL method, and may be another display method such as a liquid crystal display. The touch panel method is not limited to the capacitive method, and may be another method such as a resistive film method or a surface acoustic wave method. The touch panel is not limited to a touch panel, and may be another type of pointing device such as a touch pad, a pointing stick, a trackball, and a mouse.

  Furthermore, the terminal device 10 may include a user input device including a motion sensor such as a gyro sensor, an acceleration sensor, or a jerk sensor. In this case, for example, the CPU 103 monitors the outputs of various sensors and measures the tilt angle, angular velocity, acceleration, jerk (time change rate of acceleration), etc. of the terminal device 10 main body, and the screen scroll corresponding to the measured value is the CPU 103. And the browser 1051 are implemented. For example, when the user moves or tilts the terminal device 10 main body up, down, left, or right, screen scrolling is performed according to the moving direction, speed, tilt angle, and the like. Therefore, the user can more intuitively scroll the content display area.

  Next, functions of the browser 1051, particularly the browser engine 30 which is a basic component of the browser 1051, will be described in detail. FIG. 3 is a functional block diagram of the browser engine 30. As shown in FIG. 3, the browser engine 30 includes functional blocks of a parser 31, a page maker 32, a page editor 33, and a virtual body processing function unit 34 characteristic of the embodiment of the present invention described later. Yes.

  When an operation for starting data broadcast viewing, for example, an icon indicating a data broadcast reception function displayed on the standby screen of the display 110 receives the above-described finger touch corresponding to a mouse click operation, the browser 1051 is activated and wireless Reading of the broadcast signal supplied by the communication unit 108 is started. The data broadcast signal is a module composed of a startup file, a file list (DII), a markup document 21 described in BML constituting each page of the data broadcast content, and each monomedia file (for example, PNG format image file). Is a data stream of a DSM-CC (Digital Storage Media Command and Control) Carlousel transmission system compliant with the ARIB standard. When the browser 1051 reads one unit (one period) of the data broadcast signal (data ES in the broadcast signal), the browser 1051 extracts the markup document 21 and each monomedia file based on a predetermined data format.

  The parser 31 parses the markup document 21 extracted from the ES, and generates a document tree 23 (for example, a DOM (Document Object model) tree) expressing the logical structure of the markup document 21 in a tree structure from the analysis result. Generated and expanded in the RAM 107. The document tree 23 is data representing an outline of the markup document 21 having each element constituting the markup document 21 as a node. The document tree 23 only represents the logical structure of the markup document 21 and does not include information related to the expression of the document.

  The page maker 32 generates an element tree 25 including information on the display style (p, div, span, etc.) determined by each tag based on the document tree 23 and is specified in the markup document 21. An image request, event processing registration, and the like are performed and stored in the RAM 107. The element tree 25 expresses in what order and inclusion relation the elements such as p, div, span, etc. exist, but where these elements are displayed in what width and height on the screen. Or information about layout such as where text is wrapped.

  The page editor 33 displays the layout of each node (element) of the element tree 25 (that is, the text wrapping of each element, the position on the screen, the width, the height, etc.) according to the set display magnification and display mode. A color or the like is determined one by one, and a layout tree 27 representing the layout of each element is generated.

  The elements whose layout has been determined are sequentially rendered by the page editor 33 in the image buffer 1071 created in a predetermined area of the RAM 107. In the image buffer 1071, elements existing in a region scheduled to be displayed on the display 110 and its peripheral region are rendered. That is, the rendering result (drawing object) of the entire area of the image buffer 1071 rendered by the page editor 33 is not displayed on one screen. The display 110 displays a rendering result of an area corresponding to a window for displaying content on the image buffer 1071 designated by the browser 1051 (hereinafter, this area is referred to as “designated area”).

  The VRAM 115 is a memory that holds contents directly displayed on the display 110 and has an image area corresponding to the screen size of the display 110. In the VRAM 115, the rendered contents are written in a designated area of the image buffer 1071 together with an image such as a toolbar. Therefore, when elements whose layout has been determined are sequentially rendered in the image buffer 1071, the rendering results are sequentially written in the VRAM 115. Thereby, each element of content is displayed on the display 110 one after another. The content is displayed on the display 110 through the above processing by the browser 1051.

  The above are the elements and operations of the browser 1051 necessary for displaying normal content. The browser 1051 of this embodiment further includes a virtual body processing function unit 34 that performs processing for optimizing the behavior of the entity / virtual body content to that suitable for touch panel input. Hereinafter, a characteristic process performed when the browser 1051 of the present embodiment displays the actual / virtual content will be described in detail.

  First, the configuration of the markup document 21 that is entity / virtual content will be described. FIG. 4 shows a screen display example 21 ′ of the markup document 21. The markup document 21 is a BML file describing data broadcasting content. Small rectangular displays S1 to S4 arranged at the left end of the display screen 21 'are display of object elements as "substances". For example, the entities S1 to S4 are set to be displayed as small as possible (for example, in a size of 1 pixel × 1 pixel), arranged at the edge of the screen, or displayed in a similar color to the background. It is displayed in an inconspicuous manner that is difficult for the user to notice. Moreover, entity S1-S4 may be arrange | positioned so that it may be hidden behind the other element so that a user cannot visually recognize.

  On the other hand, “Visitor's recommendation”, “Featured new program”, “Breaking news”, and “Weather” displays V1 to V4 in the center of the screen are displayed as “virtual objects” corresponding to the entities S1 to S4, respectively. This is a representation of the element. An onfocus event handler (onfocus attribute) is set in the object elements S1 to S4. While the object elements S1 to S4 are focused, the image files of the corresponding virtual bodies V1 to V4 are different files (highlighted images). Script to replace (file).

  The markup document 21 is configured so that the object element S1 is focused by default. In the present embodiment, the browser 1051 is set so that a thick line is displayed around the focused element. Therefore, in FIG. 4, the focused object element S1 is displayed surrounded by a thick line. Further, an onfocus event handler is set for the object elements S1 to S4, and the virtual bodies V1 to V4 are displayed from the normal display image (the display patterns of V2 to 4 in FIG. 4) while the object element is focused. A script for switching to a highlighted image (display pattern V1 in FIG. 4) is executed. Further, an onclick event handler is set in each of the object elements S1 to S4. For example, the event handler is triggered by the occurrence of an event corresponding to a mouse click operation such as a pressing operation of a determination key of a fixed television receiver remote control. The set predetermined process is executed. The markup document 21 is configured such that when the onclick event handler is executed, the screen display transitions to a page corresponding to the items indicated by the displays V1 to V4.

  When viewing the above data broadcast content on a fixed television receiver or the like, if the down key of the cross key on the remote controller is pressed while the display of FIG. 4 is displayed, the screen display changes to the state of FIG. That is, the focus moves from the object element S1 to the object element S2 arranged below S1, and accordingly, the display of the virtual body V1 is highlighted from the highlighted image to the normal display image, and the display of the virtual body V2 is highlighted from the normal display image. The display image has been switched. However, since the virtual bodies V1 to V4 are not a target of focus but merely a display, a pointer is placed on the virtual bodies V1 to V4 displayed on the display 110 with the touch panel function unless processing characteristic of the present invention described later is performed. Even if they are combined, the virtual bodies V1 to V4 cannot be highlighted (that is, the focus can be shifted to the entities S1 to S4 corresponding to the virtual bodies V1 to V4).

  FIG. 6 is a flowchart showing processing when the browser 1051 of the present embodiment displays data broadcast content. FIG. 7 shows details of step 5 (virtual body detection processing) in the flowchart of FIG. In the following description and drawings, each step of processing is abbreviated as ST.

  The process of FIG. 6 starts with a user input or the like instructing the start of viewing data broadcasting as a trigger. When the processing in FIG. 6 starts, the browser 1051 acquires the ES of the data broadcast provided from the wireless communication unit 108, extracts the markup document 21 from the ES, and acquires it (ST1). Next, the parser 31 of the browser 1051 analyzes (parses) the syntax of the markup document 21, creates a document tree 23, and expands it in the RAM 107 (ST2).

  Next, the virtual body processing function unit 34 searches the document tree 23 to detect a predetermined element (entity candidate) that can be an entity that changes the display of the virtual body (ST3). In the present embodiment, object elements including only onfocus or both onfocus and onclick event handlers are detected as predetermined elements. However, suitable elements as the predetermined elements are not limited to these elements, and depend on the language in which the markup document 21 is described. For example, when the markup document is described in HTML, there are an a element and an input element other than the object element as elements that can be focused, and these elements can be entities with virtual bodies. If a predetermined element is not included from the document tree 23 (ST3: NO), the content described by the markup document 21 is displayed on the display 110 by performing the same processing as the conventional browser. That is, the browser 1051 generates a drawing object through a series of processes of creating the element tree 25 (ST7), creating the layout tree 27 (ST8), and rendering (ST9), and displays the content on the display 110 (ST10). .

  On the other hand, when it is determined that there is a high possibility that the document tree 23 includes a predetermined element, that is, an entity with a virtual body exists (ST3: YES), the browser 1051 detects a virtual body in the content. A document tree 23 used for performing the processing (hereinafter referred to as “virtual object detection processing”) (hereinafter referred to as “virtual object detection document tree 24”). The image is developed in a region different from 23 (ST4). Then, the virtual body processing function unit 34 of the browser 1051 performs a virtual body detection process described below using the copied virtual body detection document tree 24 (ST5).

  Next, details of the virtual body detection process ST5 of the present embodiment will be described with reference to FIG. In the virtual body detection process S5, the virtual body processing function unit 34 first detects an entity that is a focus point actually associated with the virtual body (ST51). This work is called collection. Specifically, an onfocus (and onclick) event handler is executed for each of the predetermined elements detected in the process of ST3 (in this embodiment, only onfocus or an object element including both onfocus and onclick event handlers) Confirms whether or not the virtual object detection document tree 24 is changed. If there is an element changed by the execution of the event handler, it is determined that the element is a virtual body.

  If the virtual body is not detected (ST52: NO), the process proceeds to ST7 in FIG. 6 and returns to the same process as that of the conventional browser. When a virtual body is detected (ST52: YES), next, rearrangement of the parent-child relationship of the virtual body (ST53), that is, when the virtual body associated with the common entity has an inclusion relationship, the highest virtual body Proceed to the process of invalidating the entity / virtual body relationship other than. Details of the arrangement of the parent-child relationship will be described later.

  When the virtual object detection process ST5 for the virtual object detection document tree 24 is finished, the process proceeds to the process for the document tree 23 for screen display. First, the virtual body processing function unit 34 cooperates with the page maker 32 to change the relationship between the entity / virtual body detected in the document tree 23 and the virtual body detection process ST5 (except for the relation invalidated in the process ST53). Based on this, the element tree 25 is created (ST6). The difference between this process ST6 and the normal element tree creation process ST7 is that various additional event handlers are set for the virtual object based on the detected entity / virtual object relationship. In the present embodiment, event handlers of onmouseover, onclick, and onmousedown are set for elements determined as virtual bodies. First, set the onmouseover event handler in the virtual body, and set the focus to move to the entity when the pointer is positioned over the virtual body (specifically, for example, execute the focus () method) . In addition, an onclick event handler is set for the virtual body, and the virtual body is clicked (ie, the click () method is executed) when the virtual body is clicked by the pointer. In addition, an onmousedown event handler is set for the virtual body. As a result, the mousedown event is canceled, and unexpected focus movement or the like can be suppressed. By setting various event handlers in ST6, a pointer operation can be performed on the virtual body, and a desired user input can be performed on the entity / virtual body content using a pointing device such as a touch panel.

  After the element tree 25 is created, the same processing ST8 to ST10 as in the conventional browser is executed, and the actual / virtual content that can be operated by the pointing device using the pointing device is displayed on the display 110.

  Next, the above-described rearrangement of parent-child relationships (ST53) will be described in detail. FIG. 8 is an example of a document tree for explaining the arrangement of parent-child relationships of virtual bodies. FIG. 8A shows a schematic configuration of a document tree before organizing the parent-child relationship, and FIG. 8B shows a schematic configuration of the document tree after organizing the parent-child relationship. FIG. 9 shows a screen display example of the content shown in FIG. Div elements can be grouped by including multiple elements (where the included element is the “parent element”, the included element is the “child (grandchild) element”, and the inclusion relationship is the “parent-child relationship”) "). This inclusion relation also gives the inclusion relation of the display area as shown in FIG. That is, each element (child element, grandchild element,...) Grouped by the parent element (div element or the like) is arranged in the display area of the parent element.

  In the example shown in FIG. 8A, the elements E4 to E6, E8, E9, and E11 are virtual bodies of the element E3 (entity A). Among these, the elements E4 to E6 are in a parent-child relationship. Elements E8 and E9 and E8 and E11 are also in a parent-child relationship. Element E13 is a virtual body of element E12 (entity B).

  First, consider the elements E4 to E6 that have a parent-child relationship. Since element E4 is a virtual body of element E3 (entity A), event handlers of onmouseover, onclick, and onmousedown are set in element E4 in process ST6. That is, when the pointer is positioned within the frame of the element E4 in FIG. 9, the corresponding entity A is focused, and the corresponding method is applied to the entity A according to the pointer operation by the pointing device performed at this time. An event handler is set to be executed. Since the child element E5 and grandchild element E6 of the element E4 are displayed in the display area of the element E4, the event handler set for the element E4 is a pointer in the display area of the child element E5 and grandchild element E6. It is also executed when is located.

  If parent-child relationships are not organized (ST53), event handlers are set (ST6) for the child element E5 and grandchild element E6. However, even if the event handler is not set for the lower elements such as the child element E5 and the grandchild element E6, the same effect as described above can be realized by setting the event handler for the parent element E4. Therefore, in rearranging the parent-child relationship (ST53), the virtual bodies associated with the same entity are detected in the parent-child relationship, and the entities other than the highest (parent) virtual body are detected as entities / A process of invalidating (or deleting) the virtual body relationship is performed. As a result, unnecessary event handlers are not set in the lower-level elements (child element E5 and grandchild element E6) having a parent-child relationship in process ST6, so that the processing amount of the CPU is reduced, the processing time is shortened and the memory is reduced. Reduction in usage is realized.

  Note that the parent-child relationship of the virtual bodies to be arranged does not have to be directly linked like the elements E4 to E6. For example, an indirect parent-child relationship (inclusion relationship) through a non-virtual element such as the element E10, such as the relationship between the elements E8 and E11, is also included. Further, even if the virtual bodies are associated with the same entity, such as the relationship between the elements E4 and E8, the elements that are not in the parent-child relationship are not targeted for organization.

  The above is the description of the embodiment of the present invention. The embodiment described above is merely an example of the embodiment of the present invention. That is, the present invention is not limited to the specific modes of the above-described embodiments, and various modifications can be made within the scope of the technical idea of the present invention. For example, the above embodiment is an example of viewing data broadcasting content described in BML. However, the content in which the effect of the present invention is exhibited is not limited to that described in BML, and HTML, Compact HTML, MML. The present invention is effective in browsing contents described in various markup languages such as (Mobile Markup Language), HDML (Handheld Device Markup Language), WML (Wireless Markup Language), XHTML, XML, and SGML (Standard Generalized Markup Language). It is. The type of content is not limited to data broadcast content, and the present invention is effective for Web content, electronic manuals, electronic dictionaries, and other various digital content. In the above embodiment, the event handler is set at the stage of generating the element tree. However, the stage of setting the event handler is not limited to the element tree generating step. For example, an event handler may be set in a markup document (or script attached to the markup document) such as a BML document or an HTML document, or other intermediate data such as a document tree, element tree, layout tree, etc. is generated. In this case, the event handler may be set in a format for processing the intermediate data once generated. Also, as in the above embodiment, instead of setting an event handler directly for intermediate data of content such as a document tree or an element tree (that is, changing content data itself), as an internal process of the browser, It is also possible to add an event handler for an arbitrary element.

  Also, as shown in the flowchart of FIG. 6, in the above embodiment, when viewing the entity / virtual body content, the first time after performing a series of processes such as the virtual body detection process ST5 and event handler setting ST6. The content is displayed. However, processes ST5 and ST6 place a certain load on the terminal. For this reason, if such processing is performed on a terminal with low processing capacity, the waiting time from the instruction by the user to the content display becomes long, so the user feels stress or misidentifies that it is not operating normally There is a problem that the processing is further delayed by repeating the instruction by the user input. Therefore, when the present invention is implemented in a terminal with low processing capability, first, the same processing as that of the conventional browser (that is, the processing of ST1, ST2, and ST7 to ST10) is first performed to display the screen once and at the same time the user It may be configured such that the acceptance of input is stopped, a series of processing shown in FIG. 6 is performed again during that time, and the processed content is displayed again, and at the same time, acceptance of user input is resumed. By performing such a process, it is possible to reduce the waiting time until the screen is displayed in a short time. Therefore, while suppressing stress felt by the user and unnecessary user input, the real / virtual content by the pointing device is suppressed. Intuitive user input is possible.

It is the figure which showed the external appearance of the terminal device by embodiment of this invention. It is the block diagram which showed schematic structure of the terminal device by embodiment of this invention. It is a functional block diagram of a browser engine included in a browser according to an embodiment of the present invention. It is a figure which shows the example of a display of the data broadcasting content used for description of embodiment of this invention. It is a figure which shows the example of a display of the data broadcasting content used for description of embodiment of this invention. It is a flowchart explaining the content optimization process by embodiment of this invention. It is a flowchart which shows the detail of process ST5 in the flowchart of FIG. It is a figure which shows the outline of the example of a document tree for demonstrating rearrangement of parent-child relationship. It is a figure which shows the example of a display of the content expressed with the document tree of FIG.

Explanation of symbols

10 terminal device 21 markup document 23 document tree 24 document tree for virtual object detection 25 element tree 27 layout tree 30 browser engine 31 parser 32 page maker 33 page editor 34 virtual object processing function unit 103 CPU
105 ROM
107 RAM
108 wireless communication unit 109 touch panel sensor 110 display 111 speaker 112 microphone 115 VRAM
116 Power Supply Unit 117 Power On / Off Key 1050 OS
1051 Browser 1071 Image buffer

Claims (23)

In an information processing apparatus capable of accepting an operation by a pointing device, a method for performing processing for interactively displaying content on a display based on a document described in a markup language,
Parsing the document to generate a first document tree;
Detecting, from the first document tree, an element set with a predetermined type of event handler that can be an entity that changes the display of a virtual body that is another element when focused;
Generating a second document tree based on the document;
Executing each of the predetermined types of event handlers set for each of the detected entity entities for the second document tree;
When the display of another element is changed by execution of the predetermined type of event handler, the element for which the event handler is set is determined to be an entity, and the element whose display has been changed is a virtual body associated with the entity A step of determining
A content information processing method including a step of performing processing for focusing the entity when a virtual object associated with the entity receives a predetermined pointer operation. The content information processing method according to claim 1, wherein the second document tree is generated by copying the first document tree. The step of performing the process for focusing the entity includes the step of setting an event handler in the virtual body that executes a method for focusing the entity when the pointer is positioned on the virtual body associated with the entity. The content information processing method according to claim 1 or 2, further comprising: 4. The step of performing processing for focusing the entity includes an onmouseover event handler set in the virtual body, and a method for focusing the entity includes a focus method. Content information processing method. 5. The content information processing method according to claim 3, wherein the step of performing the process for focusing the entity includes a step of generating intermediate data including an event handler set in the step. The content information according to any one of claims 1 to 5, further comprising a step of performing processing for clicking the entity when a virtual object associated with the entity receives a predetermined pointer operation. Processing method. The step of performing the process for clicking the entity includes the step of setting an event handler in the virtual body that executes a method of clicking the entity when the virtual body associated with the entity is clicked. The content information processing method according to claim 6, wherein: 8. The step of performing a process for clicking on the entity, the event handler set in the virtual body includes an onclick event handler, and the method for clicking on the entity includes a click method. Content information processing method. The method further comprises a step of determining whether or not each of the plurality of virtual bodies is in an inclusive relationship with each other when there are a plurality of the virtual bodies associated with one entity. 8. The content information processing method according to 8. When a plurality of virtual objects associated with the one entity have an inclusion relationship, the virtual object included in another virtual object is focused on when the virtual object receives a predetermined pointer operation. The content information processing method according to claim 9, wherein the processing for performing the processing is not performed.   The step of generating the second document tree is executed when an element in which the predetermined type of event handler is set is detected from the first document tree. The content information processing method according to any one of the above.   The computer program for making a computer perform each step of the method in any one of Claim 1 to 11. An information processing apparatus that performs processing to interactively display content on a display based on a document described in a markup language,
Pointing device input receiving means for receiving an input operation by a pointing device;
Means for obtaining the document;
First document tree generation means for parsing the document to generate a first document tree;
Entity candidate detection means for detecting, from the first document tree, an element set with a predetermined type of event handler that can be an entity that changes the display of a virtual body that is another element when focused;
Second document tree generation means for generating a second document tree based on the document;
Event handler execution means for executing each of the predetermined types of event handlers set for each of the elements that can be detected entities with respect to the second document tree;
When the display of another element is changed by execution of the predetermined type of event handler, the element for which the event handler is set is determined to be an entity, and the element whose display has been changed is associated with the entity Entity / virtual body relationship determination means for determining
A content information processing apparatus comprising: a focus processing unit that performs a process for focusing an entity when a virtual body associated with the entity receives a predetermined pointer operation. The content information processing apparatus according to claim 13, wherein the second document tree generation unit generates the second document tree by duplicating the first document tree. The focus processing means includes first event handler setting means for setting, in a virtual body, an event handler that executes a method for focusing the entity when the pointer is positioned on the virtual body associated with the entity. The content information processing apparatus according to claim 13 or 14, characterized in that: 16. The content information processing apparatus according to claim 15, wherein the focus processing means includes intermediate data generation means for generating intermediate data including an event handler set by the first event handler setting means. 17. The content information processing apparatus according to claim 15, wherein the event handler set by the first event handler setting means includes an onmouseover event handler, and the method for focusing the entity includes a focus method. 18. The click processing unit according to claim 13, further comprising click processing means for performing processing for clicking the entity when a virtual body associated with the entity receives a predetermined pointer operation. Content information processing device. The click processing means includes second event handler setting means for setting, in a virtual body, an event handler that executes a method for clicking the entity when a virtual body associated with the entity is clicked. The content information processing apparatus according to claim 18. 20. The content information processing apparatus according to claim 19, wherein the event handler set by the second event handler setting means includes an onclick event handler, and the method for clicking the entity includes a click method. The apparatus further comprises means for determining whether or not each of the plurality of virtual bodies is in an inclusive relationship with each other when there are a plurality of the virtual bodies associated with one entity. The content information processing apparatus according to any one of 20. The focus processing means, when there is an inclusion relationship among a plurality of virtual bodies associated with the one entity, with respect to a virtual body included in another virtual body, the virtual body receives a predetermined pointer operation. 22. The content information processing apparatus according to claim 21, wherein a process for focusing the entity is not performed when the content is processed. 23. The method according to claim 13, wherein the second document tree is executed when an element in which the predetermined type of event handler is set is detected from the first document tree. The content information processing apparatus described.

Images