WO2014061096A1

WO2014061096A1 - Information display device and information display method

Info

Publication number: WO2014061096A1
Application number: PCT/JP2012/076678
Authority: WO
Inventors: 英一有田; 下谷　光生
Original assignee: 三菱電機株式会社
Priority date: 2012-10-16
Filing date: 2012-10-16
Publication date: 2014-04-24
Also published as: JP6000367B2; JPWO2014061096A1

Abstract

This information display device includes a display unit which has a display surface, an input unit which receives user operations, and a control unit. If, as a user operation, a window opening operation instructing a window to be formed on the display surface and indicating the formation range of the window is performed on first display information on the display surface, the control unit forms a window corresponding to the formation range indicated by the window opening operation. The control unit displays, in the window, second display information which is related to the first display information but which has content or a display format different from that of the first display information.

Description

Information display device and information display method

The present invention relates to an information display device and an information display method.

The following Patent Document 1 discloses a system that links an in-vehicle device and a portable device. Specifically, when the navigation application is executed on the mobile device, the mobile device detects the current position and acquires map data around the current position. Then, the mobile device processes the acquired map data according to the display device of the in-vehicle device, and sends the processed data to the in-vehicle device. Thereby, map data around the host vehicle is displayed on the display device of the in-vehicle device. Similarly, if the route to the destination is calculated by the navigation function of the portable device, the map data on which the route to the destination is superimposed is displayed on the display device of the in-vehicle device.

In the system of Patent Document 1, the portable device employs a capacitive touch panel display capable of multipoint detection, whereas the in-vehicle device is basically a resistive film that is difficult to detect multipoint. The touch panel display of the system is adopted. In this case, the mobile device can accept a flick operation, an operation of tapping with two fingers, an operation of pinching with two fingers, and an operation of spreading two fingers, but the in-vehicle device accepts these operations. It is impossible. Patent Document 1 discloses a technique for enabling an in-vehicle device to handle an input operation that is accepted by a mobile device but not accepted by the in-vehicle device.

The following Patent Document 2 discloses a map information display device for moving objects. In this apparatus, a touch panel is attached to the display means. When a base screen on which a map centered on the vehicle position is displayed is touched, a window screen is displayed on the base screen. A map centered on the vehicle position is also displayed on the window screen. The map scale of the base screen is 1 / 1.25 million, whereas the map scale of the window screen is 1 / 25,000.

When a part of the base screen is touched in a two-screen display with both the base screen and the window screen displayed, the window screen is erased and a single scale of 1 / 1.25 million centered on the vehicle position. A map is displayed. Further, when a part of the window screen is touched in the two-screen display state, the window screen is deleted, and a single map of 1 / 255,000 scale centered on the vehicle position is displayed on the base screen.

JP 2012-8968 A JP-A-8-201071

In the system of Patent Document 1, the navigation application is executed by the mobile device, and the map data generated as a result is output to the display device of the in-vehicle device. At this time, only a single map is displayed on the in-vehicle device. On the other hand, in the apparatus of Patent Document 2, maps with different scales can be displayed on the base screen and the window screen.

However, in the device of Patent Document 2, the display contents themselves are basically the same except that the scale of the map is different between the base screen and the window screen. For this reason, the display content may not meet the user's request. Further, the position and size of the window screen are predetermined and fixed. For this reason, the position and size of the window screen may not match the user's preference.

An object of the present invention is to provide a technique that allows a user to manipulate the position, size, content, and the like of display information according to preferences.

An information display device according to an embodiment of the present invention includes a display unit having a display surface, an input unit that receives a user operation, and a control unit. When a window opening operation for instructing to form a window on the display surface and specifying a window forming range is performed on the first display information on the display surface as a user operation, the control unit opens the window. A window is formed according to the formation range designated by the operation. The control unit causes the window to display second display information related to the first display information but having a content or expression format different from the first display information.

According to the above embodiment, the position, size, content, etc. of the display information can be adjusted to the user's preference by adopting the window opening operation.

Also, since the window is formed at the execution position of the window opening operation, the information in the window can be viewed without moving the viewpoint greatly. For this reason, a user's cognitive load may be small.

Further, the second display information displayed in the window is information related to the first display information, but differs in content or expression format from the first display information. For this reason, the efficiency of information recognition can be improved by displaying various information in one screen. In addition, the reduction of the cognitive load produced by the window opening operation also contributes to the efficiency of information recognition.

The objects, features and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

It is a block diagram which illustrates an information display device. It is a perspective view which illustrates an input and display part. It is a conceptual diagram of 1-point touch operation. It is a conceptual diagram of a two-point touch operation. It is a conceptual diagram of drag operation. It is a conceptual diagram of flick operation. It is a conceptual diagram of pinch out operation (two-point movement type). It is a conceptual diagram of pinch out operation (one point movement type). It is a conceptual diagram of pinch-in operation (two-point movement type). It is a conceptual diagram of pinch-in operation (one-point movement type). It is a conceptual diagram of slide operation. It is a conceptual diagram of display size change operation (enlargement operation and reduction operation). It is a conceptual diagram of rotation operation. It is a block diagram which illustrates a control part. It is a flowchart which illustrates the process until a window display. It is a conceptual diagram of window opening operation (two-point touch long press). It is a conceptual diagram of window opening operation (one-point touch and surrounding drag). It is a conceptual diagram of window opening operation (use of an icon). It is a figure which illustrates the shadow modification of a window. It is a figure which illustrates the hollow modification of a window. It is a figure which illustrates a circular window. It is a figure explaining the 1st example of image composition control regarding window formation. It is a figure explaining the 2nd example of image composition control regarding window formation. It is a figure explaining the 1st example of the display information of a window. It is a figure explaining the 2nd example of the display information of a window. It is a figure explaining the 3rd example of the display information of a window. It is a figure explaining the 4th example of the display information of a window. It is a flowchart which illustrates the process after a window display. It is a figure explaining the 1st example of the display control when a user operation is performed outside the window (linkage of display). It is a figure explaining the 2nd example of display control when a user operation is performed outside a window (linkage of a display). It is a figure explaining the 3rd example of the display control when a user operation is performed outside the window (non-linkage of display). It is a conceptual diagram of information switching operation in a window. It is a conceptual diagram explaining the information switching operation in the window with respect to the hierarchical information. It is a conceptual diagram of window movement operation. It is a conceptual diagram of window size change operation. It is a conceptual diagram of window size change operation. It is a conceptual diagram of window size change operation. It is a conceptual diagram of the 1st example of window deletion operation. It is a conceptual diagram of the 2nd example of window deletion operation. It is a conceptual diagram of the 3rd example of window deletion operation. It is a figure explaining the 1st example of the window shape by bird's-eye view expression. It is a figure explaining the 2nd example of the window shape by bird's-eye view expression. It is a figure explaining the 3rd example of the window shape by bird's-eye view expression. It is a figure explaining the 4th example of the window shape by bird's-eye view expression. It is a figure explaining the window shape according to the division on a map.

<Overview of overall configuration>
FIG. 1 illustrates a block diagram of an information display device 10 according to an embodiment. According to the example of FIG. 1, the information display device 10 includes a display unit 12, an input unit 14, a control unit 16, and a storage unit 18.

Display unit 12 displays various information. The display unit 12 drives each pixel based on, for example, a display surface configured by arranging a plurality of pixels in a matrix and image data acquired from the control unit 16 (in other words, each pixel's Driving device for controlling the display state). The image displayed on the display unit 12 may be a still image, a moving image, or a combination of a still image and a moving image.

The display unit 12 can be configured by a liquid crystal display device, for example. According to this example, a display area of a display panel (here, a liquid crystal panel) corresponds to the display surface, and a drive circuit externally attached to the display panel corresponds to the drive device. Note that a part of the driver circuit may be incorporated in the display panel. In addition to the liquid crystal display device, the display unit 12 can be configured by an electroluminescence (EL) display device, a plasma display device, or the like.

The input unit 14 receives various information from the user. The input unit 14 includes, for example, a detection unit that detects an indicator used by the user for input, and a detection signal output unit that outputs a result detected by the detection unit to the control unit 16 as a detection signal. .

Here, a case where the input unit 14 is configured by a so-called contact-type touch panel is illustrated, and the input unit 14 may be referred to as a “touch panel 14” below. The touch panel may be referred to as a “touch pad” or the like. Moreover, the case where the said instruction | indication used for an input is a user's finger | toe (specifically fingertip) is illustrated.

The detection unit of the touch panel 14 provides an input surface on which a user places a fingertip, and detects the presence of a finger on the input surface by a sensor group provided for the input surface. In other words, an area where a finger can be detected by the sensor group corresponds to an input area where a user input can be received. In the case of a contact-type touch panel, the input area corresponds to an input surface of a two-dimensional area.

The sensor group may be any of electrical, optical, mechanical, etc., or a combination thereof. Various position detection methods have been developed, and any of them may be adopted for the touch panel 14. In addition to detecting the position of the finger, a configuration capable of detecting the pressing force of the finger on the input surface may be employed.

• The position of the fingertip on the input surface can be specified from the combination of the output signals of each sensor. The identified position is expressed by coordinate data on coordinates set on the input surface, for example. In this case, when the finger is moved on the input surface, the coordinate data indicating the finger position changes, so that the movement of the finger can be detected by a series of coordinate data acquired continuously.

Note that the finger position may be expressed by a method other than coordinates. That is, the coordinate data is an example of finger position data for expressing the position of the finger.

Here, an example is given in which the detection signal output unit of the touch panel 14 generates coordinate data indicating the finger position from the output signals of the sensors, and transmits the coordinate data to the control unit 16 as a detection signal. However, for example, the conversion to coordinate data may be performed by the control unit 16. In such an example, the detection signal output unit converts the output signal of each sensor into a signal in a format that can be acquired by the control unit 16, and transmits the obtained signal to the control unit 16 as a detection signal.

Further, as shown in the perspective view of FIG. 2, the input surface 34 of the touch panel 14 (see FIG. 1) and the display surface 32 of the display unit 12 (see FIG. 1) are overlapped, in other words, the input surface 34. A structure in which the display surface 32 is integrated is illustrated. With such an integrated structure, the input / display unit 20 (see FIG. 1), more specifically, the touch screen 20 is provided.

According to the integrated structure of the input surface 34 and the display surface 32, the input surface 34 and the display surface 32 are identified to the user, giving the user the feeling of performing an input operation on the display surface 32. For this reason, an intuitive operation environment is provided. In view of this point, for example, the expression “the user operates the display surface 32” may be used.

The control unit 16 performs various processes and controls in the information display device 10. For example, the control unit 16 analyzes information input from the touch panel 14, generates image data according to the analysis result, and outputs the image data to the display unit 12.

Here, the control unit 16 includes a central processing unit (for example, configured with one or a plurality of microprocessors) and a main storage unit (for example, one or a plurality of storage devices such as ROM, RAM, flash memory, etc.). ). According to this example, various functions are realized by the central processing unit executing various programs stored in the main storage unit (in other words, by software). Various functions can be realized in parallel.

Various programs may be stored in the main storage unit of the control unit 16 in advance, or may be read from the storage unit 18 during execution and stored in the main storage unit. The main storage unit is used not only for storing programs but also for storing various data. In addition, the main storage unit provides a work area when the central processing unit executes the program. The main storage unit provides an image holding unit for writing an image to be displayed on the display unit 12. The image holding unit may be referred to as “video memory”, “graphic memory”, or the like.

Note that all or part of the processing and control performed by the control unit 16 may be configured as hardware (for example, an arithmetic circuit configured to perform a specific calculation).

The storage unit 18 stores various information. Here, the storage unit 18 is provided as an auxiliary storage unit used by the control unit 16. The storage unit 18 can be configured using one or more storage devices such as a hard disk device, an optical disk, a rewritable and nonvolatile semiconductor memory, and the like.

Further, the information display device 10 may further include elements other than the

above elements

12, 14, 16, and 18. For example, an audio output unit that outputs auditory information, a communication unit that performs wired or wireless communication with various devices, and the current position of the information display device 10 conform to, for example, a GPS (Global Positioning System) system. One or more of the current position detection units to be detected may be added.

The voice output unit can output, for example, operation sounds, sound effects, guidance sounds, and the like. The communication unit can be used for, for example, new acquisition and update of information stored in the storage unit 18. Further, the current position detection unit can be used for a navigation function, for example.

The use of the information display device 10 is not particularly limited. For example, the information display device 10 may be a portable or desktop information device. Alternatively, the information display device 10 may be applied to a navigation device or an audio / visual device mounted on a moving body such as an automobile.

<User operations and their functions>
Before describing a more specific configuration and processing of the information display device 10, user operations on the touch panel 14 will be described.

User operations are roughly classified into touch operations and gesture operations based on finger movements. Hereinafter, the touch operation and the gesture operation may be referred to as “touch” and “gesture”, respectively. The touch operation is an operation in which at least one fingertip is brought into contact with the input surface of the touch panel and the contacted finger is moved away from the input surface without being moved on the input surface. On the other hand, the gesture operation is an operation in which at least one fingertip is brought into contact with the input surface, and the contacted finger is moved on the input surface (in other words, slid), and then released from the input surface. .

The coordinate data detected by the touch operation (in other words, finger position data) is basically static and static. On the other hand, the coordinate data detected by the gesture operation changes with time and is dynamic. According to such a series of changing coordinate data, the point where the finger starts moving and the point on the input surface, the locus from the moving start point to the moving end point, the moving direction, the moving amount, the moving speed, the moving acceleration, Etc. can be acquired.

FIG. 3 is a conceptual diagram illustrating a one-point touch operation (also simply referred to as “one-point touch”) as a first example of the touch operation. In FIG. 3 and FIGS. 4 to 10 described later, a top view of the input surface 34 is shown in the upper stage, and a side view or a sectional view of the input surface 34 is shown in the lower stage.

As shown in FIG. 3, in the one-point touch, the user makes a point contact with one finger on the input surface 34. In FIG. 3, touch points (in other words, finger detection points) are schematically indicated by black circles. Such an illustration technique is also used in the drawings described later. A black circle may be actually displayed on the display surface.

１ One-point touch can be classified into single tap, multi-tap and long press operations, for example. Single tap is an operation of tapping the input surface 34 once with a fingertip. A single tap is sometimes simply referred to as a “tap”. Multi-tap is an operation of repeating a tap a plurality of times. A double tap is a typical multi-tap. The long press is an operation for maintaining the point contact of the fingertip. These operations can be identified by, for example, the duration and number of times of finger contact (in other words, finger detection).

FIG. 4 is a conceptual diagram illustrating a two-point touch operation (also simply referred to as “two-point touch”) as a second example of the touch operation. The two-point touch is basically the same as the one-point touch except that two fingers are used. For this reason, it is possible to perform each operation of tap, multi-tap, and long press, for example, by two-point touch. In the two-point touch, two fingers of one hand may be used, or one finger of the right hand and one finger of the left hand may be used. The positional relationship between the two fingers is not limited to the example in FIG.

It is also possible to perform touch operations with three or more fingers.

FIG. 5 is a conceptual diagram illustrating a drag operation (also simply referred to as “drag”) as a first example of the gesture operation. Dragging is an operation of shifting the fingertip while it is placed on the input surface 34. The moving direction and moving distance of the finger are not limited to the example in FIG.

In FIG. 5, the movement start point of the finger is schematically shown by a black circle, the movement end point of the finger is schematically shown by a black triangle, the direction of movement of the finger is expressed by the direction of the triangle, and the black circle The trajectory is represented by a line connecting the triangle and the black triangle. Such an illustration technique is also used in the drawings described later. Note that the black circle, the black triangle, and the locus may be actually displayed on the display surface.

FIG. 6 is a conceptual diagram illustrating a flick operation (also simply referred to as “flick”) as a second example of the gesture operation. The flick is an operation for quickly paying the fingertip on the input surface 34. The moving direction and moving distance of the finger are not limited to the example in FIG.

In flicking, unlike dragging, the finger leaves the input surface 34 while moving. Here, since the touch panel 14 is a contact type, the finger movement after leaving the input surface 34 is not detected in principle. However, for example, it is possible to calculate the moving speed of the finger at the final point detected from a series of changes in coordinate data obtained while the finger moves on the input surface 34. A flick can be identified when the moving speed is equal to or higher than a predetermined threshold (referred to as “drag / flick identification threshold”).

Further, for example, from the detected movement direction, movement speed, and movement acceleration of the finger at the final point, the point where the finger finally arrives after it leaves the input surface 34 (more specifically, the point is defined as the input surface 34). Can be estimated. Note that this estimation process can be interpreted as a process of converting a flick into a virtual drag.

Therefore, the information display apparatus 10 treats the estimated point as the end point of finger movement. In this example, the estimation process may be executed by the touch panel 14 or may be executed by the control unit 16.

However, such an estimation is not performed, and the information display device 10 may be modified so that a point away from the input surface 34 is handled as an end point of finger movement.

FIG. 7 is a conceptual diagram illustrating a pinch out operation (also simply referred to as “pinch out”) as a third example of the gesture operation. Pinch out is an operation of moving two fingertips away on the input surface 34. Pinch out is also called “pinch open”.

FIG. 7 illustrates the case where both two fingers are dragged. On the other hand, as shown as a fourth example of the gesture operation in FIG. 8, one fingertip is fixed on the input surface 34 (in other words, one fingertip maintains a touch state), and only the other fingertip is held. It is also possible to pinch out by dragging. 7 and 8 are referred to as “two-point movement type”, and the method in FIG. 8 is referred to as “one-point movement type”.

FIG. 9 is a conceptual diagram illustrating a pinch-in operation (also simply referred to as “pinch-in”) as a fifth example of the gesture operation. Pinch-in is an operation of bringing two fingertips closer on the input surface 34. Pinch-in is also referred to as “pinch close”. FIG. 9 illustrates a two-point movement type pinch-in, but FIG. 10 illustrates a one-point movement type pinch-in as a sixth example of the gesture operation.

Here, pinch-out and pinch-in are collectively referred to as “pinch operation” or “pinch”, and the direction of finger movement is referred to as “pinch direction”. In this case, when the pinch direction is a direction in which the interval between fingers is increased, the pinch operation is particularly referred to as pinch out. Conversely, when the pinch direction is a direction in which the interval between fingers is reduced, the pinch operation is particularly called pinch-in.

In pinch-out and pinch-in, two fingers of one hand may be used, or one finger of the right hand and one finger of the left hand may be used. Further, the positional relationship, the moving direction, and the moving distance of the two fingers are not limited to the examples in FIGS. Further, in the one-point movement type pinch-out and pinch-in, the finger to be dragged is not limited to the examples of FIGS. 8 and 10. It is also possible to pinch out and pinch in using flicks instead of dragging.

Each user operation is associated with a specific function. Specifically, when a user operation is detected, a process associated with the user operation is executed by the control unit 16, thereby realizing a corresponding function. In view of this point, user operations can be classified based on functions to be realized.

For example, a double tap performed on an icon on the display surface 32 is associated with a function for executing a program or command associated with the icon. In this case, the double tap functions as an execution instruction operation.

Further, as illustrated in FIG. 11, dragging performed on display information (a map image is illustrated in FIG. 11) is associated with a slide function for sliding the display information. In this case, the drag operation functions as a slide operation. Note that it is also possible to slide by flicking instead of dragging. Here, the slide function and the slide operation are also referred to as a scroll function and a scroll operation. However, the slide direction and the scroll direction differ by 180 °.

Further, as illustrated in FIG. 12, the pinch out and the pinch in performed on the display information (a map image is illustrated in FIG. 12) changes the size (in other words, the scale) of the information display. Associated with function. In this case, pinch-out and pinch-in function as a display size change operation (may be referred to as a “display scale change operation”). More specifically, in the example of FIG. 12, pinch out corresponds to an enlargement operation, and pinch in corresponds to a reduction operation.

In addition, as illustrated in FIG. 13, when two fingers are dragged with respect to the display information (a map image is illustrated in FIG. 13) to draw a circle while maintaining a distance, The drag is associated with a function that rotates the information display. In this case, the two-point movement type rotary drag functions as a rotation operation. In addition, you may employ | adopt the rotation drag performed with three or more fingers. Further, the function of associating may be changed according to the number of fingers to be rotated and dragged.

Here, it is also possible to assign multiple types of functions to one user operation. For example, a double tap may be assigned to a folder opening operation for opening a folder associated with an icon in addition to the above execution instruction operation. Further, the drag may be assigned to a slide function and a drawing function. When a plurality of types of functions are assigned to one user operation, each function is switched according to the operation target, usage status (in other words, usage mode), and the like.

It is also possible to assign multiple types of user operations to one function. For example, the execution instruction function for the icon may be associated with double tap, long press, and flick. In this case, a program or the like associated with the icon can be executed by any of double tap, long press and flick. For example, the slide function may be associated with both dragging and flicking. Further, for example, the rotation function may be associated with both the two-point movement type rotation drag and the one-point movement type rotation drag.

Here, the functions associated with user operations are roughly classified into screen movement deformation type and non-movement deformation type from the viewpoint of screen movement and deformation. In the following, for example, a gesture operation associated with a screen movement deformation type function may be expressed as “a screen movement deformation type function gesture operation”.

The screen movement deformation type function associated with the gesture operation is a function of controlling (in other words, manipulating) display information on the display surface in a control direction set according to the gesture direction. The screen movement deformation type function includes, for example, a slide function, a display size change function, a rotation function, and a bird's eye view display function (more specifically, an elevation angle and depression angle change function). The slide function can be classified as a screen movement function. Further, if the rotation function is viewed from the viewpoint of angle movement, the rotation function can be classified as a screen movement function. Further, the display size changing function and the bird's eye view display function can be classified into screen deformation functions.

More specifically, in the slide function, a slide direction (that is, a control direction) is set according to a gesture direction (for example, a drag direction or a flick direction), and display information is slid in the slide direction.

In the display size changing function, when the gesture direction (for example, pinch direction) is the enlargement direction, the control direction is set to the enlargement direction, and when the gesture direction is the reduction direction, the control direction is set to the reduction direction. Then, the display information size is changed in the set control direction.

In the rotation function, the control direction is set to the right rotation direction when the gesture direction (for example, the rotation direction in the rotation drag) is the right rotation direction, and the control direction is set to the left rotation direction when the gesture direction is the left rotation direction. The counter is set to the left rotation direction, and the display information is rotated in the set control direction.

Note that the screen movement / deformation function may control the display information by using not only the gesture direction but also the gesture amount (for example, the length of the gesture trajectory). Specifically, the control amount (for example, the slide amount, the display size change amount, and the rotation amount) of the display information may be set larger as the gesture amount is larger.

Further, the screen movement deformation type function may control the display information using the gesture speed in addition to or instead of the gesture amount. Specifically, the display information control speed (for example, slide speed, display size change speed, and rotation speed) may be set higher as the gesture speed is higher.

On the other hand, the non-moving deformation type function does not use the gesture direction for realizing the function even if it is associated with the gesture operation. For example, even if a flick to an icon is associated with an execution instruction function of a specific program, the function belongs to the non-moving deformation type. For example, when dragging is used for the drawing function and the handwritten character input function, only the trajectory corresponding to the dragging is displayed, and the display information is not controlled according to the dragging direction.

Note that the user operations and the functions realized thereby are not limited to the above examples.

<Configuration Example of Control Unit 16>
FIG. 14 illustrates a block diagram of the control unit 16. In FIG. 14, the display unit 12, the input unit 14, and the storage unit 18 are also illustrated for explanation. According to the example of FIG. 14, the control unit 16 includes an input analysis unit 40, an overall control unit 42, a first image forming unit 44, a first image holding unit 46, a second image forming unit 48, A two-image holding unit 50, an image composition unit 52, a composite image holding unit 54, and a window management unit 56 are included.

The input analysis unit 40 analyzes the user operation detected by the input unit 14 and identifies the user operation. Specifically, the input analysis unit 40 acquires coordinate data detected along with a user operation from the input unit 14, and acquires user operation information from the coordinate data. The user operation information is, for example, information such as the type of user operation, the start and end points of finger movement, the trajectory from the start point to the end point, the moving direction, the moving amount, the moving speed, and the moving acceleration.

For identifying the type of user operation, for example, the difference between the start point and the end point can be distinguished from a predetermined threshold value (referred to as a “touch / gesture identification threshold value”) to identify the touch operation and the gesture operation. It is. Further, as described above, the drag and the flick can be identified from the finger moving speed at the end of the trajectory.

Also, for example, when two drags are identified at the same time, pinch out and pinch in can be identified from the moving direction. In addition, when the two drags draw a circle while maintaining a distance, it is possible to identify that the rotation drag has been performed. When a drag and a one-point touch are identified at the same time, it can be identified that the pinch-out, the pinch-in, and the rotational drag are one-point moving types.

The overall control unit 42 performs various processes in the control unit 16. For example, the overall control unit 42 associates a position on the input surface of the input unit 14 with a position on the display surface of the display unit 12. According to this, the touch position in the touch operation, the gesture locus in the gesture operation, and the like are associated on the display surface. With such association, it is possible to identify the position on the display surface where the user operation is intended. Such association can be realized by a so-called graphical user interface (GUI) technique.

Further, the overall control unit 42 identifies a function desired by the user, that is, a user instruction based on, for example, user operation information and function identification information. The function identification information is, for example, information in which an association between a user operation and a function to be executed is defined through the operation status information. The operation status information includes, for example, the usage status of the information display device 10 (in other words, the usage mode), the operation target on which the user operation has been performed, the type of user operation that can be accepted according to the usage status and the operation target, etc. Information.

More specifically, for example, when a drag is performed on the map image as an operation target in a situation where the map browsing software is used, the drag is identified as instructing execution of the slide function. Also, for example, when a tap is performed with an enlarged icon on the map image as an operation target, the tap is identified as instructing execution of the display size enlargement function. For example, if no function is associated with the flick for the enlarged icon, it is determined that the flick is an invalid operation.

The overall control unit 42 controls the display information on the display surface by controlling the first image forming unit 44, the second image forming unit 48, and the image composition unit 52. Note that the display information may be changed based on the identification result of the user instruction, or may be based on an instruction on program execution irrespective of the identification result of the user instruction.

The overall control unit 42 performs general control on the other

functional units

40, 44, 46, 48, 50, 52, 54, and 56, for example, adjustment of execution timing.

The first image forming unit 44 reads the first information 60 according to the instruction from the overall control unit 42 from the storage unit 18, forms the first image from the first information 60, and converts the first image into the first image holding unit 46. To store. Similarly, the second image forming unit 48 reads the second information 62 according to the instruction from the overall control unit 42 from the storage unit 18, forms a second image from the second information 62, and converts the second image into the second image. Store in the holding unit 50.

Under the instruction of the overall control unit 42, the image composition unit 52 reads the first image from the first image holding unit 46, reads the second image from the second image holding unit 50, and combines the first image and the second image. The synthesized image is stored in the synthesized image holding unit 54.

The image composition is performed so that the first image and the second image are displayed in an overlapping manner. Here, the case where the first image is the lower image (in other words, the lower layer) and the second image is the upper image (in other words, the upper layer) is illustrated. Here, “up and down” means up and down in the normal direction of the display surface, and the side closer to the user viewing the display surface is expressed as “up”. In practice, image data is superimposed based on such a concept.

In the composite image, that is, the display screen, the lower image is displayed in the transparent portion of the upper image. In other words, the drawing portion of the upper image hides the lower image. However, by setting the transparency in the drawing portion of the upper image, a composite image in which the lower image is transparent can be formed.

The setting of which of the first image and the second image is the upper image may be unchangeable or may be changeable.

Here, an example is given in which two layers of the first image and the second image are combined, but a configuration capable of combining more layers may be adopted. Also, other synthesis methods may be adopted.

The composite image stored in the composite image holding unit 54 is transferred to the display unit 12 and displayed on the display unit 12. The display screen changes when the composite image is updated, that is, when at least one of the first image and the second image is updated.

The window management unit 56 manages windows formed on the display surface under the control of the overall control unit 42. Specifically, the window management unit 56 manages information such as the window formation range (position, shape, etc.), display attributes (whether or not the window is modified and the type, etc.), and based on the window management information. The window is managed by controlling the image composition unit 52.

<Processing Example of Information Display Device 10>
Hereinafter, a process (in other words, an information display method) related to the window by the information display device 10 will be exemplified.

<Process until window formation>
FIG. 15 illustrates a processing flow S10 up to window formation. According to the example of FIG. 15, in step S11, the input unit 14 receives a user operation, and in step S12, the control unit 16 identifies the input user operation. In step S13, the control unit 16 determines whether or not the input user operation is a window opening operation defined in advance based on the identification result in step S12.

If it is determined that the user operation is not a window opening operation, the control unit 16 executes a function associated with the input user operation in step S14. Thereafter, the processing of the information display device 10 returns to step S11.

On the other hand, when it is determined in step S13 that the user operation is a window opening operation, the control unit 16 forms a window and displays information in the window in step S15. The process flow S10 in FIG. 15 ends with the formation of the window.

The window opening operation is an operation for instructing to form a window on the display surface and an operation for designating a window forming range. Examples of window opening operations are shown in FIGS.

In the example of FIG. 16, the long press operation of two-point touch is assigned to the window opening operation. According to this example, the window 80 is formed in a rectangular range having the two touched points as the vertices of the diagonal position. That is, the user designates the formation range of the window 80 by two touch points. Note that the positional relationship between the two points to be touched is not limited to the example of FIG.

In FIG. 16, in order to make the window 80 easy to understand on the drawing, an area in the window 80 is sanded. That is, this sand hatching is only given for the description of the embodiment, and does not limit the design of the window 80 or the like. Such sand hatching may also be used in later-described drawings such as FIG.

In the case of the example of FIG. 16, when the control unit 16 identifies that the two-point touch state has reached a predetermined time (referred to as “window opening instruction time”) in step S12, In S13, it is determined that a window opening operation has been input. And the control part 16 forms the window 80 on the display surface 32 according to the range designated by window opening operation. For example, the control unit 16 associates the two touched points with the coordinate system of the display surface 32 and adopts the two points on the display surface 32 as the vertices of the diagonal position to form the rectangular window 80. The formation range of the window 80 may be obtained on the coordinate system of the input surface 34, and the obtained range may be associated with the coordinate system of the display surface 32.

In the example of FIG. 17, a one-point touch and a drag (referred to as “enclosed drag” or “enclosed gesture”) that surrounds an arbitrary range starting from the one-point touched point or the vicinity thereof. The combination operation is assigned to the window opening operation. As the one-point touch in this example, any of single tap, multi-tap and long press can be adopted. Also, the end of the drag may be a flick. The vicinity of the one-point touch point refers to a range of a predetermined distance from the one-point touch point, for example. According to this example, the user designates the formation range of the window 80 by the range surrounded by the surrounding drag or by the combination of the one-point touch point and the range surrounded by the surrounding drag. The direction of the surrounding drag is not limited to the example of FIG.

In the case of the example in FIG. 17, when the control unit 16 identifies in step S12 that the one-point touch and the surrounding drag have been continuously performed, it determines that the window opening operation is input in step S13. The condition “continuously” includes a condition that a one-point touch and a surrounding drag are performed within a predetermined operation time interval, and a condition that no other operation is performed in the middle.

And the control part 16 forms the window 80 on the display surface 32 according to the range designated by window opening operation. For example, the control unit 16 associates the encircling drag locus 70 with the coordinate system of the display surface 32, and defines a range surrounded by the locus 70 on the coordinate system of the display surface 32 according to a predetermined conversion rule. And a window 80 is formed in the converted rectangular range. In the example of FIG. 17, as the conversion rule, for example, it is possible to adopt a rule for obtaining the maximum rectangle contained in the range of the enclosed locus 70. However, other conversion rules may be adopted. For example, the window formation range may be determined so that the point of the one-point touch performed at the beginning of the window opening operation becomes one vertex of the window 80. The formation range of the window 80 may be obtained on the coordinate system of the input surface 34, and the obtained range may be associated with the coordinate system of the display surface 32.

In the example of FIG. 18, an icon 90 is used. For example, as shown in FIG. 18, a combination operation of a one-point touch on the icon 90 and a pinch out on the icon 90 is assigned to the window opening operation. As the one-point touch in this example, any of single tap, multi-tap and long press can be adopted. According to this example, the user designates the formation range of the window 80 by two end points of the pinch out. 18 illustrates a two-point movement type pinch-out, but a one-point movement type pinch-out may be employed. Further, the direction of pinch out is not limited to the example of FIG.

In the case of the example in FIG. 18, when the control unit 16 identifies that the one-point touch and the pinch-out for the icon 90 have been continuously performed in step S <b> 12, the control unit 16 determines that the window opening operation is input in step S <b> 13. The condition “continuously” is the same as in the example of FIG.

And the control part 16 forms the window 80 on the display surface 32 according to the range designated by window opening operation. Such window formation processing can be executed in the same manner as in the example of FIG. That is, processing can be performed using two end points of the pinch out instead of the two points of the touch operation in the example of FIG.

In the example of FIG. 18, information associated with the icon 90 in advance is displayed on the window 80. Further, although the icon 90 illustrated in FIG. 18 simulates the appearance of a building, for example, a logotype such as a company or a product may be adopted in the design of the icon 90. Further, for example, a map symbol may be adopted in the design of the icon 90.

In addition, for example, when the icon 90 exists within the range specified by the window opening operation illustrated in FIGS. 16 and 17, the window opening operation is performed, and information associated with the icon 90 is displayed on the window 80. You may make it identify as.

16 to FIG. 18, the window 80 can be formed at the execution position of the window opening operation. In the examples of FIGS. 16 and 17, the window opening operation itself is a natural operation imagining the window 80 to be formed. In the example of FIG. 18, the window opening operation is highly similar to the operation of opening an object and looking inside in daily life. From these points, the window 80 can be formed intuitively. For this reason, high operability can be realized.

Also, since the window 80 is formed at the execution position of the window opening operation, the information in the window 80 can be viewed without moving the viewpoint greatly. For this reason, a user's cognitive load may be small.

Note that the window opening operation is not limited to the examples shown in FIGS. Various user operations or combinations thereof can be pre-assigned as window opening operations.

16 to 18, the window 80 is drawn with a simple thick frame to avoid complication of the drawings. However, the design of the window 80 is not limited to this. For example, the shaded modification exemplified in FIG. 19 and the dimple modification exemplified in FIG. 20 may be employed. According to the shadow modification, it is possible to give an impression that the window portion is positioned above the surroundings. On the contrary, according to the depression modification, it is possible to give an impression that the window portion is positioned below the surroundings. Further, the window 80 is not limited to a quadrangle. For example, it may be circular as shown in FIG. Moreover, you may employ | adopt the shape imitating a searchlight.

The setting of the modification regarding the window 80 (the presence / absence of modification, the type of modification, the degree of modification, etc.) may be fixed to one type or may be selected according to the information displayed on the window 80. Alternatively, the user may be able to set and change. The same applies to the shape of the window 80.

The formation range of the window 80 set according to the window opening operation is managed by the window management unit 56 as described above. More specifically, in the example of FIG. 14, when the overall control unit 42 detects an input of a window opening operation, the overall control unit 42 includes window management information including the formation range and display attributes of the window 80 according to the window opening operation. And the determined information is recorded in the window management unit 56. Here, for the display attribute of the window 80, a setting value (for example, an initial setting value) effective at that time is applied. The overall control unit 42 may record only the formation range of the window 80 in the window management unit 56, and the window management unit 56 may add display attributes accordingly.

Then, the window management unit 56 controls the synthesis of the first image and the second image in the image synthesis unit 52 based on the window management information stored in itself. An example of the control will be described with reference to FIGS. 22 and 23, the lower layer is the first image, and the upper layer is the second image.

In the example of FIG. 22, under the control of the window management unit 56, the image composition unit 52 excludes the portion corresponding to the formation range of the window 80 in the second image stored in the second image holding unit 50. read out. On the other hand, the image composition unit 52 reads the first image stored in the first image holding unit 46, including the part corresponding to the formation range of the window 80. Then, the image composition unit 52 sets the read first image and second image as the lower layer and the upper layer, respectively, and synthesizes both images.

As a result, on the display surface 32, the first image of the lower layer is displayed in the window inner area 82, which is the area inside the window 80, and the upper layer first image is displayed in the window outer area 83, which is the area outside the window 80. Two images are displayed. Here, it is assumed that the transparency of the upper layer is set to 0%.

Alternatively, in the example of FIG. 22, the image composition unit 52 may read the second image constituting the upper layer, including the part corresponding to the formation range of the window 80. In this case, the image composition unit 52 sets the transparency of the portion corresponding to the formation range of the window 80 in the read second image to 100% under the control of the window management unit 56 and composes it with the first image. .

In the example of FIG. 23, under the control of the window management unit 56, the image composition unit 52 reads a portion corresponding to the formation range of the window 80 in the second image stored in the second image holding unit 50. On the other hand, the image composition unit 52 reads the first image stored in the first image holding unit 46, including the part corresponding to the formation range of the window 80. Then, the image composition unit 52 sets the read first image and second image as the lower layer and the upper layer, respectively, and synthesizes both images.

Thereby, on the display surface 32, the second image of the upper layer is displayed in the window inner area 82, and the first image of the lower layer is displayed in the window outer area 84. Here, it is assumed that the transparency of the upper layer is set to 0%.

Alternatively, in the example of FIG. 23, the image composition unit 52 may read out the second image constituting the upper layer, including the part other than the part corresponding to the formation range of the window 80. In this case, under the control of the window management unit 56, the image composition unit 52 sets the transparency to 100% except for the portion corresponding to the formation range of the window 80 in the read second image, and sets the first image. And synthesize.

Note that the image composition unit 52 modifies the window 80 as necessary, for example, after composition of the upper layer and the lower layer under the control of the window management unit 56.

<Display information>
In step S15 in FIG. 15, the window 80 is related to display information (hereinafter also referred to as “first display information”) that is the object of the window opening operation, but the content or expression format is the first display information. Displays different display information (sometimes referred to as “second display information”).

The first display information and the second display information are, for example, map information, and this example will be mainly described below. The map information is various information regarding the place.

The map information is, for example, geographic information. The geographical information includes, for example, terrain information related to so-called terrain (sea and land, mountain rivers, etc.), place state information related to the state of the place (for example, usage state), name information related to the element name of the terrain information and place state information, Etc.

Terrain information is shape information such as coastline and water system, and may include height information such as altitude.

The location status information is roughly classified into, for example, ground status information and underground status information.

For simplicity of explanation, the ground (so-called land surface) is assumed as the boundary surface between the ground and the basement, but the present invention is not limited to this example. For example, it may be practical or appropriate to use the terms “above” and “underground” for the earth surface (so-called ground surface) including the ocean surface. Alternatively, for example, when focusing on the sea bottom surface, it may be practical or appropriate to use the expressions “above” and “underground” with the sea bottom surface as a boundary surface.

Examples of ground state information include transportation networks (roads, railways, etc.), buildings, fields, forests, deserts, etc. The building state information may include state information of each floor (for example, provided as a floor guide). Moreover, regional boundaries such as administrative divisions are exemplified as the state information on the ground.

Examples of underground state information include underground malls, underground facilities (water supply, electricity, gas, communications, railways, etc.), ruins, etc. In the case of a building having an underground part, the floor state of the underground part may be classified into underground state information, or the state of the building is combined with the floor state information of the underground part and the ground part. It may be information.

As the name information, so-called place names such as mountains, rivers, seas, and addresses are exemplified. Examples of the name information include names of roads, railways, buildings, stores, facilities, and the like.

Terrain information is visualized mainly by graphics such as maps. Further, the place state information can be visualized by a diagram, a symbol, a character, etc., and is mainly displayed at a corresponding place on the map. The name information can be visualized by characters or the like, and is displayed at a corresponding place on the map, for example. Alternatively, the name information may be displayed in a list format or the like instead of being arranged on the map.

Also, the map information is, for example, weather information. The weather information is, for example, information on the situation (sunny, cloudy, rain, snow, etc.), temperature, humidity, rainfall, alarm, and the like. Weather information can be visualized by symbols, characters, and the like. For example, a symbol may be displayed at a corresponding place on the map, or a character may be displayed at a position unrelated to the corresponding place on the map. Alternatively, the weather information may be visualized by a graphic such as a weather map.

Further, the map information is useful information such as store information, sightseeing information, traffic jam information, and the like. Useful information can be visualized by symbols, characters and the like. For example, a symbol may be displayed at a corresponding place on the map, or a character may be displayed at a position unrelated to the corresponding place on the map.

Further, the map information is route information, for example. The route information is information on a route (for example, a navigation route) connecting a plurality of points. For example, the route information can be visualized by applying an expression format such as a thick line to the corresponding route on the map. Alternatively, a symbol such as an arrow may be displayed on the corresponding route on the map.

The map information may include only one type of information such as topographic information, or may include a plurality of types of information.

The first display information and the second display information have the same category in that both are the map information. Hereinafter, the map information as the first display information may be referred to as first map information, and the map information as the second display information may be referred to as second map information.

On the other hand, the first map information and the second map information are different in specific configuration. That is, the second map information is information related to the first map information, but differs from the first map information in content or expression format.

Regarding the relationship between the first map information and the second map information, for example, when the first map information and the second map information include the same location on the map, the first map information and the second map information are related. . In this case, the same place is actually displayed as a part of the first map information (that is, the first display information).

Also, for example, when the first map information and the second map information include a specific route on the map, the first map information and the second map information are related. The specific route is a road, an iron road, or the like having the same name. The specific route may be a route connecting a plurality of points (for example, a navigation route). When the first map information and the second map information are related via a specific route, the portion on the second map information side in the specific route may not appear on the display surface.

Regarding the difference in content between the first map information and the second map information, for example, if the target range is different on the map, the content that the map information provides to the user is different.

Also, for example, even in the same place, a map including only terrain information is different from a map including terrain information and location state information. Further, for example, even in the same place, the contents of the ground state information and the underground state information are different. That is, map information having different contents is configured according to the type and combination of information displayed on the map. Note that the amount of information to be displayed on the map may be omitted or added according to the scale of the map.

Also, for example, even at the same place, the contents of the current map information and the old map information are different.

Regarding the difference in the expression format between the first map information and the second map information, for example, even if they are maps of the same place, a line diagram, a monochrome diagram, a colored diagram, a photograph (including aerial photographs), a plane The representation format of map information is different between a target diagram, a three-dimensional diagram, a bird's eye view, a realistic diagram, a simplified diagram, and a deformed map (a map in which some elements such as a navigation route are emphasized).

Also, for example, the display format differs depending on whether the location status information with the same content is displayed as symbols or characters. The same applies to weather information, useful information, and the like.

FIGS. 24 to 27 illustrate display information of the window 80. FIG.

In the example of FIG. 24, an underground map is displayed in the window 80 as an example of an underground map. More specifically, a map is displayed as the first display information. On the map, the user can open a window for a place where he / she wants to see the map of the underground mall (in FIG. 24, two-point touch long press is performed). As illustrated). Thereby, the window 80 is formed in the range designated by the window opening operation, and the map of the underground shopping area in the range is displayed on the window 80 as the second display information.

In FIG. 24, only the underground shopping street is shown in a diagram in order to avoid complication of the drawing, but other various information (for example, store information) may be posted on the map. The underground map is a visualization of underground state information, and may be a map of underground facilities (water, electricity, gas, communication, railway, etc.), ruins, and the like.

Thus, when the second display information is an underground map, the spatial vertical position relationship of the same place can be easily grasped.

Instead of the underground map, an aerial photograph, a deformed map, etc. may be displayed as the second display information. According to the aerial photograph, it is possible to easily grasp the relationship between the normal map appearance and the photograph appearance. Further, according to the deformed map, a place related to a normal map can be easily grasped by highlighting based on a specific viewpoint (for example, a navigation route).

In the example of FIG. 25, an old map at the window position is displayed in the window 80. Comparing FIG. 25 with the upper diagram of FIG. 24, it can be seen that there was no railway in the past and the current station is not maintained. Thus, when the second display information is an old map, the temporal relationship between the same places can be easily grasped. An old map may be displayed in an expression format such as an aerial photograph or a deformed map.

Here, the map of the first display information may be transparent in the range of the window 80 by adjusting the transparency of the upper layer (see FIGS. 22 and 23) for the range of the window 80. According to this, since different maps of the same place are displayed in an overlapping manner, it is easy to compare the two maps.

24 and 25, the second display information is a map, but the second display information may be displayed by a visualization method such as characters.

Here, in both the examples of FIGS. 24 and 25, when the window opening operation is performed on the first display information visualized by the map, the information related to the location where the window 80 is located on the map is first displayed. 2 is displayed as display information. In other words, the map of the first display information is partially replaced by the map of the second display information. According to this, the first display information and the second display information can be viewed with the continuity of the viewpoint (position) for viewing the map, in other words, with the continuity of the map information. For this reason, a user's cognitive load can be reduced.

In the example of FIG. 26, map information near the current location is displayed as first display information by a deformed map with emphasized route information. Further, as the second display information in the window 80, route information from the current location to the destination is displayed by a simplified map. For reference, in FIG. 26, the display in the window 80 is shown enlarged in the upper right in the figure. According to this, the user can grasp the whole image up to the destination (specifically, the direction to the destination, the distance, the shape of the route, etc.).

In addition, if useful information along the route is added to the second display information, the user can use the information, which is convenient. For example, the traffic jam information may be displayed instead of or in addition to the route information. Thereby, the user can estimate the time of the journey to the destination, etc., and can change the journey as necessary. For example, the route can be changed to a route that avoids traffic jams, or the time distribution can be adjusted by taking a break earlier.

FIG. 27 shows an example in which weather information around the destination is displayed as the second display information in the window 80. According to this, the user can make an action plan when arriving near the destination before arriving at the destination.

Of course, as the second display information in the window 80, geographical information around the destination may be displayed. According to this, the user can make an action plan when approaching the destination before arriving at the destination.

It should be noted that the second display information may be information regarding the waypoint instead of the destination. That is, the second display information can be related to a preset setting place. In addition, the set place and the route may be for the purpose of actual navigation, or may be for the purpose of simple route search without navigation.

By displaying the information related to the set location in the window 80, the information related to the set location can be displayed in a preferred size on the preferred location on the display surface 32. For this reason, improvement in work efficiency can be expected for the planning and change of the process plan.

Note that the second display information displayed in the window 80 is not limited to the one exemplified here. The second display information displayed immediately after the window 80 is formed may be set in advance, for example. Further, the setting may be varied depending on the contents of the first display information, the usage status of the information display device 10, and the like.

As described above, the second display information displayed in the window 80 is information related to the first display information, but is different from the first display information in content or expression format. For this reason, the efficiency of information recognition can be improved by displaying various information in one screen. Moreover, the high intuition and the reduction of the cognitive load that the window opening operation plays as described above also contribute to the efficiency of information recognition.

The first display information and the second display information are information (corresponding to the first information 60 and the second information 62 illustrated in FIG. 14) stored in the storage unit 18 (see FIG. 14). Information in the storage unit 18 can be newly acquired or updated by the communication unit as described above. According to the communication unit, for example, traffic jam information and the like can be acquired from outside the information display device 10.

Further, the second display information is obtained by searching and extracting information stored in the storage unit 18 and a storage device (including a server on the network) external to the information display device 10 when necessary. Is possible. Alternatively, the first display information and the second display information may be stored in a database in advance by organizing the relevance with the information used as the first display information.

<Processing after window formation>
FIG. 28 illustrates a processing flow S30 after the window is formed. In the example of FIG. 28, steps S31 and S32 are the same as steps S11 and S12 of FIG. That is, in step S31, the input unit 14 receives a user operation, and in step S32, the control unit 16 identifies the input user operation.

In step S33, the control unit 16 determines that the user operation received in step S31 includes the first display information outside the window 80, the second display information inside the window 80, and the window 80 itself. Identify the target. Specifically, the control target by the user operation is identified from the input position and type of the user operation. At that time, it is possible to identify whether or not the input position of the user operation is related to the window 80 by referring to the management information by the window management unit 56 (see FIG. 14).

<Operations outside the window>
When it is determined in step S33 that the user operation is performed on the first display information outside the window 80, the control unit 16 controls the first display information according to the user operation in step S34. That is, control of slide (in other words, scroll), enlargement, reduction, rotation, and the like is performed according to the function associated with the user operation.

FIG. 29 illustrates the control in step S34. In FIG. 29, P, Q, and R schematically show information drawn on the upper layer, and p, q, and r schematically show information drawn on the lower layer. In FIG. 29, as in FIG. 22, the portion corresponding to the window 80 in the upper layer is set as a non-drawing portion (in other words, a transparent portion), and thus the upper layer displays the first display information. The lower layer provides the second display information.

In the example of FIG. 29, a slide operation is performed on the outside of the window 80, and the contents of the upper layer are updated accordingly. As a result, the content of the first display information outside the window changes. At this time, the position of the window 80 on the display surface 32 is not slid.

In particular, in the example of FIG. 29, the upper layer and the lower layer are linked, and the content of the lower layer is also updated according to the slide operation. That is, the content of the second display information in the window 80 also changes.

Such interlocking of the display inside and outside the window 80 can be employed in the examples of FIGS. 24 and 25, for example (see FIG. 30). That is, when the first display information is partially replaced by the second display information, and the first display information and the second display information have continuity or integrity on the display even after the replacement, the display is performed. Interlocking is useful.

Note that the display interlocking can also be applied to other user operations such as an enlargement operation. In addition, FIG. 29 illustrates the case where the upper layer corresponds to the first display information outside the window 80, but display interlocking can also be applied when the first display information corresponds to the lower layer (see FIG. 23). is there.

On the other hand, in the example of FIG. 31, the display is not interlocked in and out of the window 80 (display is not interlocked). In FIG. 31, α schematically shows information drawn on the upper layer, and P, Q, and R schematically show information drawn on the lower layer. In addition, in FIG. 31, a portion corresponding to the window 80 is configured in the upper layer as in FIG. 23, whereby the upper layer provides the second display information and the lower layer provides the first display information. Yes.

In the example of FIG. 31, a slide operation is performed on the outside of the window 80, and the contents of the lower layer are updated accordingly. As a result, the content of the first display information outside the window changes. On the other hand, the contents of the upper layer are not updated, and therefore the second display information in the window does not change.

Such non-interlocking of the display inside and outside the window 80 can be adopted in the examples of FIGS. 26 and 27, for example. That is, when the first display information and the second display information do not have display continuity or unity, display non-linkage is useful.

Note that the non-linkage of the display can be applied to other user operations such as an enlargement operation. In addition, FIG. 31 illustrates the case where the lower layer corresponds to the first display information outside the window 80, but display non-linkage can also be applied when the first display information corresponds to the upper layer (see FIG. 22). It is.

Whether to link the display may be set in advance, for example. Further, the setting may be made different depending on the visualization method of the first display information and the second display information, the usage status of the information display device 10, and the like.

28, after the execution of step S34, the processing of the information display device 10 returns to step S31.

<Operations inside the window>
On the other hand, if it is determined in step S33 that the user operation is performed on the second display information inside the window 80, the control unit 16 controls the second display information in accordance with the user operation in step S35. That is, control of slide (in other words, scroll), enlargement, reduction, rotation, and the like is performed according to the function associated with the user operation. It should be noted that also in step S35, interlocking display inside and outside the window may be employed.

As another example of the user operation on the inside of the window 80, there is an in-window information switching operation instructing switching of display information in the window 80. By such an in-window information switching operation, the control unit 16 causes the window 80 to display the second display information having different contents or another expression format.

FIG. 32 shows a conceptual diagram of the switching operation of the second display information in the window 80. In the example of FIG. 32, N pieces of predetermined second display information (N is a natural number) are cyclically switched in a predetermined order by an in-window information switching operation. For example, it is possible to switch between an aerial photograph, an underground map, and an old map. Alternatively, it is possible to switch between route information from the current location to the set location, weather information around the set location, geographic information around the set location, and traffic information from the current location to the set location.

Note that the second display information displayed immediately after the window 80 is formed may be, for example, always the same second display information, or may be the second display information displayed last in the previous display. In addition, the number and type of second display information to be switched, the switching order, on / off of cyclic switching, and the like are set in advance, and the setting may not be changed or may be changed. Also good.

For example, flick, double tap, etc. are assigned as the information switching operation in the window. The window information switching operation is accepted at an arbitrary position in the window 80. For this reason, compared with the case where, for example, a specific button is pressed, the user does not have to worry about the execution position of the in-window information switching operation. That is, the user can perform the switching operation while keeping the viewpoint on the second display information in the window 80. For this reason, a user's cognitive load can be reduced. Moreover, since the user can instruct the switching speed, in other words, the switching timing, the cognitive load can be reduced in this respect as well.

The second display information to be switched may be hierarchized information, for example. As the second display information layered, map information of each floor, which is internal map information of a building, is exemplified. That is, the map information on each floor can be hierarchized according to the position in the direction of gravity. For example, as shown in FIG. 33, the first floor, the second floor,..., The top floor, the basement bottom floor,. Or the map information which shows the outline of all the floors may be displayed immediately after window formation, and cyclic switching may be started from the floor which the user selected.

Also, information that is hierarchized according to the passage of time may be constituted by old map information about the same place. For example, location information about 10 years ago, 20 years ago, and 30 years ago for the same location can be hierarchized over time. Further, the transition of the name information of the same place may be hierarchized. In addition to the old map information, the current map information may be added to the hierarchy.

Also, for example, when the underground state information is a ruins, etc., it may be understood that the stratification of the ground state information and the underground state information is based on the position in the direction of gravity, or the time elapses You may understand that it is based on.

The above effect can also be obtained when the second display information layered is switched. In particular, since layered information is more related and continuity of each information, it is efficient to see a plurality of such highly related and continuous information in the same window 80.

Note that the second display information layered may be associated with the icon 90 (see FIG. 18). In particular, when the hierarchical second display information is internal map information of each floor of a building, the window opening operation for the icon 90 and the window information switching operation can be combined to provide higher intuition.

28, after the execution of step S35, the processing of the information display device 10 returns to step S31.

<Operations on windows>
If it is determined in step S33 in FIG. 28 that the user operation is a window control operation for controlling the window 80 itself, the control unit 16 controls the control contents assigned to the input window control operation in step S36. The window 80 is controlled accordingly. According to this, as exemplified below, the position and size of the window 80 can be controlled after the window 80 is formed, and control for deleting the window 80 can be performed by a gesture operation.

<Window move operation>
The window control operation is an operation for moving the window 80, for example. FIG. 34 shows a conceptual diagram of such a window moving operation. According to the example of FIG. 34, by performing a drag operation in a state where a predetermined portion (for example, a frame portion of the window) of the window 80 is touched, the window 80 moves in the drag direction. According to this, since the drag operation is highly similar to an operation of moving an object on a desk in daily life, the window 80 can be moved intuitively. For this reason, high operability can be realized.

<Window size change operation>
The window control operation is an operation for changing the size of the window 80, for example. 35 to 37 show conceptual diagrams of such window size changing operation. 35 to 37, the size of the window 80 is changed by a one-point moving type pinch operation in a state where a predetermined portion (for example, a frame portion of the window) of the window 80 is touched. At this time, if it is pinch out, the window 80 is enlarged (see FIGS. 35 to 37), and if it is pinch in, the window 80 is reduced. That is, whether to enlarge or reduce is instructed according to the pinch direction.

According to this, the window size changing operation has high similarity or continuity with the display size changing operation for the display information inside and outside the window. As a result, it is possible to prevent the user from getting lost in the operation and shorten the operation time. That is, for this reason, high operability can be realized.

Further, the enlargement direction and the reduction direction, that is, the deformation direction are indicated by the pinch direction. Specifically, in the example of FIG. 35, pinch out is performed in the left direction, and the window 80 extends in the left direction. In the example of FIG. 36, pinching out is performed in the downward direction, and the window 80 extends downward. In the example of FIG. 37, pinch-out is performed in the diagonally downward left direction, and the window 80 extends in the left and downward directions. According to this, the deformation direction of the window 80 can be intuitively and easily instructed.

35 and 36 exemplify the case where the starting point of the finger to be moved in the one-point moving type pinch operation is in the frame portion of the window 80, but the present invention is not limited to this example. . That is, as shown in FIG. 37, the starting point of finger movement may be placed inside the window 80. This is based on the fact that the finger to be fixed in the one-point moving type pinch operation is in the frame portion of the window 80 so that the window size changing operation can be distinguished from the operation for the display information in the window 80.

<Window deletion operation>
The window control operation is, for example, an operation for deleting the window 80, in other words, an operation for ending the display of the window 80. 38 to 40 show conceptual diagrams of such window erasing operation.

38, the window 80 is deleted by performing a flick operation on the window 80. Since the flick operation is highly similar to the operation of flipping off objects on the desk and erasing them from view in daily life, the window 80 can be erased intuitively. For this reason, high operability can be realized.

Note that the flick direction is not limited to the example of FIG. In the example of FIG. 38, a predetermined portion of the window 80 (for example, a frame portion of the window) is set as the flick start point. However, if the flick in the window 80 is not assigned to the control (slide etc.) of the second display information in the window 80, the start point of the flick for the window erasing operation may be in the window 80.

According to the example of FIG. 39, a drag operation is performed so as to enter the window 80 from the outside of the window 80 and exit outside the window 80 on a side different from the entry side, in other words, to divide the window 80. As a result, the window 80 is erased. Since such a drag operation is highly similar to the operation of erasing unnecessary portions on a document by slashing them in daily life, the window 80 can be erased intuitively. For this reason, high operability can be realized.

Note that the drag direction is not limited to the example of FIG. Also, other gestures, specifically flicks, may be employed for the window erasing operation in the example of FIG.

40, by performing a drag operation so as to sandwich the window 80, the window 80 is erased. According to this, such a drag operation is highly similar to the operation of closing the window of the room and erasing the scenery outside the room from the field of view in daily life, so the window 80 can be erased intuitively. For this reason, high operability can be realized.

In FIG. 40, a two-point movement type pinch-in is illustrated as a drag operation so as to sandwich the window 80, but a one-point movement type pinch-in may be employed. Further, the pinch-in direction is not limited to the example of FIG. Also, the end of the drag may be a flick.

34 to 38 illustrate the case where the upper layer corresponds to the first display information outside the window 80, but the window control operation is also performed when the first display information corresponds to the lower layer (see FIG. 23). Applicable.

Referring back to FIG. 28, after executing step S36, the control unit 16 determines whether or not step S36 is deletion control of the window 80 in step S37. When step S36 is not erasure control of the window 80, the process of the information display apparatus 10 returns to step S31. On the other hand, when step S36 is the deletion control of the window 80, the information display apparatus 10 ends the processing flow S30 of FIG. 28 and returns to the processing flow S10 of FIG.

<Other examples of window formation>
Hereinafter, an example in which the control unit 16 converts the range specified by the window opening operation according to a predetermined conversion rule and forms a window in the converted range will be described.

For example, as illustrated in FIG. 41, when the first display information is displayed in a bird's eye view, the shape of the window 80 may be set according to the depression angle of the bird's eye view. Further, the second display information in the window 80 may also be displayed by a bird's eye view expression of the same depression angle. According to this, by adopting the same expression format, the continuity between the first display information and the second display information is increased, and the user's cognitive load can be reduced.

Various methods of bird's-eye view expression are known, and such known methods are used here. For example, a method of converting an image drawn as a top view or a front view into a bird's eye view representation is employed. The generation of the bird's eye view image may be performed by the overall control unit 42, or may be performed by the first image forming unit 44 and the second image forming unit 48. The bird's-eye view expression can be applied not only to figures but also to characters and the like.

More specifically, as illustrated in FIGS. 42 and 43, the viewpoint in the bird's-eye view is set for the original image, and the parallel lines running in the vertical direction of the original image are bundled at the viewpoint. Deform. 42 and 43, windows 80 are provided at various positions for the sake of convenience in order to facilitate understanding of the bird's-eye view conversion. 42 and 43 have different viewpoint positions, and a broken-line rectangle in FIG. 43 corresponds to the window 80 in FIG.

As described above, the rectangular range (see FIGS. 16 to 18) designated by the user by the window opening operation is converted into a substantially trapezoid when the control unit 16 converts it into a bird's eye view expression. According to this, the user does not need to specify the formation range of the window 80 while being aware of the bird's-eye view expression of the first display information. That is, if a normal rectangular range is designated, the control unit 16 automatically forms a window 80 that matches the bird's-eye view.

It should be noted that the trapezoidal window 80 may be deformed. For example, in the example of FIG. 44, the trapezoidal side is curved.

FIG. 45 shows a further example of the shape of the window 80. Schematically, the first display information is visualized by a map, and a window 80 is formed in accordance with a section or the like in the map. At this time, the control unit 16 converts the range specified by the window opening operation according to the following first rule and second rule, and forms the window 80 in the converted range.

The first rule is that the periphery of the range specified by the window opening operation (drag trajectory 70 in FIG. 45), the boundary of the partition in the map, and the periphery of the map display area (the entire display surface 32 in the example of FIG. 45). In addition, the content is to be deformed together.

For example, the user-specified range is expanded so as to inflate a balloon, and the periphery of the user-specified range is made coincident with the partition boundary in the map. Note that the partition boundaries are roads, rivers, administrative partitions, and the like. The reason why the first rule considers the periphery of the map display area in addition to the partition boundaries in the map is to prevent the window 80 from exceeding the display area as a result of the expansion of the user-specified range.

The content of the second rule is that the user-specified range converted in accordance with the first rule includes a range originally specified by the user in a predetermined ratio or more. The second rule is to set an upper limit value in order to prevent the window 80 from becoming too larger than the range specified by the user.

In the example of FIG. 45, the window 80 is formed so as to accommodate the entire initial user-specified range. On the other hand, the window 80 may be partially retracted from the initial user-specified range. For example, in order to find a partition boundary that satisfies the second rule, it may occur that a partition boundary at a position backward from the partition boundary selected as a candidate is selected again.

The adoption of the first and second rules described above eliminates the need for the user to track complicated partition boundaries. That is, if a normal rectangular range is designated, the control unit 16 automatically forms a window 80 that matches the partition boundary. In addition, a large section obtained by combining a plurality of adjacent sections can be easily specified.

<Effect>
According to the information display device 10, the position, size, content, etc. of the display information can be adjusted to the user's preference by adopting a window opening operation or the like. In addition, the above various effects can be obtained.

<Modification>
Although the case where the 1st display information is displayed on the whole display surface 32 is illustrated above, it is not limited to this example. In addition, a plurality of windows 80 may exist on the display surface 32 at the same time.

Further, by performing a window opening operation on the information in the window 80, a further window 80 may be formed in the existing window 80 (window multiplexing). In this case, the second display information in the existing window 80 is grasped as new first display information, and the display information in the further window 80 is grasped as new second display information. Everything applies to window multiplexing. Multiplexing of windows can be used for switching display information, for example, in the same way as in-window information switching operation.

In addition, although the case where the first display information and the second display information are map information has been illustrated above, the present invention is not limited to this example. For example, the first display information and the second display information may be music information.

The music information can be composed of various information such as a song name, artist name, songwriter name, composer name, arranger name, recorded album name, release date, release source, and the like.

More specifically, it is assumed that a list of a plurality of artist names is displayed as the first display information in an expression format using characters, icons, and the like. When a window opening operation is performed on a certain artist name in the list, a window 80 is displayed, and information on one album of the artist is displayed in the window 80 as second display information. When the in-window information switching operation is performed, information on another album is displayed. At this time, the album information is switched in order of release date.

Note that the music information and operation are not limited to this example, and all of the above description applies to the case where the first display information and the second display information are music information.

In the above, a contact type touch panel is exemplified as the input unit 14. On the other hand, a non-contact type (also referred to as a three-dimensional (3D) type) touch panel can be used for the input unit 14.

According to the non-contact type, a detectable region of the sensor group (in other words, an input region that can accept user input) is provided as a three-dimensional space on the input surface, and a finger in the three-dimensional space is placed on the input surface. The position projected on is detected. Some non-contact types can detect the distance from the input surface to the finger. According to this method, the finger position can be detected as a three-dimensional position, and further, the approach and retreat of the finger can also be detected. Various systems have been developed as non-contact type touch panels. For example, a projection capacity system which is one of electrostatic capacity systems is known.

In the above description, the finger is exemplified as the indicator used by the user for input. However, for example, a part other than the finger can be used as the indicator. For example, a tool such as a touch pen (also referred to as a stylus pen) may be used as an indicator.

Also, a so-called motion sensing technology may be used for the input unit 14. Various methods have been developed as motion sensing technology. For example, a method is known in which a user's movement is detected by a user holding or wearing a controller equipped with an acceleration sensor or the like. In addition, for example, a method of extracting a feature point such as a finger from a captured image of a camera and detecting a user's movement from the extraction result is known. An intuitive operation environment is also provided by the input unit 14 using the motion sensing technology.

In addition, although the input / display unit 20 is illustrated above, the display unit 12 and the input unit 14 may be arranged separately. Even in this case, an intuitive operation environment is provided by configuring the input unit 14 with a touch panel or the like.

Further, in the present invention, the embodiments can be appropriately modified and omitted within the scope of the invention.

10 information display device, 12 display unit, 14 input unit, 16 control unit, 18 storage unit, 20 input / display unit, 32 display surface, 34 input surface (input area), 80 windows, 90 icons, S10, S30 processing flow .

Claims

A display unit having a display surface;
An input unit that accepts user operations;
When a window opening operation for instructing to form a window on the display surface and designating a window forming range is performed on the first display information on the display surface as the user operation, the window The window is formed according to the formation range designated by the opening operation, and second display information related to the first display information but having a content or expression format different from the first display information is displayed on the window. An information display device comprising a control unit.
The first display information is first map information visualized by a map,
The second display information is second map information related to a place where the window is located on the map.
The information display device according to claim 1.
The information display device according to claim 2, wherein the second map information includes any one of an underground map, an old map, an aerial photograph, and a deformed map.
When the in-window information switching operation instructing switching of display information in the window is performed at an arbitrary position in the window as the user operation, the control unit has another content or another expression format. The information display device according to claim 1, wherein second display information is displayed on the window.
The second display information is a plurality of hierarchical display information,
When the window information switching operation instructing switching of display information in the window is performed at an arbitrary position in the window as the user operation, the control unit switches the hierarchy of the plurality of display information. Display in the window,
The information display device according to claim 1.
6. The information display device according to claim 5, wherein the plurality of display information are a plurality of map information hierarchized according to positions in a gravity direction.
The information display device according to claim 5, wherein the plurality of display information are a plurality of information hierarchized according to the passage of time.
When the window opening operation is performed on an icon in the display area of the first display information, the control unit displays information associated with the icon as the second display information on the window. The information display device according to claim 1.
The first display information is visualized by a map;
The second display information associated with the icon is internal map information of a building existing at the position of the icon on the map.
The information display device according to claim 8.
The second display information is a plurality of hierarchical internal map information,
When the window information switching operation instructing switching of display information in the window is performed at an arbitrary position in the window as the user operation, the control unit switches the hierarchy of the plurality of internal map information. To display in the window,
The information display device according to claim 9.
The information display device according to claim 1, wherein the first display information is first map information, and the second display information is second map information related to a preset setting place.
When the in-window information switching operation instructing switching of display information in the window is performed at an arbitrary position in the window as the user operation, the control unit has another content or another expression format. The information display device according to claim 11, wherein second map information is displayed on the window.
The second map information includes route information from a current location to the set location, weather information around the set location, geographic information around the set location, and traffic jam information from the current location to the set location. The information display device according to claim 12, comprising any one of them.
The information display device according to claim 1, wherein when the first display information is displayed by a bird's eye view, the control unit sets the shape of the window according to a depression angle of the bird's eye view.
The first display information is visualized by a map;
The control unit converts the formation range specified by the window opening operation according to a predetermined conversion rule, forms the window in the converted range,
The predetermined conversion rule is:
A first rule that a periphery of the formation range designated by the window opening operation is deformed according to a partition boundary in the map and a periphery of the display area of the map;
The information display device according to claim 1, wherein the range after conversion includes a second rule that includes the formation range specified by the window opening operation at a predetermined ratio or more.
The information display device according to claim 1, wherein the control unit controls the window according to a control content assigned to the user operation when the user operation is an operation for controlling the window.
The control unit enlarges or reduces the window according to a pinch direction when the user operation is a one-point moving type pinch operation performed in a state where a predetermined portion of the window is touched. 16. The information display device according to 16.
The information display device according to claim 17, wherein the control unit enlarges or reduces the window by deforming the window in the pinch direction.
The information display according to claim 16, wherein, when the user operation is a drag operation performed in a state where a predetermined portion of the window is touched, the control unit moves the window in accordance with the drag operation. apparatus.
The information display device according to claim 16, wherein the control unit deletes the window when the user operation is a flick operation on the window.
The control unit deletes the window when a gesture operation for entering the window from the outside of the window and exiting to the outside of the window on the side different from the entry side is performed as the user operation. Item 17. The information display device according to Item 16.
The information display device according to claim 16, wherein the control unit erases the window when a gesture operation for sandwiching the window is performed as the user operation.
(A) receiving a user operation;
(B) identifying the user operation;
(C) When a window opening operation for instructing to form a window on the display surface and designating a window forming range is performed on the first display information on the display surface as the user operation, Forming the window according to the formation range specified by the window opening operation;
(D) An information display method comprising: displaying, on the window, second display information related to the first display information but having a content or an expression format different from the first display information.