JP5994903B2 - Image display device, image display method, and image display program - Google Patents

Description

  The present invention relates to an image display device, an image display method, and an image display program.

  Conventionally, electronic blackboards have been used in companies, educational institutions, administrative institutions, and the like. The electronic blackboard has a large display, displays a background image on the display, and displays a drawing image such as letters, numbers, and figures written by the user on the background image. For example, an electronic blackboard having a touch panel displays a background image on a screen serving as a blackboard, and displays a drawn image drawn using the touch panel superimposed on the background image.

  Currently, SmartBoard's SmartBoard, Hitachi Software Engineering's StarBoard (registered trademark), Pioneer Cyberconference (registered trademark), and the like are provided as electronic blackboards. There is also an electronic blackboard system including a coordinate input / detection device.

  There is also a device for displaying a digest of a board written as an electronic blackboard. A device that displays a digest of a blackboard captures the screen of the blackboard at regular time intervals. Then, the apparatus for displaying the digest of the blackboard discriminates the character direction type into vertical writing, horizontal writing, and other types according to the shape and size of the rectangle surrounding the drawing portion of the captured screen. Then, the device for displaying the digest of the board writing uses a different method depending on the type of the determined character direction, determines the semantic structure of the board writing screen, divides the area into blocks, and the block is completed. The estimated screen is organized in time series, and the organized screen is displayed as a digest.

  However, the electronic blackboard described above has a problem in that only a stored past board is displayed. For example, on an electronic blackboard, a drawing image is superimposed on a background image, but on the electronic blackboard described above, only a blackboard is displayed.

  The present invention solves the above-described problems, and an object thereof is to provide an image display device, an image display method, and an image display program that enable efficient browsing of past display screens.

  In order to solve the above-described problems and achieve the object, an image display device according to the present invention includes a display control unit that displays video information received from a video information output device in a display area, and a drawing image superimposed on the video information. A superimposing drawing unit for drawing, a switching control unit that switches between a first state in which the drawing image is superimposed and drawn and a second state in which the drawing image is not drawn, and the switching control unit switches to the first state. When switched, the first image information is stored, and when switched to the second state by the switching control unit, the second image information is stored in association with the first image information.

  According to the present invention, it is possible to efficiently browse past display screens.

FIG. 1 is a diagram illustrating an example of an image display system according to the first embodiment. FIG. 2 is a connection diagram illustrating an example of connections inside the image display apparatus according to the first embodiment. FIG. 3 is a diagram illustrating an example of a hardware configuration of the image display apparatus according to the first embodiment. FIG. 4 is a diagram illustrating an example of a controller unit and storage in the first embodiment. FIG. 5 is a diagram illustrating an example of information stored in the snapshot table according to the first embodiment. FIG. 6 is a diagram illustrating an example of information stored in the representative image table according to the first embodiment. FIG. 7 is a diagram illustrating an example of a handwriting stroke generated by the handwriting processing unit according to the first embodiment. FIG. 8 is a diagram illustrating an example of a screen displayed in the display area according to the first embodiment. FIG. 9 is a diagram illustrating an example of information stored in the snapshot table after the representative image ID is stored by the representative image storage unit according to the first embodiment. FIG. 10 is a sequence diagram illustrating an example of a series of processes for storing the second image information according to the first embodiment. FIG. 11 is a diagram illustrating a point where the representative image and the snapshot are browsed side by side. FIG. 12 is a diagram illustrating the point of browsing the representative image and the snapshot side by side.

  Hereinafter, embodiments of an image display device, an image display method, and an image display program will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a diagram illustrating an example of an image display system according to the first embodiment. The image display system 100 includes an image display device 110 and a user PC (Personal Computer) 130. The user PC 130 is also referred to as a “video information output device”. In the example shown in FIG. 1, the image display device 110 and the user PC 130 are connected by cables 123 and 124.

  As will be described in detail below, the image display device 110 displays video information received from the user PC 130 in the display area as a background image. In addition, the image display device 110 superimposes and draws a drawing image drawn by the user on the background image, as will be described in detail below. Note that the image display device 110 may display the video information on the entire display area or a part of the display area.

  In the example illustrated in FIG. 1, the image display device 110 includes a display unit 112, a touch panel 113, operation buttons 114, operation buttons 116, a video information interface 120, and a display image interface 122. Although not shown in FIG. 1, the image display device 110 includes a controller unit 126 and a storage 125.

  The display unit 112 has a display area, and displays a background image and a drawing image in the display area. The touch panel 113 is provided on the display area. Note that when a predetermined object approaches or comes into contact with the touch panel 113, the contact position coordinates and the approach position coordinates are detected by the controller unit 126 described later. The predetermined object is, for example, a user's finger or a dedicated pen.

  The operation button 114 and the operation button 116 receive an instruction to switch the operation mode of the image display apparatus 110 from the user. In the first embodiment, the case where the operation button 114 and the operation button 116 are buttons formed by hardware will be described. However, the present invention is not limited to this, and is a button formed by software. May be.

  In the following, description will be given using a case where the image display apparatus 110 executes processing in the “first state” or the “second state” as the operation mode. In other words, the case where the image display device 110 has two operation modes of “first state” and “second state” will be described. The first state indicates a state in which the drawn image is superimposed and drawn on the background image when the contact position coordinates or the approach position coordinates are detected. The second state indicates a state in which the drawn image is not superimposed and drawn on the background image even if the contact position coordinates or the approach position coordinates are detected.

  For example, in the following description, the case where the image display device 110 is in the first state when the operation button 114 is operated and the image display device 110 is in the second state when the operation button 116 is operated will be described. To do. For example, when the operation button 114 or the operation button 116 is operated by the user, the image display device 110 can be switched alternately between the first state and the second state in accordance with the user's instruction. Note that the operation mode is not limited to the first state and the second state, and there may be another operation mode separately from the first state and the second state.

  The video information interface 120 is an interface that receives video information displayed as a background image on the display unit 112 from the user PC 130. The video information interface 120 is a VGA input connector configured by, for example, a VGA (Video Graphics Array) terminal. The video information is a video signal transmitted from the user PC 130. The video information is, for example, a VGA signal. The video information interface 120 receives a VGA signal from the user PC 130 via a cable 123 such as a VGA cable, and outputs the received VGA signal to the controller unit 126 of the image display device 110.

  Note that the video information interface 120 may receive video information by any method. For example, the video information interface 120 may receive the video information from the user PC 130 using wireless communication conforming to a wireless communication protocol such as Bluetooth (registered trademark) or WiFi.

  The display image interface 122 is an interface used for processing such as acquisition and deletion of a display image displayed on the image display device 110. The display image interface 122 is, for example, a USB (Universal Serial Bus) socket. The display image interface 122 outputs various instruction information from the user PC 130 to the controller unit 126 and outputs instruction information from the controller unit 126 to the user PC via a cable 124 such as a USB cable. The instruction information corresponds to, for example, instruction information indicating a mouse event in which an output coordinate corresponding to a touched portion in the display area is clicked once.

  Here, an example of the connection inside the image display apparatus 110 in the first embodiment is shown. FIG. 2 is a connection diagram illustrating an example of connections inside the image display apparatus according to the first embodiment. In the example shown in FIG. 2, the user PC 130 is also shown for reference. In the example illustrated in FIG. 2, the display unit 112, the touch panel 113, the operation buttons 114, the operation buttons 116, the video information interface 120, the display image interface 122, and the storage 125 are each connected to the controller unit 126. The The video information interface 120 and the display image interface 122 are connected to the user PC 130 via the cable 123 and the cable 124, respectively.

  In the example illustrated in FIG. 2, the image display apparatus 110 further includes an arbitrary external interface 127 and the controller unit 126 is further connected to the arbitrary external interface 127. However, the present invention is not limited thereto. Is not to be done. The external interface 127 may be an arbitrary interface, such as a LAN (Local Area Network) port or IEEE 1394 (Institute of Electrical and Electronic Engineers 1394).

  For example, a VGA signal input from the user PC 130 is input to the controller unit 126 via the video information interface 120. Further, the user PC 130 can acquire data from the controller unit 126 as a result of being connected to the image display device 110 via a cable 124 such as a USB cable. For example, the user PC 130 can acquire a display image displayed on the image display device 110, a mouse event, and the like. That is, the user PC 130 can recognize the image display device 110 as an HID (Human Interface Device) device or a storage device in the USB standard. As a result, the user PC 130 can also acquire a mouse event or file from the image display device 110 without installing a driver or the like.

  Here, an example of the hardware configuration of the image display apparatus according to the first embodiment will be described. FIG. 3 is a diagram illustrating an example of a hardware configuration of the image display apparatus according to the first embodiment.

  In the example illustrated in FIG. 3, the image display device 110 includes a processor 150, a ROM 151, a RAM 152, a hard disk device (HDD, Hard Disk Dive) 153, a graphics board 154, a display 155, a detection device 156, A network interface 157 and an external device connection interface 158 are included.

  A processor 150, a ROM (Read Only Memory) 151, a RAM (Random Access Memory) 152, a graphics board 154, and the like in FIG. 3 correspond to the controller unit 126 in FIG. 1 and FIG. 3 corresponds to the storage 125 in FIGS. 1 and 2, for example. Also, the network interface 157 and the external device connection interface 158 in FIG. 3 correspond to, for example, the video information interface 120 and the display image interface 122 in FIG. 1 and FIG. 3 corresponds to, for example, the touch panel 113 in FIGS. 1 and 2. The display 155 in FIG. 3 corresponds to the display unit 112 in FIGS. 1 and 2, for example.

  The processor 150 calculates processing executed by the image display device 110. The processor 150 is, for example, a CPU (Central Processing Unit) such as PENTIUM (registered trademark) or Athlon, an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable). Various processors such as (Gate Array). The ROM 151 is a non-volatile memory in which a boot program such as BIOS (Basic Input / Output System) is stored. The RAM 152 is a non-volatile memory that provides an execution space for executing a program executed by the image display apparatus 110 of the present embodiment.

  In the image display device 110, a program written in a program language such as assembler, C, C ++, Java (registered trademark), JavaScript (registered trademark), PERL, RUBY, or PYTHON is developed under the management of an OS (Operating System). By executing the above, a series of processing described later is realized in the image display device 110.

  The hard disk device 153 is a large-capacity nonvolatile memory. The hard disk device 153 includes various OSs such as WINDOWS (registered trademark) series, UNIX (registered trademark), LINUX (registered trademark), Android (registered trademark), ITRON, and μITRON that control the entire system of the image display device 110, for example. Remember. The hard disk device 153 stores various data such as device drivers such as the graphics board 154, the display 155, and the detection device 156, and programs executed by the image display device 110 of the present embodiment.

  Note that programs executed by the image display apparatus 110 according to the present embodiment are CD-ROM, MO (Magneto Optical disk), DVD (Digital Versatile Disk) flexible disk, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM ( Erasable Programmable Read Only Memory) or any other device-readable recording medium may be stored and installed, or may be installed via a network.

  The graphics board 154 is a device that processes drawing data to be drawn on the display 155 and includes a VRAM (Video RAM) that holds an image and a connection terminal of the display 155. The display 155 is a display device that draws drawing data generated by the graphics board 154.

  The detection device 156 is a detection device that detects that an object has contacted or approached. The detection device 156 corresponds to, for example, a touch panel including a capacitance sensor that detects a capacitance that changes when an object comes into contact. However, the present invention is not limited to this, and the detection device 156 may be a touch panel provided with a pressure sensor that detects the contact pressure of an object. An optical detection device that detects that light in two lateral directions is blocked and detects that an object approaches or contacts the display 155 may be used, or any device may be used. Note that when the detection device 156 detects that the light is blocked, the touch panel 113 may not be provided on the display area.

  The network interface 157 is an interface connected to an external network, and is a physical interface such as a LAN (Local Area Network) cable connector or a modular cable connector. The external device connection interface 158 is an interface for connecting an external device such as the user PC 130. The network interface 157 is a physical interface such as a VGA input connector or a USB port.

  Returning to the description of FIG. The user PC 130 outputs video information used as a background image in the image display device 110 to the image display device 110. In the example illustrated in FIG. 1, the user PC 130 is a notebook PC, but the present invention is not limited to this. The user PC 130 may be any information terminal that can output video information, and may be, for example, a desktop PC, a tablet PC, or a PDA (Personal Digital Assistants). .

  The user PC 130 includes an interface that outputs a display image displayed on the display screen of the user PC 130, and outputs the display image displayed on the display screen of the user PC 130 to the image display device 110 as video information. For example, the user PC 130 includes a VGA output terminal as an interface, and outputs video information to the image display device 110 as a VGA signal. However, the present invention is not limited to this, and the user PC 130 may output the video information using wireless communication complying with any of various wireless communication protocols.

  Further, the user PC 130 acquires a display image displayed by the image display device 110 or receives instruction information indicating an instruction given by the user to the image display device 110. For example, the user PC 130 has a USB port, and uses a general-purpose driver such as Use Mass Storage Class to acquire a display image stored in the image display device 110 connected by USB or receive instruction information. Or

  FIG. 4 is a diagram illustrating an example of a controller unit and storage in the first embodiment. In the example shown in FIG. 4, the user PC 130 is also shown for convenience of explanation.

  In the example illustrated in FIG. 4, the storage 125 includes a snapshot table 201 and a representative image table 202. The storage 125 is also referred to as a “storage device”. The storage 125 is, for example, a semiconductor memory element such as a flash memory, a hard disk, an optical disk, or the like. In the example illustrated in FIG. 2, the storage 125 is provided separately from the controller unit 126, but the present invention is not limited to this. For example, the storage 125 may be integrated with the controller unit 126. In other words, the storage 125 may be provided inside the controller unit 126.

  The snapshot table 201 stores first image information that is information for reproducing the display image displayed in the display area of the display unit 112 in the first state. Specifically, the snapshot table 201 stores a history of snapshots of the display image displayed in the display area in the first state in time series.

  For example, the snapshot table 201 includes, as the first image information, the contact position coordinates or the approach position for drawing the video information displayed as the background image in the display area and the drawn image superimposed on the video information. The coordinates are stored in association with each other. However, the first image information stored by the snapshot table 201 is not limited to this. The snapshot table 201 may store the display image itself displayed in the display area as the first image information as a snapshot. The snapshot table 201 may store the drawn image itself instead of the contact position coordinates or the approach position coordinates.

  The representative image table 202 stores second image information that is information for reproducing the display image displayed in the display area of the display unit 112 in the second state. For example, the representative image table 202 stores a snapshot of the display image displayed in the display area when the contact position coordinates or the approach position coordinates are detected in the second state. To explain with a more detailed example, the representative image table 202 is displayed in the display area after the instruction information is transmitted to the user PC 130 as a result of detecting the contact position coordinates or the approach position coordinates in the second state. A snapshot of the displayed image is stored. The representative image table 202 stores the history of the second image information in time series. Hereinafter, the second image information is also referred to as “representative image”.

  However, the second image information stored by the representative image table 202 is not limited to this. The representative image table 202 may store the video information and the contact position coordinates or the approach position coordinates as the second image information in association with each other, and stores the drawing image itself instead of the contact position coordinates or the approach position coordinates. It may be stored in any format.

  Here, the first image information stored in the snapshot table 201 is associated with the second image information stored in the representative image table 202 by the controller unit 126. Hereinafter, the snapshot table 201 will be described using a case where the association between the first image information and the second image information is stored. However, the present invention is not limited to this. For example, the representative image table 202 may store the association, or a table different from the snapshot table 201 or the representative image table 202 may store the association.

  An example of information stored in the snapshot table 201 and an example of information stored in the representative image table 202 will be described with reference to FIGS. 5 and 6.

  FIG. 5 is a diagram illustrating an example of information stored in the snapshot table according to the first embodiment. In the example illustrated in FIG. 5, the snapshot table 201 stores “display screen”, “handwritten stroke”, and “representative image ID” in association with each other. The “display screen” in FIG. 5 shows a bitmap of the background image displayed in the display area of the display unit 112 in the first state. The “handwritten stroke” indicates a handwritten stroke that forms a drawn image superimposed on the “display screen”. The “handwritten stroke” includes one or a plurality of handwritten strokes. Each handwritten stroke includes, for example, contact position coordinates or approach position coordinates by the user. Here, the “display screen” and the “handwritten stroke” correspond to “first image information”. The “representative image ID” is identification information for identifying the second image information associated with the first image information. In the example illustrated in FIG. 5, the representative image ID “100” is associated with the second image information identified by the representative image ID “100” among the second image information stored in the representative image table 202. Indicates. In the example illustrated in FIG. 5, the representative image ID “NULL” indicates that the second image information is not yet associated.

  FIG. 6 is a diagram illustrating an example of information stored in the representative image table according to the first embodiment. In the example illustrated in FIG. 6, the representative image table 202 stores “representative image ID” and “display screen” in association with each other. Here, the “display screen” in FIG. 6 shows a bitmap of an image displayed in the display area of the display unit 112 in the second state. The “display screen” in FIG. 6 corresponds to “second image information” and “representative image”.

  In the example shown in FIGS. 5 and 6, the case where a bitmap is stored as a display screen is shown, but the present invention is not limited to this. For example, a combination of a numeric string and a character string indicating a storage format of video information may be used as the display screen, and information in an arbitrary format may be used. In the example shown in FIG. 5, the case where the contact position coordinates or the approach position coordinates are stored as the handwritten stroke is shown, but the present invention is not limited to this. For example, in addition to the contact position coordinates or the approach position coordinates, attribute information of the drawn image such as the display color of the drawn image and the thickness of the line segment may be used as the handwritten stroke.

  Here, the difference between the timing at which the first image information is stored in the snapshot table 201 and the timing at which the second image information is stored in the representative image table 202 will be supplemented. In the first state, the snapshot table 201 stores first image information for each arbitrary timing. For example, the snapshot table 201 stores the first image information at every predetermined timing, or stores the first image information every time a new handwritten stroke occurs. Here, when the first image information is stored, “NULL” is set as the initial value in the “representative image ID”.

  On the other hand, in the second state, the second image information is stored in the representative image table 202 after a new representative image ID is added at every arbitrary timing. For example, the representative image table 202 stores a new representative image ID and second image information each time instruction information is output to the user PC 130 in the second state. As will be described later, for example, when the representative image ID and the second image information are stored in the representative image table 202, the newly stored representative image ID is stored in the snapshot table 201. It is stored in the snapshot table 201 in association with the image information.

  Returning to the description of FIG. The controller unit 126 is, for example, an ASIC, FPGA, CPU, MPU or the like. In the example shown in FIG. 4, the controller unit 126 includes a video acquisition unit 211, a mode switching control unit 212, a touch panel control unit 213, a handwriting processing unit 214, a video processing unit 215, a display control unit 216, and a snap. A shot storage unit 217, an event transfer unit 218, and a representative image storage unit 219 are included.

  The video acquisition unit 211 receives video information from the user PC 130. Specifically, the video acquisition unit 211 acquires video information output by the user PC 130 via the video information interface 120. For example, the video acquisition unit 211 acquires video information displayed in the display area of the user PC 130 for each period T. To explain with a more detailed example, the video acquisition unit 211 acquires an image as a background image from the user PC 130 every T = 1/60 [s].

  The mode switching control unit 212 has a first state in which a drawing image is superimposed and drawn when a contact position coordinate or an approach position coordinate is detected, and a drawing image is superimposed and drawn even if a contact position coordinate or an approach position coordinate is detected To switch to the second state in which the superimposed drawing is not performed. Specifically, the mode switching control unit 212 switches between the first state and the second state alternately in accordance with a user instruction.

  For example, the mode switching control unit 212 switches the operation mode according to the type of operation button operated by the user. To explain with a more detailed example, the mode switching control unit 212 switches to the first state when the operation button 114 is pressed, and changes to the second state when the operation button 116 is pressed. Switch to.

  An example of operation mode switching processing by the mode switching control unit 212 will be further described. For example, when the operation button 114 is pressed, the mode switching control unit 212 switches to the second state by outputting a control command “handwrite disable” to the handwriting processing unit 214 and the touch panel control unit 213. As a result, as will be described later in detail, the process of superimposing and drawing the drawing image on the background image becomes invalid. For example, when the handwriting processing unit 214 becomes invalid, the process of superimposing and drawing the drawing image on the background image becomes invalid.

  Further, for example, when the operation button 116 is pressed, the mode switching control unit 212 switches to the first state by outputting a control command “handwrite enable” to the handwriting processing unit 214 and the touch panel control unit 213. As a result, as will be described later in detail, the process of superimposing and drawing the drawing image on the background image becomes effective. For example, when the handwriting processing unit 214 is enabled, the process of superimposing and drawing the drawing image on the background image is enabled.

  The touch panel control unit 213 controls the touch panel 113 to detect contact position coordinates or approach position coordinates on the display area. For example, the touch panel control unit 213 detects coordinates “(Xt, Yt)” indicating the position touched by the user or the position approached by the user, with an arbitrary point in the display area as the origin. To explain with a more detailed example, the touch panel control unit 213 uses the upper left corner of the display area as the origin in both the display area of the image display device and the display area of the user PC 130, and then sets the horizontal direction. The x axis or the X axis is used, and the vertical direction is used as the y axis or the Y axis. The touch panel control unit 213 is also referred to as a “detection unit”.

  Here, when the touch panel control unit 213 receives the control command “handwrite enable” from the mode switching control unit 212, in other words, in the first state, the detected touch position coordinates or the approach position coordinates are handwritten. To the unit 214. On the other hand, when the touch panel control unit 213 receives the control command “handwrite disable” from the mode switching control unit 212, in other words, in the second state, the touch position coordinate or the approach position coordinate is detected as the video processing unit. To 215.

  The handwriting processing unit 214 executes a series of processes described below in the first state. When the touch position control unit 213 detects the contact position coordinates or the approach position coordinates, the handwriting processing unit 214 receives the detected contact position coordinates or the approach position coordinates, and based on the received contact position coordinates or the approach position coordinates. The rendered image to be rendered is superimposed and rendered on the video information displayed in the display area. Specifically, the handwriting processing unit 214 outputs an instruction to superimpose and draw the superimposed image to the display control unit 216, thereby superimposing and drawing the rendered image on the video information. The handwriting processing unit 214 and the display control unit 216 are also referred to as “superimposed drawing unit”.

  For example, the handwriting processing unit 214 generates a coordinate group detected by the touch panel control unit 213 as a stroke (vector) from touch (contact) to detachment (non-contact), and the generated handwriting stroke is displayed on the background image. Superimpose drawing. The handwriting processing unit 214 may specify, for example, a stroke color or a target pixel range (corresponding to a pen width).

  FIG. 7 is a diagram illustrating an example of a handwriting stroke generated by the handwriting processing unit according to the first embodiment. As shown in FIG. 7, the strokes 301, 302, 303, and 304 are set at arbitrary positions on the display area. In other words, the range of the display area where the handwritten stroke is set is the entire display area. 7 is generated by the handwriting processing unit 214 when the touch panel 113 is touched by the user.

  Note that the handwriting processing unit 214 does not receive the contact position coordinates or the approach position coordinates from the touch panel control unit 213 in the second state. As a result, a new handwritten stroke is not generated, and a new drawn image is not superimposed and drawn on the background image. However, the drawing image that has already been drawn in a superimposed manner remains superimposed on the background image unless the user performs a special operation.

  The video processing unit 215 displays the video information acquired by the video acquisition unit 211 in the display area. Specifically, the video acquisition unit 211 instructs the display control unit 126 to display the video information acquired by the video acquisition unit 211 as a background image in the display area or as an application window in the display area. Or display. In other words, the video processing unit 215 displays a window corresponding to an arbitrary application, and the video information acquired by the video acquisition unit 211 is rendered in the window. Alternatively, the display control unit 216 may be instructed.

  In addition, the video processing unit 215 may display the received video information in the display area after performing image processing such as resizing and clipping as appropriate. The video processing unit 215 and the display control unit 216 are also referred to as “display control unit”.

  In addition, when the video processing unit 215 receives the contact position coordinates or the approach position coordinates from the touch panel control unit 213 in the second state, the video processing unit 215 adds the coordinates of the display area in which the video information is displayed, To 218.

  FIG. 8 is a diagram illustrating an example of a screen displayed in the display area according to the first embodiment. In the example shown in FIG. 8, the user PC 130 is also shown for convenience of description. In the example illustrated in FIG. 8, the video processing unit 215 displays the video information acquired by the video acquisition unit 211 as an application window 310 instead of displaying it on the entire display area.

  An example of processing when the video processing unit 215 receives the contact position coordinates or the approach position coordinates from the touch panel control unit 213 will be described using the example illustrated in FIG. 8. In the example illustrated in FIG. 8, the video processing unit 215 adds the coordinates “Xa, Ya” at which the application window 310 is displayed to the event transfer unit 218 in addition to the coordinates “Xt, Yt” received from the touch panel control unit 213. Output. In the example shown in FIG. 8, the coordinate system of the display area of the image display device 110 is described as “XY coordinate system”, and the coordinate system of the display area of the user PC 130 is described as “xy coordinate system”.

  In the first embodiment, the video processing unit 215 displays the acquired video information as it is when the video acquisition unit 211 acquires the video information displayed in the display area of the user PC. explain. In other words, an example will be described in which display is performed from the display area without changing the resolution of the video information acquired by the video acquisition unit 211. In this case, based on the fact that the coordinate system in the display area of the image display device 110 and the coordinate system in the display area of the user PC 130 do not necessarily match, the video processing unit 215 detects the contact position coordinates detected by the touch panel control unit 213. Alternatively, in addition to the approach position coordinates, coordinates in the display area of the display unit 112 on which the video information is displayed are output. However, it is not limited to this. For example, when the coordinate system in the display area of the image display device 110 matches the coordinate system in the display area of the user PC 130, the video processing unit 215 outputs the contact position coordinates or the approach position coordinates, while the video information It is not necessary to output the coordinates in the display area of the display unit 112 on which is displayed.

  When the display control unit 216 receives an instruction from the handwriting processing unit 214 or the video processing unit 215, the display control unit 216 displays a video in the display area of the display unit 112 based on the received instruction. For example, the display control unit 216 displays video information in the display area and displays a drawing image.

  The snapshot storage unit 217 stores first image information, which is information for reproducing an image displayed in the display area, in the snapshot table 201 at any timing in the first state. For example, the snapshot storage unit 217 stores, as first image information, a bitmap of an image displayed in the display area and a handwritten stroke of a drawn image that is superimposed and drawn in association with each other. In the example illustrated in FIG. 5, the snapshot storage unit 217 stores the display screen and the handwritten stroke in association with each other at any timing in the first state. At this time, the representative image ID is “NULL” as an initial value.

  In other words, the snapshot storage unit 217 stores a snapshot of the image displayed in the display area of the display unit 112 in the first state. The snapshot storage unit 217 is also referred to as a “first storage unit”.

  The snapshot storage unit 217 stores one or more pieces of first image information in the first state. For example, the snapshot storage unit 217 stores a snapshot each time a new handwritten stroke is generated by the handwriting processing unit 214. However, the present invention is not limited to this, and the snapshot storage unit 217 may store snapshots at arbitrary timings, for example. To explain with a more detailed example, the snapshot storage unit 217 may store a snapshot every Ts = 1 [s].

  When the mode switching control unit 212 switches from the first state to the second state, the event transfer unit 218 detects the contact position coordinates or the approach position coordinates based on the contact position coordinates or the approach position coordinates. Instruction information for the user PC 130 is generated, and the generated instruction information is output to the user PC 130. The event transfer unit 218 is also referred to as an “output unit”.

  For example, when receiving the coordinates “Xt, Yt” and the coordinates “Xa, Ya” from the video processing unit 215, the event transfer unit 218 calculates the output coordinates “xo, yo” based on (Equation 1). Then, the event transfer unit 218 outputs instruction information indicating that the output coordinates “xo, yo” has been touched, in other words, instruction information indicating that “xo, yo” has been clicked, to the user PC 130. Note that “xo, yo” are coordinates on the user PC 130 corresponding to the detected coordinates.

  (Xo, yo) = (Xt−Xa, Yt−Ya) (Equation 1)

  Here, the event transfer unit 218 generates different instruction information depending on how to touch the touch panel 113. For example, when the touch panel 113 is touched once by the user, the event transfer unit 218 generates instruction information indicating a mouse event that clicks once on the output coordinates corresponding to the touched location. For example, when the touch panel 113 is touched twice by the user, the event transfer unit 218 generates instruction information indicating a mouse event that double-clicks an output coordinate corresponding to the touched location. Also, for example, when the event transfer unit 218 is touched once and then moved while being touched to another place as it is and then released, the event transfer unit 218 determines from the first touched coordinate to the coordinate where the touch is released. Instruction information indicating a mouse event to be dragged is generated.

  Here, as to whether the touch panel 113 is touched once, touched twice, or touched once and then moved to another location while being touched and released, the event transfer unit 218 May be determined, or the touch panel control unit 213 or the video processing unit 215 may determine.

  When the contact position coordinates or the approach position coordinates are detected in the second state, the representative image storage unit 219 snaps the second image information that is information for reproducing the image displayed in the display area. The image data is stored in association with the first image information stored by the shot storage unit 217. The representative image storage unit 219 is also referred to as a “second storage unit”.

  Specifically, the representative image storage unit 219 stores the second image information when the instruction information is transmitted from the event transfer unit 218 in the second state. For example, the representative image storage unit 219 stores the second image information that is information for reproducing the screen displayed in the display area after the instruction information is transmitted by the event transfer unit 218 in the second state. . Note that the representative image storage unit 219, for example, captures a screen displayed in the display area, stores a bitmap of the captured screen as second image information, or uses the second image information using an arbitrary format. Or store.

  The point stored in association with the first image information will be further described. The representative image storage unit 219 stores the second image information in the second state, and one or more first images stored by the snapshot storage unit 217 in the first state immediately before the second state. The second image information is stored in association with the information.

  For example, when storing new second image information, the representative image storage unit 219 stores the second image information in the representative image table 202 after giving a new representative image ID. For example, the representative image storage unit 219 initializes the representative image ID with “1” when the snapshot table 201 is generated, and then increases the representative image ID by “1” each time it is stored. Is granted.

  In addition, the representative image storage unit 219 uses the newly assigned representative image ID as the first image information that is not associated with the second image information among the first image information stored in the snapshot table 201. Is stored in association with the second image information.

  FIG. 9 is a diagram illustrating an example of information stored in the snapshot table after the representative image ID is stored by the representative image storage unit according to the first embodiment. FIG. 9 shows an example after the representative image storage unit 219 stores the representative image ID in the snapshot table 201 shown in FIG. As shown in FIGS. 5 and 9, the representative image storage unit 219 stores the newly assigned representative image ID “101” for the location where the representative image ID is “NULL” in FIG. 5. Thus, the newly stored second image information is associated with one or a plurality of first image information stored by the snapshot storage unit 217 in the first state immediately before the second state.

  In other words, when the instruction information is transmitted from the event transfer unit 218, the representative image storage unit 219 acquires the display screen displayed in the display area as an image. The representative image storage unit 219 acquires the new representative image ID, stores the display screen together with the acquired new representative image ID in the representative image table 202, and the representative image ID is stored in the snapshot table 201. A record whose column has a NULL value is acquired, and the record is updated by overwriting the NULL value with the acquired representative image ID.

[Processing in image display device]
Hereinafter, a series of processing for storing the second image information will be described as an example of processing by the image display device 110. FIG. 10 is a sequence diagram illustrating an example of a series of processes for storing the second image information according to the first embodiment.

  As shown in FIG. 10, in the image display device 110, when the video processing unit 215 receives touch event information (Yes at Step S101), it transmits the touch event information and its own coordinates (Step S102). That is, when the video processing unit 215 receives the contact position coordinates or the approach position coordinates from the touch panel control unit 213 in the second state, the video processing unit 215 adds the coordinates of the display area of the display unit 112 on which the video information is displayed. To the event transfer unit 218.

  In the image display device 110, when the event transfer unit 218 receives the touch event information (step S103), in other words, the display area of the display unit 112 in which the contact position coordinates or the approach position coordinates and the video information are displayed. Are received from the video processing unit 215, output coordinates are calculated (step S104). For example, the event transfer unit 218 calculates the output coordinates according to (Equation 1).

  Then, the event transfer unit 218 generates a mouse event based on the event type and output coordinates of the touch event information (Step S105), and transmits the generated mouse event to the user PC 130 (Step S106). Thereafter, the event transfer unit 218 notifies the representative image storage unit 219 of completion of event transmission (step S107). That is, when the contact position coordinates or the approach position coordinates are detected, the event transfer unit 218 generates instruction information for the user PC 130 based on the contact position coordinates or the approach position coordinates, and outputs the generated instruction information to the user PC 130. . In addition, the event transfer unit 218 outputs the output to the representative image storage unit 219.

  In the image display device 110, when the representative image storage unit 219 receives the event transmission completion notification (step S108), the display screen displayed in the display area is acquired as an image (step S109). Then, the representative image storage unit 219 acquires a new representative image ID, and stores the display screen together with the acquired new representative image ID in the representative image table 202 (step S110). Then, a record whose representative image ID string is a NULL value is acquired, and the record is updated by overwriting the NULL value with the acquired representative image ID (step S111).

[Effect of the first embodiment]
As described above, according to the first embodiment, the image display apparatus 110 receives video information from the user PC 130 and displays the received video information in the display area. In addition, the image display device 110 detects the contact position coordinates or the approach position coordinates on the display area, and displays the rendered image drawn based on the detected contact position coordinates or the approach position coordinates on the display area. Superimpose on information. In addition, the image display device 110 has a first state in which a drawing image is superimposed and drawn when a contact position coordinate or an approach position coordinate is detected; The second state is not switched. Further, in the first state, the image display device 110 stores, in the snapshot table 201, first image information that is information for reproducing an image displayed in the display area at any timing. In addition, when the contact position coordinates or the approach position coordinates are detected in the second state, the image display device 110 displays second image information that is information for reproducing the image displayed in the display area, Stored in association with the first image information stored in the snapshot table 201. As a result, the past display screen can be browsed efficiently. Specifically, as a result of easily associating a representative image with a series of snapshots in which handwritten scroll is reflected, information stored as snapshots can be browsed efficiently.

  Further, according to the first embodiment, when the image display device 110 is switched from the first state to the second state and the contact position coordinates or the approach position coordinates are detected by the detection unit, the contact is detected. Instruction information for the user PC 130 is generated based on the position coordinates or the approach position coordinates, and the generated instruction information is output to the user PC 130. Further, the image display device 110 stores the second image information when the instruction information is transmitted in the second state.

  Further, according to the first embodiment, the image display apparatus 110 detects the contact position coordinates or the approach position coordinates on the touch panel provided on the display area. As a result, for example, the coordinates touched by the user or the coordinates approached can be reliably detected.

  In addition, according to the first embodiment, the image display device 110 alternately switches between the first state and the second state in accordance with a user instruction, and in the first state, one or more first One or a plurality of first images stored in the first state immediately before the second state and the second image information stored in the first state immediately before the second state. Store in association with information. As a result, the first image stored in the immediately preceding first state is reliably associated with the second image information in the next second state, so that past display screens can be efficiently browsed.

  That is, for example, when receiving image information from the user PC 130, the image display apparatus 110 displays the received image information in the display area. In the image display device 110, handwriting can be performed on the entire display screen. In addition, when the display area is touched in the second state, the image display device 110 transmits a mouse event to the user PC 130 and stores the display screen as a representative image in association with another snapshot.

  Thus, for example, in addition to storing the snapshot in the first state, the representative image is stored and associated in the representative image table 202 in the second state in which the drawing image drawn by the user is not superimposed and drawn. By doing so, it is possible to browse the representative image and the snapshot side by side, and the past display screen can be viewed more efficiently compared to browsing the snapshot stored in the first state one by one. It becomes possible.

  Here, it will be further described that the information stored as the snapshot can be efficiently browsed by associating the snapshot representative image. FIG. 11 and FIG. 12 are diagrams showing a point where the representative image and the snapshot are browsed side by side. In the example shown in FIG. 11 and FIG. 12, a representative image display lane 330 that is an area for displaying a representative image in time series, and a snap associated with the representative image displayed in the representative image display lane 330. The case where the related snapshot display lane 340 for displaying shots in chronological order is displayed. In the example shown in FIG. 11 and FIG. 12, among the representative images or snapshots stored along the time series, a button 331 for displaying a representative image or snapshot located on the past side in the time series, 341, buttons 332 and 342 for displaying a representative image or snapshot positioned on the future side in time series are also shown. In other words, the buttons 33 and 341 for reversely feeding the representative image and the snapshot are shown together, and the buttons 332 and 342 for sequentially feeding the representative image and the snapshot are also shown.

  In the example illustrated in FIG. 11, snapshots associated with the representative image 333 are displayed in time series together with the representative image 333 displayed in the representative image display lane 330. As a result, the past display screen can be browsed efficiently. In the example illustrated in FIG. 11, a snapshot is stored for each handwritten stroke, and a plurality of snapshots 343 associated with the representative image 333 are displayed in the related snapshot display lane 340 in units of handwritten strokes. .

  In the example illustrated in FIG. 12, the representative image displayed in the representative image display lane 330 has been forwarded by one from the representative image 333 to become the representative image 334. In this case, the snapshot displayed in the related snapshot display lane 340 is a snapshot 344 associated with the representative image 334.

(Second Embodiment)
Although the embodiments of the present invention have been described so far, the present invention may be implemented in other embodiments besides the above-described embodiments. Therefore, other embodiments will be described below.

[Browsing process of first image information and second image information]
For example, when the image display device 110 receives an instruction from the user, the first image information and the second image information are acquired from the representative image table 202 and the snapshot table 201 and displayed in the display area of the display unit 112. Also good. That is, when the image display device 110 receives an instruction to display information stored in the storage device, the second image information stored in the representative image storage unit 219 is acquired and reproduced by the second image information. An image information display control unit that displays the image to be reproduced along the time series and also displays the image reproduced by the first image information associated with the second image information along the time series. May be.

[Representative image ID]
Further, for example, in the above-described embodiment, when the second image information is stored after the first image information is stored, the representative image ID indicating the newly stored second image information is set to the first image ID. The case where the second image information is associated with the first image information by associating with the image information is shown. However, it is not limited to this. For example, when the first image information is stored, the second image information is associated with the representative image ID indicating the second image information stored last among the already stored second image information. The information may be associated with the first image information.

[System configuration]
Also, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or the processes described as being performed manually can be performed. All or a part can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, information including various data and parameters (FIGS. 1 to 12) shown in the above-mentioned document and drawings are arbitrarily changed unless otherwise specified. be able to.

  Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or a part of the distribution / integration may be functionally or physically distributed in arbitrary units according to various loads or usage conditions. Can be integrated and configured. For example, in the example illustrated in FIG. 4, the storage 125 may be a separate device from the image display device 110 or may be integrated with the user PC 130.

  Further, for example, in the above-described embodiment, the case where the image display device 110 includes the display unit 112 including a display area has been described, but the present invention is not limited to this. For example, the image display device 110 may not have the display unit 112. In this case, for example, the image display device 110 displays an image on a display area provided in another device via a network or directly connected, or irradiates an image using a projector or the like. An image is displayed in the display area. When an image is irradiated using a projector or the like, a normal blackboard, an arbitrary wall surface, a screen, or the like on which the image is irradiated is a display area.

[Others]
Note that the image display program executed by the image display apparatus 110 of the present embodiment is provided by being incorporated in advance in a ROM or the like. The image display program executed by the image display apparatus 110 according to the present embodiment is a file in an installable format or an executable format, and is a computer such as a CD-ROM, a flexible disk (FD), a CD-R, or a DVD. You may comprise so that it may record and provide on a readable recording medium.

  Furthermore, the image display program executed by the image display apparatus 110 of the present embodiment may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. The image display program executed in the present embodiment may be provided or distributed via a network such as the Internet.

  The image display program executed by the image display device 110 according to the present embodiment includes the above-described units (for example, the video acquisition unit 211, the mode switching control unit 212, the touch panel control unit 213, the handwriting processing unit 214, the video processing unit 215, and the display). The module configuration includes a control unit 216, a snapshot storage unit 217, an event transfer unit 218, and a representative image storage unit 219). As actual hardware, a CPU (processor) reads an image display program from the ROM. By executing, the above-described units are loaded on the main storage device, and the video acquisition unit 211, the mode switching control unit 212, the touch panel control unit 213, the handwriting processing unit 214, the video processing unit 215, the display control unit 216, and the snapshot storage Unit 217, event transfer unit 218, and representative image storage unit 219 are main memory It is generated in 置上.

100 image display system 110 image display device 112 display unit 113 touch panel 114 operation button 116 operation button 120 video information interface 122 display image interface 123 cable 124 cable 125 storage 126 controller unit 127 external interface 150 processor 153 hard disk device 154 graphics board 155 display 156 Detection Device 157 Network Interface 158 External Device Connection Interface 201 Snapshot Table 202 Representative Image Table 211 Video Acquisition Unit 212 Mode Switching Control Unit 213 Touch Panel Control Unit 214 Handwriting Processing Unit 215 Video Processing Unit 216 Display Control Unit 217 Snapshot Storage Unit 218 Event transfer unit 219 Representative image storage unit 301 Stroke 310 Application Window 330 Representative Image Display Lane 331 Button 332 Button 333 Representative Image 334 Representative Image 340 Related Snapshot Display Lane 343 Representative Image 344 Snapshot

JP 2008-33393 A

Claims (3)

A display control unit for displaying video information received from the video information output device in a display area;
A superimposing drawing unit for superposing and drawing a drawing image on the video information;
A switching control unit that switches between a first state in which the drawing image is superimposed and a second state in which the drawing image is not drawn;
When the switching control unit switches to the first state, the first image information is stored, and when the switching control unit switches to the second state, the second image information is stored in the first image. An image display device characterized by being stored in association with information. An image display method executed by an image display device,
A display control step of displaying video information received from the video information output device in a display area;
A superimposing drawing step of superposing and drawing a drawing image on the video information;
A switching control step for switching between a first state in which the drawn image is drawn and a second state in which the drawn image is not drawn;
When the switching control unit switches to the first state, the first image information is stored, and when the switching control unit switches to the second state, the second image information is stored in the first image. A storage step of storing the information in association with the information. A display control step for displaying video information received from the video information output device in a display area;
A superimposing drawing step of superposing and drawing a drawing image on the video information;
A switching control step for switching between a first state in which the drawn image is drawn in a superimposed manner and a second state in which the drawn image is not drawn;
When the switching control unit switches to the first state, the first image information is stored, and when the switching control unit switches to the second state, the second image information is stored in the first image. A storing step for storing the information in association with the information;
Display program for causing a computer to execute.

Images