WO2016006294A1 - Image display apparatus, image processing system, and image processing method - Google Patents

Image display apparatus, image processing system, and image processing method Download PDF

Info

Publication number
WO2016006294A1
WO2016006294A1 PCT/JP2015/061290 JP2015061290W WO2016006294A1 WO 2016006294 A1 WO2016006294 A1 WO 2016006294A1 JP 2015061290 W JP2015061290 W JP 2015061290W WO 2016006294 A1 WO2016006294 A1 WO 2016006294A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
image display
electronic information
information board
display device
Prior art date
Application number
PCT/JP2015/061290
Other languages
French (fr)
Japanese (ja)
Inventor
基倫 江森
Original Assignee
株式会社リコー
基倫 江森
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社リコー, 基倫 江森 filed Critical 株式会社リコー
Publication of WO2016006294A1 publication Critical patent/WO2016006294A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F13/00—Interconnection of, or transfer of information or other signals between, memories, input/output devices or central processing units
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures

Definitions

  • the present invention relates to an image display device, an image processing system, and an image processing method.
  • a so-called “electronic information board” product that has a touch panel mounted on a large display with a size of about 40 inches to 80 inches using a flat panel or projector such as liquid crystal or plasma is on the market.
  • a typical appearance of the electronic information board is shown in FIG.
  • Typical functions using a touch panel in these products include a personal computer operation function and a handwriting function.
  • the personal computer operation function is to operate the personal computer displaying the screen by directly touching the projected screen instead of the mouse operation using the built-in touch panel function.
  • the handwriting function is a method for inputting characters and the like by handwriting through a touch panel on the screen serving as a blackboard, taking a personal computer screen and superimposing handwriting on the screen.
  • This function is realized by electronic blackboard application software provided with the electronic information board.
  • An important thing in a conference using an electronic information board having such a function is that it is possible to display and input necessary information by operating the electronic information board without disturbing the progress of the conference.
  • the electronic information board since the embedded software of the electronic information board is operated, the electronic information board requires a software processing capability, resulting in a problem that the cost is increased.
  • each electronic information board had to hold handwritten data and read file data, and thus there was a problem that a large network bandwidth was required to share it.
  • Japanese Patent Application Laid-Open No. H10-228561 acquires pen strokes written on a physical whiteboard as an image for the purpose of sharing writing on the whiteboard, and obtains pen stroke attribute information therefrom.
  • a system for extracting and drawing pen strokes on another digital writing surface is disclosed.
  • Patent Document 1 since the invention described in Patent Document 1 enables stroke information to be digitized and transmitted, the stroke information can be shared by a plurality of devices.
  • the processing resources for operating the embedded software are required on the whiteboard side, which is the terminal side, the above-mentioned electronic information board requires software processing capability, which results in an increase in cost. It is not solved.
  • the present invention has been made in view of the above, and an object of the present invention is to provide an image display device, an image processing system, and an image processing method capable of performing handwritten editing without delay without executing a heavy processing.
  • An image display apparatus includes an image display means for displaying an image, and an operation means for performing an operation for drawing and editing a stroke on the image displayed by the image display means.
  • An operation state display unit that displays an operation state of the operation unit, an operation event transmission unit that transmits an operation event generated by the operation unit, and a moving image that receives a moving image indicating a stroke based on the operation event It has a receiving means and a moving image display means for displaying the moving image received by the moving image receiving means.
  • FIG. 1 is a diagram illustrating an overall configuration of an image processing system according to an embodiment.
  • FIG. 2 is a block diagram illustrating a configuration of the electronic information board and the server according to the embodiment.
  • FIG. 3 is a flowchart illustrating operations of the electronic information board and the server according to the embodiment.
  • FIG. 4 is a diagram showing the configuration of the layers of the electronic information board.
  • FIG. 5 is a diagram showing an outline of video distribution.
  • FIG. 6 is a diagram illustrating an operation when changing the operation mode of the image processing system according to the embodiment.
  • FIG. 7 is a diagram illustrating an example of a list of commands that can be used in the electronic information board.
  • FIG. 8 is a diagram illustrating an operation when a mouse event is transmitted.
  • FIG. 8 is a diagram illustrating an operation when a mouse event is transmitted.
  • FIG. 9 is a diagram showing an operation when the delete command on the operation panel is selected.
  • FIG. 10A is a diagram showing a screen before erasing the electronic information board.
  • FIG. 10B is a diagram showing a screen during erasure of the electronic information board.
  • FIG. 11 is a flowchart showing an operation at the time of erasing the electronic information board.
  • FIG. 12 is another example of a flowchart showing an operation at the time of erasing the electronic information board.
  • FIG. 13 is a diagram illustrating an operation at the time of background image distribution of the image processing system.
  • FIG. 14 is a flowchart showing an operation at the time of background image distribution of the image processing system.
  • FIG. 15A is a diagram showing a screen before erasing the electronic information board.
  • FIG. 15B is a diagram showing a screen during erasure of the electronic information board.
  • FIG. 16 is a flowchart showing an operation at the time of selective erasure of the electronic information board.
  • FIG. 17A is a diagram showing a screen before erasing the electronic information board.
  • FIG. 17B is a diagram showing a screen during erasure of the electronic information board.
  • FIG. 17C is a diagram showing a screen after the electronic information board is erased.
  • FIG. 18 is a flowchart showing the operation at the time of erasing the stroke unit of the electronic information board.
  • FIG. 19A is a diagram showing a state before selection by the lasso of the electronic information board.
  • FIG. 19B is a diagram showing a state during selection by a lasso of the electronic information board.
  • FIG. 19C is a diagram illustrating a state in which the electronic information board is being selected by a lasso.
  • FIG. 19D is a diagram showing a state after selection by the lasso of the electronic information board.
  • FIG. 20 is a flowchart showing the selection operation by the lasso of the electronic information board.
  • FIG. 21A is a diagram showing a state before selection by the lasso of the electronic information board.
  • FIG. 21B is a diagram showing a state during selection by a lasso of the electronic information board.
  • FIG. 21C is a diagram illustrating a state in which the electronic information board is being selected by a lasso.
  • FIG. 22 is a flowchart showing another selection operation of the electronic information board.
  • FIG. 23A is a diagram illustrating a state in which the selection area of the electronic information board is being moved.
  • FIG. 23B is a diagram showing a state after the selection area of the electronic information board is moved.
  • FIG. 24 is a flowchart showing an operation of moving the selection area of the electronic information board.
  • FIG. 25 is a diagram illustrating a state immediately after the selection area of the electronic information board is moved.
  • FIG. 26 is a flowchart showing the operation immediately after the selection area of the electronic information board is moved.
  • FIG. 27A is a diagram illustrating a state before the electronic information board is enlarged.
  • FIG. 27B is a diagram showing a state after the enlargement operation of the electronic information board.
  • FIG. 28 is a flowchart showing the operation of enlarging the electronic information board.
  • FIG. 29 is a diagram illustrating a state immediately after the electronic information board is enlarged.
  • FIG. 30 is a flowchart showing an operation immediately after enlargement / reduction of the electronic information board.
  • FIG. 31A is a diagram illustrating a state of the electronic information board before zooming.
  • FIG. 31B is a diagram showing how to set the zoom magnification of the electronic information board.
  • FIG. 31C is a diagram illustrating a result of zooming the electronic information board.
  • FIG. 31D is a diagram illustrating an area where an image of the electronic information board cannot be displayed.
  • FIG. 32 is a flowchart showing the zoom operation of the electronic information board.
  • FIG. 33 is a diagram showing another zoom operation of the electronic information board.
  • FIG. 31A is a diagram illustrating a state of the electronic information board before zooming.
  • FIG. 31B is a diagram showing how to set the zoom magnification of the electronic information board.
  • FIG. 31C is a diagram illustrating a result of zoom
  • FIG. 34 is a flowchart showing another zoom operation of the electronic information board.
  • FIG. 35A is a diagram showing a state before the pan operation of the electronic information board.
  • FIG. 35B is a diagram illustrating a state during a pan operation of the electronic information board.
  • FIG. 35C is a diagram illustrating a state after the pan operation of the electronic information board.
  • FIG. 36 is a flowchart showing an operation of panning the electronic information board.
  • FIG. 37 is a flowchart showing the panning operation of the electronic information board.
  • FIG. 38A is a diagram showing a state before the pan operation of the electronic information board.
  • FIG. 38B is a diagram illustrating a state during a pan operation of the electronic information board.
  • FIG. 38C is a diagram illustrating a state after the pan operation of the electronic information board.
  • FIG. 39 is a flowchart showing another panning operation of the electronic information board.
  • FIG. 1 is a diagram illustrating an overall configuration of an image processing system according to an embodiment.
  • the electronic information board 1 is used as an image display device
  • the server 2 functions as an image processing device
  • the electronic information board 1 and the server 2 are connected by a network.
  • FIG. 2 is a block diagram illustrating a configuration of the electronic information board and the server according to the embodiment.
  • the electronic information board 1 has a touch panel display 101.
  • the touch panel display 101 functions as an image display unit.
  • the operation means may be touching the touch panel display 101 or may be input using a device such as a touch pen or a mouse. Therefore, the touch panel display 101 may function as an operation unit.
  • the electronic information board 1 includes an operation event receiving unit 103 that receives an operation event that has occurred on the touch panel display 101, and an operation event transmitting unit 102 that transmits the received operation event to the server 2.
  • the operation event transmission unit 102 functions as a transmission unit.
  • the stroke editing state display unit 105 has a stroke editing interface and a stroke editing state display unit 105 for displaying a state during editing by the received operation event.
  • the stroke editing state display unit 105 is also referred to as local feedback in the following description and functions as an operation state display unit.
  • the event receiving unit 104 that receives an event transmitted from the stroke display / editing unit 201 of the server 2 via the event transmitting unit 204, the video receiving unit 106 that receives video from the server 2, and the video that displays the received video A display unit 107 is included.
  • the video receiving unit 106 functions as a moving image receiving unit
  • the video display unit 107 functions as a moving image display unit.
  • the server 2 includes an operation event receiving unit 203 that functions as an event receiving unit that receives an operation event, and an operation event queue 202 for temporarily storing the operation event. Further, it has a stroke display / editing unit 201 that takes out an operation event from the operation event queue and displays / edits the stroke.
  • the server 2 includes an event transmission unit 204 that transmits an event generated during stroke editing to the electronic information board, and a background video display unit 205 that displays a background video.
  • it includes a video composition unit 206 that synthesizes the background video and the stroke video to generate a video, and a video transmission unit 207 that distributes the generated video to the electronic information board.
  • the video composition unit 206 functions as a moving image creation unit
  • the video transmission unit 207 functions as a moving image transmission unit.
  • FIG. 3 is a sequence diagram illustrating operations of the electronic information board and the server according to the embodiment. The subject of each sequence and the target constituent elements will be described using the names in FIG. First, when a stroke is drawn on the electronic information board 1 by touch or the like, a mouse event is generated (step S101).
  • the operation state is displayed on the screen by the stroke editing state display unit 105 (step S103), and the mouse event is transmitted to the server 2 by the operation event transmission unit 102 (step S102).
  • the server 2 draws a stroke with the stroke drawing application of the stroke display / editing unit 201 (step S201).
  • the application image is encoded with the video composition unit 206 (step S202), and the video transmission unit 207 distributes the application screen to the electronic information board 1 (step S203).
  • the electronic information board 1 receives the moving image by the video receiving unit 106, and displays the received moving image on the screen by the video display unit 107 (step S104).
  • the “moving image” in the description represents a general moving image in which images are continuously displayed.
  • a video encoded in a specific format such as H.264 is defined as “video”.
  • the moving image distributed from the server 2 to the electronic information board is particularly called a video.
  • FIG. 4 is a diagram showing a layer configuration of the electronic information board.
  • the electronic information board 1 includes an operation panel display layer 101c, an interface for stroke editing, a local feedback layer 101b for displaying a state during editing, and a video display layer 101a for displaying video distributed from a server.
  • FIG. 5 is a diagram showing an outline of video distribution.
  • the server 2 combines the stroke and the background, encodes it into a moving image, and distributes it to the electronic information board.
  • the moving image is decoded and displayed as a video.
  • FIG. 6 is a diagram illustrating an operation when changing the operation mode of the image processing system according to the embodiment.
  • a button on the operation panel of the electronic information board 1 is pressed, a command is transmitted to the server 2 as an operation event.
  • the button is pressed, the mode is immediately changed on the electronic information board.
  • the mode is changed after receiving the command.
  • the operation panel indicates a list of commands displayed on the left side of the operation panel display layer 101c in FIG.
  • FIG. 7 is a diagram showing an example of a list of commands that can be used in the electronic information board. As shown in FIG. 7, there are a pen mode, an erase mode, a selection mode, a zoom mode, and the like. Each command can send necessary information to the server as an argument.
  • FIG. 8 is a diagram showing an operation when a mouse event is transmitted.
  • Mouse events include coordinate information.
  • mouse move event event transmitted to the server 2 side is once saved in the queue. This is because if a large number of mouse move event events are delivered to the stroke editing application at once, the processing of the application may not catch up.
  • the operations to be described indicate local feedback operations unless otherwise specified.
  • the video distributed from the server side basically includes only an image obtained as a result of the operation. However, for the purpose of debugging or demonstration, the same operation as the local feedback may be included in the video distributed from the server side.
  • the video indicates a video distributed from the server and is displayed on the video display layer 101a in FIG.
  • the stroke indicated by the local feedback on the terminal side and the video created and distributed on the server side do not exactly match.
  • the local feedback is displayed as a multiline that connects the coordinates of the mouse movement with a straight line, but on the server side, smoothing is performed based on the coordinates of the mouse movement at the time of drawing. Create a curved line. This is to reduce the load on the terminal side by performing a process with a large load called smoothing on the server side.
  • FIG. 9 is a diagram showing an operation when the delete icon on the operation panel is selected. Press the erase icon to select the erase range. For example, a stroke drawn with a size selected from four types of ranges can be deleted.
  • FIG. 10A and FIG. 10B are diagrams showing an operation at the time of erasing the electronic information board.
  • FIG. 10A shows a screen before erasing the electronic information board
  • FIG. 10B shows a screen during erasing of the electronic information board.
  • the blackboard erase icon 502 when operated, the area 501 where the icon has been swept is filled with the size of the selected erase range, and appears to have been erased. The fill is faded out after a certain period of time.
  • the fixed time is a delay time + ⁇ caused by video distribution.
  • the blackboard erase icon 502 when operated roughly, an unerased portion as indicated by 503 often remains. Therefore, when the erased stroke becomes a predetermined length or less, or when the length becomes a predetermined ratio or less, the unerased portion may be erased.
  • FIG. 11 is a flowchart showing the operation at the time of erasing the electronic information board.
  • the area swept by the blackboard eraser icon 502 is painted with a specific color.
  • white that is adopted as the background color of the whiteboard can be considered easily.
  • a PC image or the like may be displayed below the stroke on the electronic information board 1, if it is simply painted in white, it gives the impression that the background has disappeared.
  • FIG. 12 is another example of a flowchart showing an operation at the time of erasing the electronic information board.
  • FIG. 12 shows an erase operation by a method different from that in FIG. In the erasure shown in FIG. 11, the specific color is used for painting. However, since the color of the image differs depending on the position, it is not effective to fill it with one color. In view of this, it is possible to give a more effective erasing sensation by displaying the area swept by the blackboard eraser icon 502 by blurring the image behind it.
  • the video is captured, the captured image is once reduced, and the image is blurred.
  • a blurring filter a mosaic, a Gauntian filter, etc. are well known. After applying the blur filter, return it to its original size. By performing the blurring process on the reduced image, the processing load is reduced.
  • an image A that is an empty image of the same size is created for post-processing.
  • the periphery of the coordinates of the blackboard erase icon 502 is cut out from the blurred image and pasted on the video.
  • the clipped image is pasted on the image A for post-processing.
  • the image pasted on the video is deleted and the image A is pasted instead. Then, the image A is faded out after a predetermined time has elapsed.
  • FIG. 13 is a diagram illustrating an operation when the background image of the image processing system is distributed.
  • the captured image is blurred and pasted on the cleaned area of the blackboard erase icon 502. This occurs because a video in which a stroke and a background image are combined is distributed.
  • FIG. 14 is a flowchart illustrating an operation when a background image is distributed in the image processing system.
  • FIG. 15A and FIG. 15B are diagrams showing the operation of selectively erasing the electronic information board.
  • FIG. 15A shows a screen display before the electronic information board is erased.
  • FIG. 15B shows the display of the screen while the electronic information board is being erased.
  • the selective erasing is a function for erasing a stroke in an area surrounded by a lasso.
  • FIG. 15B is a flowchart showing the operation of selectively erasing the electronic information board.
  • FIG. 17A, FIG. 17B, and FIG. 17C are diagrams showing the operation of deleting the stroke unit of the electronic information board.
  • FIG. 17A shows a screen before the electronic information board is erased.
  • FIG. 17B shows a screen during erasing of the electronic information board.
  • FIG. 17C shows a screen after the electronic information board is erased.
  • the stroke unit erasing is a method of erasing a stroke that intersects or touches the locus of the eraser icon 511 for each stroke.
  • erasing the stroke unit erases the entire stroke just by touching the locus, so that the stroke cannot be filled with white like the above-described erasure so as not to feel the delay. Therefore, as shown in FIG. 17B, the locus of the eraser and the afterimage of the eraser are displayed to reduce the stress that does not disappear immediately.
  • the eraser's locus is painted white in the figure, but it is drawn with a dotted line, or it is painted with a blurred image of the image, or the background image is distributed separately and painted using it, as in the normal erase operation described above. Also good.
  • the afterimage of the eraser is configured to be deleted in order from the previously displayed icon. The number of afterimages to be displayed and the time until the deletion is started may be proportional to the delay time.
  • FIG. 18 is a flowchart showing the operation of deleting the stroke unit of the electronic information board.
  • FIG. 19A, FIG. 19B, FIG. 19C, and FIG. 19D are diagrams showing the selection operation by the lasso of the electronic information board.
  • FIG. 19A shows a state before selection by the lasso of the electronic information board. When you start selecting with a lasso, the selection results up to the middle are displayed.
  • FIG. 19B and FIG. 19C show a state where the electronic information board is being selected by a lasso. Since the selection is in progress, the displayed rectangle is an undetermined selection area.
  • FIG. 19D shows a state after selection by the lasso of the electronic information board.
  • the coordinates and size are required to draw the indeterminate selection area, but pen move events are sent to the server one by one and returned as a response.
  • the size of the undetermined selection region changes between FIG. 19B and FIG. 19C, the change in size may be animated.
  • a pen-up event is transmitted to the server, and when the result is returned, the selection is confirmed, and as shown in FIG. Enlargement / reduction knobs are displayed in the four corners. Further, the color of the frame of the confirmed selection region is darker than the color of the unconfirmed selection region. For example, the color of the confirmed selection region is gray, and the color of the unconfirmed selection region is light gray.
  • the unconfirmed selection area is configured so that it can be moved by pen operation. As a result, the moving operation can be performed even in a state where the selection is not confirmed, so that it is not necessary to wait for the user until the selection is confirmed, and stress due to delay can be reduced.
  • FIG. 20 is a flowchart showing the selection operation by the lasso of the electronic information board.
  • FIG. 21A, FIG. 21B, and FIG. 21C are diagrams showing the operation of another selection method of the electronic information board. In this case, the unconfirmed selection area as described above is not displayed even if the lasso is selected.
  • FIG. 21A is a diagram showing a state before selection by the lasso of the electronic information board.
  • FIG. 21B is a diagram showing a state during selection by a lasso of the electronic information board. As shown in FIG. 21B, after pen-up, the lasso bounding box is displayed as the unconfirmed selection area.
  • FIG. 21C is a diagram showing a state after selection by the lasso of the electronic information board.
  • FIG. 22 is a flowchart showing another selection operation of the electronic information board.
  • FIG. 23A and FIG. 23B are diagrams showing an operation of moving the electronic information board.
  • FIG. 23A is a diagram showing a state in which the selected area of the electronic information boat is moving.
  • FIG. 23B is a diagram showing a state after the selection area of the electronic information board is moved.
  • the stroke can be moved by dragging the selection area with a pen.
  • a guide frame is displayed during movement. Only the guide frame may be moved, or the selected area portion of the captured image of the video may be pasted on the guide frame. Make it translucent because it is moving.
  • FIG. 24 is a flowchart showing an operation of moving the selection area of the electronic information board.
  • FIG. 25 is a diagram showing a state immediately after the selection area of the electronic information board is moved. As shown in FIG. 25, the selected area has moved, but due to the delay, the characters on the video have not moved yet. The problem is that the characters on the video remain in their original positions, causing the user to wait until the characters move on the video.
  • FIG. 26 is a flowchart showing the operation immediately after the selection area of the electronic information board is moved.
  • FIG. 27A and FIG. 27B are diagrams showing the operation of enlarging the electronic information board.
  • FIG. 27A is a diagram illustrating a state before the electronic information board is enlarged.
  • FIG. 27B is a diagram showing a state after the enlargement operation of the electronic information board.
  • the size of the stroke that is, enlargement / reduction can be changed by dragging the knobs at the four corners of the selection area with a pen.
  • the basic operation is the same as moving. For example, after performing the enlargement operation, the state shown in FIG. 27B is obtained.
  • FIG. 28 is a flowchart showing the operation of enlarging the electronic information board.
  • ⁇ ⁇ ⁇ ⁇ Enlarge / reduce the selected area by dragging the knob with the pen.
  • a guide frame is displayed while changing the size. Only the guide frame may be resized, or a selection area portion of a captured video image may be pasted on the guide frame. Because the size is changing, make it translucent.
  • FIG. 29 is a diagram showing a state immediately after the electronic information board is expanded. As shown in FIG. 29, as in the case of movement, even with enlargement / reduction, a delay occurs until the image is enlarged / reduced. This problem can be solved by a method similar to that of the movement shown in FIG.
  • FIG. 30 is a flowchart showing an operation immediately after enlargement / reduction of the electronic information board.
  • FIG. 31A, FIG. 31B, FIG. 31C, and FIG. 31D are diagrams showing the zoom operation of the electronic information board.
  • FIG. 31A shows a state of the electronic information board before zooming
  • FIG. 31B shows a state where the zoom magnification of the electronic information board is set.
  • To zoom the screen first capture the video and zoom it. Fade out the captured image after zooming.
  • FIG. 31C shows the result of the zoom operation of the electronic information board.
  • FIG. 32 is a flowchart showing the zoom operation of the electronic information board.
  • the zoom rate when the zoom rate is changed from 200% to 100%, the video of 200% is distributed only in the portion displayed on the terminal. Therefore, when the video is captured and the zoom rate is changed, the video is displayed as shown in FIG. 31D. An area that cannot be displayed is generated. This area is filled with gray. In order to show the current zoom rate during zooming, the zoom rate may be displayed at the center of the screen.
  • FIG. 33 is a diagram showing another zoom operation of the electronic information board.
  • a gray portion is visible when the zoom magnification is changed from 200% to 100%.
  • This problem can be solved by distributing full-size video even when enlarged. That is, when the zoom magnification is reduced, a full-size image is displayed in a place where the gray portion is present.
  • FIG. 33 is a flowchart showing another zoom operation of the electronic information board.
  • FIG. 35A, FIG. 35B, and FIG. 35C are diagrams illustrating the panning operation of the electronic information board.
  • FIG. 35A is a diagram showing a state before the pan operation of the electronic information board.
  • FIG. 35B is a diagram illustrating a state during a pan operation of the electronic information board.
  • FIG. 35C is a diagram illustrating a state after the pan operation of the electronic information board.
  • the screen can be panned to view the desired location. Similar to zooming, this can be achieved by capturing and moving the video.
  • the size of the captured image is the same as the screen size of the electronic information board, when panning, as shown in FIG. 35B, the area behind is visible, and a delay occurs until the state of FIG. 35C is reached.
  • FIG. 38A, 38B, and 38C are diagrams illustrating other pan operations of the electronic information board.
  • FIG. 38A is a diagram showing a state before the pan operation of the electronic information board.
  • FIG. 38B is a diagram illustrating a state during a pan operation of the electronic information board.
  • FIG. 38C is a diagram illustrating a state after the pan operation of the electronic information board.
  • FIG. 38B instead of panning the captured image, it may be possible to display a frame line and move it to indicate that it is panning. As shown in FIG. 38C, the frame line is erased after a predetermined time.
  • FIG. 39 is a flowchart showing another panning operation of the electronic information board.
  • the electronic information board 1 that is an image display device and the server 2 that is an image processing device may be connected via a network to form an image processing system.
  • a program for executing these image processes may be installed in the electronic information board 1 and the server 2 and executed.
  • the image display device is not limited to the electronic information board, and may be a display-integrated personal computer, a tablet terminal, or the like.

Abstract

An image display apparatus according to the present invention is an electronic information board comprising a touch panel display 101 that displays an image; a stroke-editing-state display unit 105 that displays the state at the time when an operation for drawing and editing a stroke on the image displayed on the touch panel display 101 is performed; an operation-event transmission unit 102 that transmits an operation event produced by the operation; a video reception unit 106 that receives a video showing the stroke based on the operating event; and a video display unit 107 that displays the video received by the video reception unit 106.

Description

画像表示装置、画像処理システム、及び画像処理方法Image display device, image processing system, and image processing method
　本発明は、画像表示装置、画像処理システム、及び画像処理方法に関する。 The present invention relates to an image display device, an image processing system, and an image processing method.
　液晶やプラズマなどのフラットパネルやプロジェクタを用いた４０インチ～８０インチ程度のサイズの大型のディスプレイに、タッチパネルを搭載した、いわゆる「電子情報ボード」製品が市場に出ている。電子情報ボードの典型的な外観を図１に示す。 A so-called “electronic information board” product that has a touch panel mounted on a large display with a size of about 40 inches to 80 inches using a flat panel or projector such as liquid crystal or plasma is on the market. A typical appearance of the electronic information board is shown in FIG.
　これらの製品はパソコンを接続することによって、接続したパソコンの画面を大きく映すことができ、企業や行政機関の会議におけるプレゼンテーション等や教育機関等で用いられている。これらの製品におけるタッチパネルを用いた代表的な機能として、パソコン操作機能及び手書き機能がある。 These products, when connected to a personal computer, can display a large screen on the connected personal computer, and are used in presentations at conferences of companies and government agencies and educational institutions. Typical functions using a touch panel in these products include a personal computer operation function and a handwriting function.
　まず、パソコン操作機能とは、搭載するタッチパネル機能を用いて、映されている画面にマウス操作の代わりに直接タッチすることで、画面を表示しているパソコンを操作するものである。 First, the personal computer operation function is to operate the personal computer displaying the screen by directly touching the projected screen instead of the mouse operation using the built-in touch panel function.
　次に、手書き機能とは、黒板の役割をする画面上のタッチパネルを介して文字等を手書きで入力し、また、パソコンの画面を取り込んでその上に手書きを重畳して描くものである。この機能は、電子情報ボードと共に提供される電子黒板アプリケーションソフトウエアによって実現される。 Next, the handwriting function is a method for inputting characters and the like by handwriting through a touch panel on the screen serving as a blackboard, taking a personal computer screen and superimposing handwriting on the screen. This function is realized by electronic blackboard application software provided with the electronic information board.
　具体的な製品事例では、例えばシャープ社の「ＢＩＧ　ＰＡＤ」（登録商標）、日立ソフトウェアエンジニアリング社の「スターボード」（登録商標）や、パイオニア社の「サイバーカンファレンス」（登録商標）、が既に知られている。 Specific product examples include Sharp's “BIG PAD” (registered trademark), Hitachi Software Engineering's “Starboard” (registered trademark), and Pioneer's “Cyber Conference” (registered trademark). It has been.
　このような手書き機能が動作する電子情報ボードを利用すると、オフィスの会議シーン等において、説明のための表示資料を操作しながら、適宜画面上に直接指摘事項等の書き込みを行い、さらに必要に応じ書き込みを含む画面内容を記録することができ、結果、会議終了時に画面内容を振り返り、画面内容を再利用することで効率的に結論などをまとめることが期待できる。 When using an electronic information board with such a handwriting function, you can directly write indications etc. directly on the screen while manipulating display materials for explanation in office meeting scenes, etc. It is possible to record the screen content including writing, and as a result, it is expected that the screen content can be looked back at the end of the conference and conclusions can be efficiently compiled by reusing the screen content.
　このような機能を有する電子情報ボードを利用した会議において重要なことは、会議の進行を妨げずに電子情報ボードが操作されて必要な情報の表示と入力が可能であることである。 An important thing in a conference using an electronic information board having such a function is that it is possible to display and input necessary information by operating the electronic information board without disturbing the progress of the conference.
　しかし、今までの電子情報ボードでは、電子情報ボードの組み込みソフトウェアを動作させるため、電子情報ボードにソフトウェア処理能力が必要とされ、その結果コストが高くなってしまうという問題があった。 However, in the conventional electronic information board, since the embedded software of the electronic information board is operated, the electronic information board requires a software processing capability, resulting in a problem that the cost is increased.
　また、遠隔会議においては、各電子情報ボードが手書きデータや、読み込んだファイルのデータを保持する必要があったため、それを共有するために多くのネットワーク帯域を必要とするという問題があった。 Also, in the remote conference, each electronic information board had to hold handwritten data and read file data, and thus there was a problem that a large network bandwidth was required to share it.
　さらに、現行機種には処理の重い機能を新たに追加したい場合に、端末の交換が必要となるという問題があった。これら問題を避けるために、外部のサーバでソフトウェア処理を実行するという方法が提案されている。この場合には、ネットワーク遅延のために、手書き画像が遅れてしまうという問題があった。 Furthermore, there was a problem that the terminal needs to be replaced when it is desired to add a heavy processing function to the current model. In order to avoid these problems, a method of executing software processing on an external server has been proposed. In this case, there is a problem that the handwritten image is delayed due to network delay.
　この問題を解決するために、端末側で一時的に手書き描画を行い、手書き描画の遅れをユーザに感じさせないようにする技術が提案されている。しかし、サーバ側で描画された手書きを削除、移動、拡大・縮小、等の編集操作をする場合、編集すべき手書きのデータはサーバ側にあるので、手書き描画のときのように、手書きデータをサーバに送信する前に端末で描画するという手法で遅延を感じさせないようにさせることはできない、という問題があった。 In order to solve this problem, a technique has been proposed in which handwritten drawing is temporarily performed on the terminal side so that the user does not feel a delay in handwritten drawing. However, when editing operations such as deleting, moving, enlarging / reducing, etc., handwritten drawn on the server side, the handwritten data to be edited is on the server side. There was a problem that it was not possible to avoid delays by drawing on the terminal before sending to the server.
　これらの問題を解決するために、特許文献１には、ホワイトボードへの書き込みを共有する目的で、物理的なホワイトボードに書き込まれたペンストロークをイメージとして取得し、そこからペンストローク属性情報を抽出し別のデジタル書き込み面にペンストロークを描画するシステムが開示されている。 In order to solve these problems, Japanese Patent Application Laid-Open No. H10-228561 acquires pen strokes written on a physical whiteboard as an image for the purpose of sharing writing on the whiteboard, and obtains pen stroke attribute information therefrom. A system for extracting and drawing pen strokes on another digital writing surface is disclosed.
　しかしながら、特許文献１に記載の発明は、ストロークの情報をデジタル化し送信可能としているため、複数の機器でストローク情報を共有可能である。しかし、端末側であるホワイトボード側に組み込みソフトウェアを動作させるための処理リソースが必要となるため、前述の電子情報ボードにソフトウェア処理能力が必要とされ、その結果コストが高くなってしまうという問題が解決されない。 However, since the invention described in Patent Document 1 enables stroke information to be digitized and transmitted, the stroke information can be shared by a plurality of devices. However, since the processing resources for operating the embedded software are required on the whiteboard side, which is the terminal side, the above-mentioned electronic information board requires software processing capability, which results in an increase in cost. It is not solved.
　本発明は、上記に鑑みてなされたものであって、負担が大きい処理を実行することなく、遅延なく手書き編集が可能な画像表示装置、画像処理システム、及び画像処理方法を提供することを目的とする。 The present invention has been made in view of the above, and an object of the present invention is to provide an image display device, an image processing system, and an image processing method capable of performing handwritten editing without delay without executing a heavy processing. And
　本発明の画像表示装置は、画像を表示する画像表示手段と、前記画像表示手段が表示している画像の上にストロークを描画し編集するための操作を行う操作手段と、を有する画像表示装置であって、前記操作手段による操作の状態を表示する操作状態表示手段と、前記操作手段によって発生する操作イベントを送信する操作イベント送信手段と、前記操作イベントに基づくストロークを示す動画を受信する動画受信手段と、前記動画受信手段によって受信した動画を表示する動画表示手段と、を有することを特徴とする。 An image display apparatus according to the present invention includes an image display means for displaying an image, and an operation means for performing an operation for drawing and editing a stroke on the image displayed by the image display means. An operation state display unit that displays an operation state of the operation unit, an operation event transmission unit that transmits an operation event generated by the operation unit, and a moving image that receives a moving image indicating a stroke based on the operation event It has a receiving means and a moving image display means for displaying the moving image received by the moving image receiving means.
　本発明によれば、負担が大きい処理を実行することなく、遅延なく手書き編集ができるという効果を奏する。 According to the present invention, there is an effect that handwritten editing can be performed without delay without executing a heavy processing.
図１は、実施形態の画像処理システムの全体構成を示す図である。FIG. 1 is a diagram illustrating an overall configuration of an image processing system according to an embodiment. 図２は、実施形態の電子情報ボード及びサーバの構成を示すブロック図である。FIG. 2 is a block diagram illustrating a configuration of the electronic information board and the server according to the embodiment. 図３は、実施形態の電子情報ボード及びサーバの動作を示すフローチャート図である。FIG. 3 is a flowchart illustrating operations of the electronic information board and the server according to the embodiment. 図４は、電子情報ボードのレイヤの構成を示す図である。FIG. 4 is a diagram showing the configuration of the layers of the electronic information board. 図５は、映像配信の概略を示す図である。FIG. 5 is a diagram showing an outline of video distribution. 図６は、実施形態の画像処理システムの操作のモード変更時の動作を示す図である。FIG. 6 is a diagram illustrating an operation when changing the operation mode of the image processing system according to the embodiment. 図７は、電子情報ボードにおいて使用可能なコマンドの一覧の例を示す図である。FIG. 7 is a diagram illustrating an example of a list of commands that can be used in the electronic information board. 図８は、マウスイベントが送信される際の動作を示す図である。FIG. 8 is a diagram illustrating an operation when a mouse event is transmitted. 図９は、操作パネルの消去コマンドを選択した時の動作を示す図である。FIG. 9 is a diagram showing an operation when the delete command on the operation panel is selected. 図１０Ａは、電子情報ボードの消去前の画面を示す図である。FIG. 10A is a diagram showing a screen before erasing the electronic information board. 図１０Ｂは、電子情報ボードの消去中の画面を示す図である。FIG. 10B is a diagram showing a screen during erasure of the electronic information board. 図１１は、電子情報ボードの消去時の動作を示すフローチャートである。FIG. 11 is a flowchart showing an operation at the time of erasing the electronic information board. 図１２は、電子情報ボードの消去時の動作を示すフローチャートの他の例である。FIG. 12 is another example of a flowchart showing an operation at the time of erasing the electronic information board. 図１３は、画像処理システムの背景画像配信時の動作を示す図である。FIG. 13 is a diagram illustrating an operation at the time of background image distribution of the image processing system. 図１４は、画像処理システムの背景画像配信時の動作を示すフローチャートである。FIG. 14 is a flowchart showing an operation at the time of background image distribution of the image processing system. 図１５Ａは、電子情報ボードの消去前の画面を示す図である。FIG. 15A is a diagram showing a screen before erasing the electronic information board. 図１５Ｂは、電子情報ボードの消去中の画面を示す図である。FIG. 15B is a diagram showing a screen during erasure of the electronic information board. 図１６は、電子情報ボードの選択消去時の動作を示すフローチャートである。FIG. 16 is a flowchart showing an operation at the time of selective erasure of the electronic information board. 図１７Ａは、電子情報ボードの消去前の画面を示す図である。FIG. 17A is a diagram showing a screen before erasing the electronic information board. 図１７Ｂは、電子情報ボードの消去中の画面を示す図である。FIG. 17B is a diagram showing a screen during erasure of the electronic information board. 図１７Ｃは、電子情報ボードの消去後の画面を示す図である。FIG. 17C is a diagram showing a screen after the electronic information board is erased. 図１８は、電子情報ボードのストローク単位消去時の動作を示すフローチャートである。FIG. 18 is a flowchart showing the operation at the time of erasing the stroke unit of the electronic information board. 図１９Ａは、電子情報ボードの投げ縄による選択前の状態を示す図である。FIG. 19A is a diagram showing a state before selection by the lasso of the electronic information board. 図１９Ｂは、電子情報ボードの投げ縄による選択中の状態を示す図である。FIG. 19B is a diagram showing a state during selection by a lasso of the electronic information board. 図１９Ｃは、電子情報ボードの投げ縄による選択中の状態を示す図である。FIG. 19C is a diagram illustrating a state in which the electronic information board is being selected by a lasso. 図１９Ｄは、電子情報ボードの投げ縄による選択後の状態を示す図である。FIG. 19D is a diagram showing a state after selection by the lasso of the electronic information board. 図２０は、電子情報ボードの投げ縄による選択の動作を示すフローチャートである。FIG. 20 is a flowchart showing the selection operation by the lasso of the electronic information board. 図２１Ａは、電子情報ボードの投げ縄による選択前の状態を示す図である。FIG. 21A is a diagram showing a state before selection by the lasso of the electronic information board. 図２１Ｂは、電子情報ボードの投げ縄による選択中の状態を示す図である。FIG. 21B is a diagram showing a state during selection by a lasso of the electronic information board. 図２１Ｃは、電子情報ボードの投げ縄による選択中の状態を示す図である。FIG. 21C is a diagram illustrating a state in which the electronic information board is being selected by a lasso. 図２２は、電子情報ボードの他の選択の動作を示すフローチャートである。FIG. 22 is a flowchart showing another selection operation of the electronic information board. 図２３Ａは、電子情報ボードの選択領域の移動中の状態を示す図である。FIG. 23A is a diagram illustrating a state in which the selection area of the electronic information board is being moved. 図２３Ｂは、電子情報ボードの選択領域の移動後の状態を示す図である。FIG. 23B is a diagram showing a state after the selection area of the electronic information board is moved. 図２４は、電子情報ボードの選択領域の移動の動作を示すフローチャートである。FIG. 24 is a flowchart showing an operation of moving the selection area of the electronic information board. 図２５は、電子情報ボードの選択領域の移動直後の状態を示す図である。FIG. 25 is a diagram illustrating a state immediately after the selection area of the electronic information board is moved. 図２６は、電子情報ボードの選択領域の移動直後の動作を示すフローチャートである。FIG. 26 is a flowchart showing the operation immediately after the selection area of the electronic information board is moved. 図２７Ａは、電子情報ボードの拡大動作前の状態を示す図である。FIG. 27A is a diagram illustrating a state before the electronic information board is enlarged. 図２７Ｂは、電子情報ボードの拡大動作後の状態を示す図である。FIG. 27B is a diagram showing a state after the enlargement operation of the electronic information board. 図２８は、電子情報ボードの拡大の動作を示すフローチャートである。FIG. 28 is a flowchart showing the operation of enlarging the electronic information board. 図２９は、電子情報ボードの拡大直後の状態を示す図である。FIG. 29 is a diagram illustrating a state immediately after the electronic information board is enlarged. 図３０は、電子情報ボードの拡大・縮小直後の動作を示すフローチャートである。FIG. 30 is a flowchart showing an operation immediately after enlargement / reduction of the electronic information board. 図３１Ａは、電子情報ボードのズーム前の状態を示す図である。FIG. 31A is a diagram illustrating a state of the electronic information board before zooming. 図３１Ｂは、電子情報ボードのズームの倍率を設定する様子を示す図である。FIG. 31B is a diagram showing how to set the zoom magnification of the electronic information board. 図３１Ｃは、電子情報ボードのズームを行った結果を示す図である。FIG. 31C is a diagram illustrating a result of zooming the electronic information board. 図３１Ｄは、電子情報ボードの映像が表示できない領域を示す図である。FIG. 31D is a diagram illustrating an area where an image of the electronic information board cannot be displayed. 図３２は、電子情報ボードのズームの動作を示すフローチャートである。FIG. 32 is a flowchart showing the zoom operation of the electronic information board. 図３３は、電子情報ボードの他のズームの動作を示す図である。FIG. 33 is a diagram showing another zoom operation of the electronic information board. 図３４は、電子情報ボードの他のズームの動作を示すフローチャートである。FIG. 34 is a flowchart showing another zoom operation of the electronic information board. 図３５Ａは、電子情報ボードのパン動作前の状態を示す図である。FIG. 35A is a diagram showing a state before the pan operation of the electronic information board. 図３５Ｂは、電子情報ボードのパン動作中の状態を示す図である。FIG. 35B is a diagram illustrating a state during a pan operation of the electronic information board. 図３５Ｃは、電子情報ボードのパン動作後の状態を示す図である。FIG. 35C is a diagram illustrating a state after the pan operation of the electronic information board. 図３６は、電子情報ボードのパンの動作を示すフローチャートである。FIG. 36 is a flowchart showing an operation of panning the electronic information board. 図３７は、電子情報ボードのパンの動作を示すフローチャートである。FIG. 37 is a flowchart showing the panning operation of the electronic information board. 図３８Ａは、電子情報ボードのパン動作前の状態を示す図である。FIG. 38A is a diagram showing a state before the pan operation of the electronic information board. 図３８Ｂは、電子情報ボードのパン動作中の状態を示す図である。FIG. 38B is a diagram illustrating a state during a pan operation of the electronic information board. 図３８Ｃは、電子情報ボードのパン動作後の状態を示す図である。FIG. 38C is a diagram illustrating a state after the pan operation of the electronic information board. 図３９は、電子情報ボードのその他のパンの動作を示すフローチャートである。FIG. 39 is a flowchart showing another panning operation of the electronic information board.
　＜実施形態の全体の構成と動作の概要＞
　以下、本発明の実施形態における構成及び動作について説明する。図１は、実施形態の画像処理システムの全体構成を示す図である。本実施形態においては、画像表示装置として電子情報ボード１を使用し、サーバ２は画像処理装置として機能し、図１に示すように、電子情報ボード１とサーバ２はネットワークによって接続されている。図２は、実施形態の電子情報ボード及びサーバの構成を示すブロック図である。 <Overview of Overall Configuration and Operation of Embodiment>
The configuration and operation in the embodiment of the present invention will be described below. FIG. 1 is a diagram illustrating an overall configuration of an image processing system according to an embodiment. In this embodiment, the electronic information board 1 is used as an image display device, the server 2 functions as an image processing device, and as shown in FIG. 1, the electronic information board 1 and the server 2 are connected by a network. FIG. 2 is a block diagram illustrating a configuration of the electronic information board and the server according to the embodiment.
　図２に示すように、電子情報ボード１は、タッチパネルディスプレイ１０１を有する。タッチパネルディスプレイ１０１は、画像表示手段として機能する。また、操作手段は、タッチパネルディスプレイ１０１をタッチすることであっても良いし、タッチペンやマウス等の機器を用いて入力することであっても良い。そのため、タッチパネルディスプレイ１０１が操作手段として機能する場合もある。 As shown in FIG. 2, the electronic information board 1 has a touch panel display 101. The touch panel display 101 functions as an image display unit. The operation means may be touching the touch panel display 101 or may be input using a device such as a touch pen or a mouse. Therefore, the touch panel display 101 may function as an operation unit.
　また、電子情報ボード１は、タッチパネルディスプレイ１０１で発生した操作イベントを受信する操作イベント受信部１０３、受信した操作イベントをサーバ２に送信する操作イベント送信部１０２を有する。操作イベント送信部１０２は送信手段として機能する。 In addition, the electronic information board 1 includes an operation event receiving unit 103 that receives an operation event that has occurred on the touch panel display 101, and an operation event transmitting unit 102 that transmits the received operation event to the server 2. The operation event transmission unit 102 functions as a transmission unit.
　また、受信した操作イベントでストローク編集用のインタフェースや、編集中の状態等を表示するストローク編集状態表示部１０５、を有する。なお、ストローク編集状態表示部１０５は、以降の説明においてローカルフィードバックとも呼ばれ、操作状態表示手段として機能する。 Also, it has a stroke editing interface and a stroke editing state display unit 105 for displaying a state during editing by the received operation event. The stroke editing state display unit 105 is also referred to as local feedback in the following description and functions as an operation state display unit.
　さらに、サーバ２のストローク表示・編集部２０１からイベント送信部２０４を介して送信されたイベントを受信するイベント受信部１０４、サーバ２から映像を受信する映像受信部１０６、受信した映像を表示する映像表示部１０７を有する。映像受信部１０６は動画受信手段として、映像表示部１０７は動画表示手段として機能する。 Furthermore, the event receiving unit 104 that receives an event transmitted from the stroke display / editing unit 201 of the server 2 via the event transmitting unit 204, the video receiving unit 106 that receives video from the server 2, and the video that displays the received video A display unit 107 is included. The video receiving unit 106 functions as a moving image receiving unit, and the video display unit 107 functions as a moving image display unit.
　サーバ２は、操作イベントを受信するイベント受信手段として機能する操作イベント受信部２０３、操作イベントを一旦保存するための操作イベントキュー２０２を有する。また、操作イベントキューから操作イベントを取り出してストロークを表示・編集するストローク表示・編集部２０１を有する。 The server 2 includes an operation event receiving unit 203 that functions as an event receiving unit that receives an operation event, and an operation event queue 202 for temporarily storing the operation event. Further, it has a stroke display / editing unit 201 that takes out an operation event from the operation event queue and displays / edits the stroke.
　その他にも、サーバ２は、ストローク編集中に発生したイベントを電子情報ボードに送信するイベント送信部２０４、背景映像を表示する背景映像表示部２０５を有する。また、背景映像とストローク映像を合成し動画を生成する映像合成部２０６、生成された動画を電子情報ボードに配信する映像送信部２０７からなる。映像合成部２０６は動画作成手段として機能し、映像送信部２０７は動画送信手段として機能する。 In addition, the server 2 includes an event transmission unit 204 that transmits an event generated during stroke editing to the electronic information board, and a background video display unit 205 that displays a background video. In addition, it includes a video composition unit 206 that synthesizes the background video and the stroke video to generate a video, and a video transmission unit 207 that distributes the generated video to the electronic information board. The video composition unit 206 functions as a moving image creation unit, and the video transmission unit 207 functions as a moving image transmission unit.
　図３は、実施形態の電子情報ボード及びサーバの動作を示すシーケンス図である。各シーケンスの主体及び対象となる構成要素については、図２の名称を用いて説明する。まず、タッチ等により電子情報ボード１にストロークの描画を行うと、マウスイベントが発生する（ステップＳ１０１）。 FIG. 3 is a sequence diagram illustrating operations of the electronic information board and the server according to the embodiment. The subject of each sequence and the target constituent elements will be described using the names in FIG. First, when a stroke is drawn on the electronic information board 1 by touch or the like, a mouse event is generated (step S101).
　操作の状態をストローク編集状態表示部１０５により画面に表示する（ステップＳ１０３）と共に、マウスイベントを操作イベント送信部１０２によりサーバ２に送信する（ステップＳ１０２）。 The operation state is displayed on the screen by the stroke editing state display unit 105 (step S103), and the mouse event is transmitted to the server 2 by the operation event transmission unit 102 (step S102).
　そして、サーバ２ではストローク表示・編集部２０１のストローク描画アプリケーションでストロークを描画する（ステップＳ２０１）。アプリケーションの画面を映像合成部２０６で動画エンコーディングし（ステップＳ２０２）、映像送信部２０７により電子情報ボード１に配信する（ステップＳ２０３）。 Then, the server 2 draws a stroke with the stroke drawing application of the stroke display / editing unit 201 (step S201). The application image is encoded with the video composition unit 206 (step S202), and the video transmission unit 207 distributes the application screen to the electronic information board 1 (step S203).
　電子情報ボード１は、映像受信部１０６により動画を受信し、受信した動画を映像表示部１０７により画面に表示する（ステップＳ１０４）。なお、説明中の「動画」とは画像を連続的に表示する一般的な動画を表すが、特に、動画の中でもＨ．２６４等の特定の形式にエンコードされ配信されたものを「映像」としている。本実施形態においては、前述の通り、サーバ２より電子情報ボードに配信される動画を特に映像と呼んでいる。 The electronic information board 1 receives the moving image by the video receiving unit 106, and displays the received moving image on the screen by the video display unit 107 (step S104). Note that the “moving image” in the description represents a general moving image in which images are continuously displayed. A video encoded in a specific format such as H.264 is defined as “video”. In the present embodiment, as described above, the moving image distributed from the server 2 to the electronic information board is particularly called a video.
　＜実施形態の構成と動作の詳細＞
　ここまでは全体的な構成と動作の概要について説明したが、以降は本実施形態の構成と動作の詳細について説明する。図４は、電子情報ボードのレイヤ構成を示す図である。電子情報ボード１は、操作パネル表示レイヤ１０１ｃ、ストローク編集用のインタフェースや、編集中の状態等を表示するローカルフィードバックレイヤ１０１ｂ、サーバから配信された映像を表示する映像表示レイヤ１０１ａからなる。 <Details of Configuration and Operation of Embodiment>
The overall configuration and the outline of the operation have been described so far, but the details of the configuration and the operation of the present embodiment will be described below. FIG. 4 is a diagram showing a layer configuration of the electronic information board. The electronic information board 1 includes an operation panel display layer 101c, an interface for stroke editing, a local feedback layer 101b for displaying a state during editing, and a video display layer 101a for displaying video distributed from a server.
　図５は、映像配信の概略を示す図である。図５に示すように、サーバ２はストロークと背景を合成し、それを動画にエンコーディングして、電子情報ボードに配信する。電子情報ボードでは、動画がデコードされ、映像として表示される。 FIG. 5 is a diagram showing an outline of video distribution. As shown in FIG. 5, the server 2 combines the stroke and the background, encodes it into a moving image, and distributes it to the electronic information board. On the electronic information board, the moving image is decoded and displayed as a video.
　図６は、実施形態の画像処理システムの操作のモード変更時の動作を示す図である。図６に示すように、電子情報ボード１の操作パネルにあるボタンを押すと、操作イベントとして、サーバ２にコマンドが送信される。ボタンが押下されると、電子情報ボードでは即座にモードが変更される。サーバ２側では、コマンド受信後にモードが変更される。なお、操作パネルとは、図４の操作パネル表示レイヤ１０１ｃの左側に表示されているコマンドの一覧を示すものである。 FIG. 6 is a diagram illustrating an operation when changing the operation mode of the image processing system according to the embodiment. As shown in FIG. 6, when a button on the operation panel of the electronic information board 1 is pressed, a command is transmitted to the server 2 as an operation event. When the button is pressed, the mode is immediately changed on the electronic information board. On the server 2 side, the mode is changed after receiving the command. The operation panel indicates a list of commands displayed on the left side of the operation panel display layer 101c in FIG.
　図７は、電子情報ボードにおいて使用可能なコマンドの一覧の例を示す図である。図７に示すように、ペンモード、消去モード、選択モード、ズームモード等がある。各コマンドは、必要な情報を引数としてサーバに送信することができる。 FIG. 7 is a diagram showing an example of a list of commands that can be used in the electronic information board. As shown in FIG. 7, there are a pen mode, an erase mode, a selection mode, a zoom mode, and the like. Each command can send necessary information to the server as an argument.
　図８は、マウスイベントが送信される際の動作を示す図である。操作パネル以外の、ストローク描画領域でペン操作をすると、操作イベントとして、マウスイベントがサーバに送信される。マウスイベントは座標情報を含む。 FIG. 8 is a diagram showing an operation when a mouse event is transmitted. When a pen operation is performed in a stroke drawing area other than the operation panel, a mouse event is transmitted to the server as an operation event. Mouse events include coordinate information.
　また、マウスムーブイベント（ＭｏｕｓｅＭｏｖｅイベント）は、ペンムーブ中に大量に発生するので、順序制御するために、イベントＩＤやシーケンス番号を持つことが望ましい。また、サーバ２側に送信されたマウスムーブイベントイベントは、一旦キューに保存されるのが望ましい。大量のマウスムーブイベントイベントが一度にストローク編集アプリに届けられると、アプリの処理が追いつかない可能性があるからである。 Also, since a large number of mouse move events (mouse move events) occur during a pen move, it is desirable to have an event ID or sequence number for order control. Further, it is desirable that the mouse move event event transmitted to the server 2 side is once saved in the queue. This is because if a large number of mouse move event events are delivered to the stroke editing application at once, the processing of the application may not catch up.
　以降、説明する動作は、特に記載がない場合はローカルフィードバックの動作を示すものである。サーバ側から配信される映像には、基本的には操作の結果得られる画像のみが含まれる。ただし、デバッグやデモ等の目的で、サーバ側から配信される映像中にもローカルフィードバックと同じ動作が含まれるようにしても良い。なお、前述の通り、映像とはサーバから配信された映像を示しており、図４の映像表示レイヤ１０１ａに表示されるものである。 Hereafter, the operations to be described indicate local feedback operations unless otherwise specified. The video distributed from the server side basically includes only an image obtained as a result of the operation. However, for the purpose of debugging or demonstration, the same operation as the local feedback may be included in the video distributed from the server side. As described above, the video indicates a video distributed from the server and is displayed on the video display layer 101a in FIG.
　また、端末側であるローカルフィードバックで示すストロークと、サーバ側で作成し配信される映像とは、当然厳密には一致しない。例えば、端末側で曲線を描画した場合、ローカルフィードバックにおいてはマウスが移動した座標を直線で結んだマルチラインで表示されるが、サーバ側では描画の際のマウスが移動した座標を基に、スムージングを行った曲線を作成する。これは、スムージングという負荷の大きい処理をサーバ側で行うことによって、端末側にかかる負荷を小さくするためである。 Of course, the stroke indicated by the local feedback on the terminal side and the video created and distributed on the server side do not exactly match. For example, when a curve is drawn on the terminal side, the local feedback is displayed as a multiline that connects the coordinates of the mouse movement with a straight line, but on the server side, smoothing is performed based on the coordinates of the mouse movement at the time of drawing. Create a curved line. This is to reduce the load on the terminal side by performing a process with a large load called smoothing on the server side.
　図９は、操作パネルの消去アイコンを選択した時の動作を示す図である。消去アイコンを押すと、消去範囲を選択できる。例えば４種類の範囲の中から選択した大きさで描画したストロークの消去を行うことができる。 FIG. 9 is a diagram showing an operation when the delete icon on the operation panel is selected. Press the erase icon to select the erase range. For example, a stroke drawn with a size selected from four types of ranges can be deleted.
　図１０Ａ、図１０Ｂは、電子情報ボードの消去時の動作を示す図である。図１０Ａは、電子情報ボードの消去前の画面を示し、図１０Ｂは、電子情報ボードの消去中の画面を示している。図１０Ｂに示すように、黒板消しアイコン５０２を操作すると、アイコンが掃いた領域５０１が選択した消去範囲の大きさで塗りつぶされ、消したように見える。塗りつぶしは、一定時間経過後にフェードアウトさせる。一定時間とは、映像配信により生じる遅延時間＋αである。 FIG. 10A and FIG. 10B are diagrams showing an operation at the time of erasing the electronic information board. FIG. 10A shows a screen before erasing the electronic information board, and FIG. 10B shows a screen during erasing of the electronic information board. As shown in FIG. 10B, when the blackboard erase icon 502 is operated, the area 501 where the icon has been swept is filled with the size of the selected erase range, and appears to have been erased. The fill is faded out after a certain period of time. The fixed time is a delay time + α caused by video distribution.
　また、ざっと黒板消しアイコン５０２を操作すると、５０３に示すような消し残しが残ることがよくある。そこで、消去されたストロークが所定の長さ以下になった場合、あるいは、長さが所定の割合以下になった場合、消し残しも消去するようにしても良い。 In addition, when the blackboard erase icon 502 is operated roughly, an unerased portion as indicated by 503 often remains. Therefore, when the erased stroke becomes a predetermined length or less, or when the length becomes a predetermined ratio or less, the unerased portion may be erased.
　図１１は、電子情報ボードの消去時の動作を示すフローチャートである。消去中は、黒板消しアイコン５０２が掃いた領域を特定の色で塗りつぶす。特定の色とは、簡単にはホワイトボードの背景色として採用されている白が考えられる。しかし、電子情報ボード１にはＰＣ画像などがストロークの下に表示されることがあるため、単純に白で塗りつぶすと背景が消えてしまったという印象を与えてしまう。 FIG. 11 is a flowchart showing the operation at the time of erasing the electronic information board. During erasure, the area swept by the blackboard eraser icon 502 is painted with a specific color. As the specific color, white that is adopted as the background color of the whiteboard can be considered easily. However, since a PC image or the like may be displayed below the stroke on the electronic information board 1, if it is simply painted in white, it gives the impression that the background has disappeared.
　そこで、画像全体の中央値の色や、消去開始した座標周辺の中央値の色などで塗りつぶすとより効果的である。さらに、これらの色を半透明にするとより効果的である。半透明の効果は、明るい色ほど透明効果を低くし、暗色ほど透明効果を大きくするとよい。色の明暗は、ＹＣｂＣｒのＹ値で判断する。 Therefore, it is more effective to paint with the median color of the entire image or the median color around the coordinates where erasing has started. Furthermore, it is more effective to make these colors translucent. For the translucent effect, it is preferable to lower the transparency effect for lighter colors and increase the transparency effect for darker colors. The brightness of the color is determined by the Y value of YCbCr.
　図１２は、電子情報ボードの消去時の動作を示すフローチャートの他の例である。図１２は、図１１とは異なる方法による消去時の動作を示している。図１１で示した消去では特定色で塗りつぶしていた。しかし、映像はその位置によって色が異なるので、一色で塗りつぶすのは効果的でない。そこで、黒板消しアイコン５０２が掃いた領域をその裏の映像をぼかして表示することでより効果的な消去感覚を与えることができる。 FIG. 12 is another example of a flowchart showing an operation at the time of erasing the electronic information board. FIG. 12 shows an erase operation by a method different from that in FIG. In the erasure shown in FIG. 11, the specific color is used for painting. However, since the color of the image differs depending on the position, it is not effective to fill it with one color. In view of this, it is possible to give a more effective erasing sensation by displaying the area swept by the blackboard eraser icon 502 by blurring the image behind it.
　図１２に示すように、前処理としては、映像をキャプチャーし、キャプチャーした画像を一旦縮小し、それにぼかしをかける。ぼかしフィルターとしては、モザイク、ガウンシアンフィルター等がよく知られている。ぼかしフィルターを掛けた後、それをもとのサイズに戻す。縮小した画像にぼかし処理を行うことにより、処理の負荷を小さくしている。 As shown in FIG. 12, as preprocessing, the video is captured, the captured image is once reduced, and the image is blurred. As a blurring filter, a mosaic, a Gauntian filter, etc. are well known. After applying the blur filter, return it to its original size. By performing the blurring process on the reduced image, the processing load is reduced.
　なお、後処理のため、同サイズの空画像である画像Ａを作成しておく。消去中の処理としては、黒板消しアイコン５０２の座標周辺を前記ぼかし画像から切り出し、映像の上に貼り付ける。また、後処理のため、切り出した画像は、画像Ａにも貼り付けておく。後処理としては、前記映像の上に貼り付けた画像を削除し、代わりに、画像Ａを貼り付ける。そして、一定時間経過後に画像Ａをフェードアウトさせる。 Note that an image A that is an empty image of the same size is created for post-processing. As the process during erasure, the periphery of the coordinates of the blackboard erase icon 502 is cut out from the blurred image and pasted on the video. Further, the clipped image is pasted on the image A for post-processing. As post-processing, the image pasted on the video is deleted and the image A is pasted instead. Then, the image A is faded out after a predetermined time has elapsed.
　映像の上に貼り付けた画像をフェードアウトしないのは、これら画像は大量に存在するため、処理が煩雑になるためである。映像の上に貼り付けた画像を集約した１枚の画像をフェードアウトさせることで、自然なフェードアウトを実現する。 The reason why the images pasted on the video are not faded out is that these images exist in large quantities and the processing becomes complicated. By fading out a single image that is a collection of images pasted on the video, a natural fade-out is realized.
　図１３は、画像処理システムの背景画像を配信する際の動作を示す図である。図１０では、ストロークが消えたように見せるため、映像をキャプチャーした画像にぼかしを入れて、それを黒板消しアイコン５０２の掃いた領域に貼り付けていた。これは、ストロークと背景画像が合成された映像が配信されるために発生する。 FIG. 13 is a diagram illustrating an operation when the background image of the image processing system is distributed. In FIG. 10, in order to make the stroke appear to disappear, the captured image is blurred and pasted on the cleaned area of the blackboard erase icon 502. This occurs because a video in which a stroke and a background image are combined is distributed.
　しかし、図１３の図のように背景画像が別途エンコードして配信されていれば、このようなぼかし処理は必要ない。背景画像の黒板消しが掃いた領域を切り取って貼り付ければよい。ただし、常に背景画像を動画送信する必要はなく、映像と同じｆｐｓで送信される必要はない。消去モードに変更されたときに、１ｆｐｓ程度で配信されれば十分である。 However, such a blurring process is not necessary if the background image is separately encoded and distributed as shown in FIG. What is necessary is just to cut and paste the area where the blackboard eraser of the background image has been swept away. However, it is not always necessary to transmit the background image as a moving image, and it is not necessary to transmit the background image at the same fps as the video. When changed to the erasure mode, it is sufficient to deliver at about 1 fps.
　また、配信時は通常の３０ｆｐｓ中、１ｆｐｓ分を背景送信に割り当てることで、ネットワーク帯域を余計に消費せずにすむ。通常の動画フレーム中に割り込んだ背景映像フレームはそれを受信した電子情報ボード中で分離する。分離するために、フレームのヘッダ情報を参照するとよい。図１４は、画像処理システムの背景画像を配信する際の動作を示すフローチャートである。 Also, at the time of distribution, it is possible to allocate 1 fps for background transmission out of the usual 30 fps, so that no extra network bandwidth is consumed. The background video frame interrupted in the normal moving image frame is separated in the electronic information board that has received it. In order to separate them, it is preferable to refer to the header information of the frame. FIG. 14 is a flowchart illustrating an operation when a background image is distributed in the image processing system.
　図１５Ａ、図１５Ｂは、電子情報ボードの選択消去の動作を示す図である。図１５Ａは、電子情報ボードの消去を行う前の画面の表示を示している。図１５Ｂは、電子情報ボードの消去中の画面の表示を示している。選択消去とは、投げ縄で囲んだ領域内にあるストロークを消去する機能である。 FIG. 15A and FIG. 15B are diagrams showing the operation of selectively erasing the electronic information board. FIG. 15A shows a screen display before the electronic information board is erased. FIG. 15B shows the display of the screen while the electronic information board is being erased. The selective erasing is a function for erasing a stroke in an area surrounded by a lasso.
　図１５Ｂに示すように、囲んだ領域を塗りつぶすことで、遅延を感じさせず消したかのように感じさせることができる。図１５Ｂでは白で塗りつぶしているが、前述の通常の消去の動作の説明で述べたように、映像のぼかし画像で塗りつぶしたり、背景映像を別途配信させ、それを使って塗りつぶしたりしても良い。図１６は、電子情報ボードの選択消去の動作を示すフローチャートである。 As shown in FIG. 15B, by filling the enclosed area, it can be felt as if it was erased without feeling a delay. Although it is painted in white in FIG. 15B, as described in the description of the normal erasing operation described above, it may be painted with a blurred image of the video, or the background video may be separately distributed and painted using it. . FIG. 16 is a flowchart showing the operation of selectively erasing the electronic information board.
　図１７Ａ、図１７Ｂ、図１７Ｃは、電子情報ボードのストローク単位消去の動作を示す図である。図１７Ａは、電子情報ボードの消去を行う前の画面を示している。図１７Ｂは、電子情報ボードの消去中の画面を示している。図１７Ｃは、電子情報ボードの消去を行った後の画面を示している。ストローク単位消去とは、消しゴムアイコン５１１の軌跡と交差したり、触れたりしたストロークをストロークごとに消去する方法である。 FIG. 17A, FIG. 17B, and FIG. 17C are diagrams showing the operation of deleting the stroke unit of the electronic information board. FIG. 17A shows a screen before the electronic information board is erased. FIG. 17B shows a screen during erasing of the electronic information board. FIG. 17C shows a screen after the electronic information board is erased. The stroke unit erasing is a method of erasing a stroke that intersects or touches the locus of the eraser icon 511 for each stroke.
　しかし、ストローク単位消去は、軌跡に触れただけでストローク全体を消去するため、前述の消去のようにストロークを白で塗りつぶして、遅延を感じさせないようにすることができない。そこで、図１７Ｂに記載のように、消しゴムの軌跡と消しゴムの残像を表示することで、すぐに消えないストレスを軽減させる。 However, erasing the stroke unit erases the entire stroke just by touching the locus, so that the stroke cannot be filled with white like the above-described erasure so as not to feel the delay. Therefore, as shown in FIG. 17B, the locus of the eraser and the afterimage of the eraser are displayed to reduce the stress that does not disappear immediately.
　消しゴムの軌跡は図では白で塗りつぶしているが、点線で描いたり、前述の通常の消去の動作と同様に、映像のぼかし画像で塗りつぶしたり、背景映像を別途配信させ、それを使って塗りつぶしてもよい。また、消しゴムの残像は、先に表示したアイコンから順に削除するよう構成されている。表示する残像の数、削除を開始するまでの時間は遅延時間に比例させるようにしても良い。図１８は、電子情報ボードのストローク単位消去の動作を示すフローチャートである。 The eraser's locus is painted white in the figure, but it is drawn with a dotted line, or it is painted with a blurred image of the image, or the background image is distributed separately and painted using it, as in the normal erase operation described above. Also good. In addition, the afterimage of the eraser is configured to be deleted in order from the previously displayed icon. The number of afterimages to be displayed and the time until the deletion is started may be proportional to the delay time. FIG. 18 is a flowchart showing the operation of deleting the stroke unit of the electronic information board.
　図１９Ａ、図１９Ｂ、図１９Ｃ、図１９Ｄは、電子情報ボードの投げ縄による選択の動作を示す図である。図１９Ａは、電子情報ボードの投げ縄による選択前の状態を示している。投げ縄で選択を開始すると、途中までの選択結果が表示される。図１９Ｂ、図１９Ｃは、電子情報ボードの投げ縄による選択中の状態を示している。選択中であるため、表示される矩形は、未確定の選択領域となる。図１９Ｄは、電子情報ボードの投げ縄による選択後の状態を示している。 FIG. 19A, FIG. 19B, FIG. 19C, and FIG. 19D are diagrams showing the selection operation by the lasso of the electronic information board. FIG. 19A shows a state before selection by the lasso of the electronic information board. When you start selecting with a lasso, the selection results up to the middle are displayed. FIG. 19B and FIG. 19C show a state where the electronic information board is being selected by a lasso. Since the selection is in progress, the displayed rectangle is an undetermined selection area. FIG. 19D shows a state after selection by the lasso of the electronic information board.
　未確定選択領域を描画するためその座標とサイズが必要となるが、ペンムーブイベントが逐一サーバに送信され、そのレスポンスとして返される。図１９Ｂと図１９Ｃとの間で未確定選択領域の大きさが変化しているが、大きさの変化は、アニメーションさせるようにしても良い。 The coordinates and size are required to draw the indeterminate selection area, but pen move events are sent to the server one by one and returned as a response. Although the size of the undetermined selection region changes between FIG. 19B and FIG. 19C, the change in size may be animated.
　ペンアップがされた後、ペンアップイベントがサーバに送信され、その結果が返ってきた時に選択確定となり、図１９Ｄに示すように、未確定選択領域は確定選択領域となる。四隅には拡大・縮小用のつまみが表示される。また、確定選択領域の枠の色は、未確定選択領域の色よりも濃くする。例えば、確定選択領域の色をグレーとし、未確定選択領域の色をライトグレーとする。 After pen-up, a pen-up event is transmitted to the server, and when the result is returned, the selection is confirmed, and as shown in FIG. Enlargement / reduction knobs are displayed in the four corners. Further, the color of the frame of the confirmed selection region is darker than the color of the unconfirmed selection region. For example, the color of the confirmed selection region is gray, and the color of the unconfirmed selection region is light gray.
　なお、未確定選択領域は、ペン操作で移動できるように構成する。これにより、選択未確定状態でも移動操作できることで、選択確定までユーザを待たせる必要がなくなり、遅延によるストレスを軽減できる。 Note that the unconfirmed selection area is configured so that it can be moved by pen operation. As a result, the moving operation can be performed even in a state where the selection is not confirmed, so that it is not necessary to wait for the user until the selection is confirmed, and stress due to delay can be reduced.
　他に、ストロークの選択状態は選択領域以外の部分でペンダウンすることで解除できる。このときは、ペンダウン時に選択領域の表示を削除する。削除はフェードアウトにしてもよい。図２０は、電子情報ボードの投げ縄による選択の動作を示すフローチャートである。 In addition, the selected state of the stroke can be canceled by pen-down at a part other than the selected area. In this case, the display of the selection area is deleted at the time of pen down. The deletion may be faded out. FIG. 20 is a flowchart showing the selection operation by the lasso of the electronic information board.
　図２１Ａ、図２１Ｂ、図２１Ｃは、電子情報ボードの他の選択方法の動作を示す図である。この場合は、投げ縄で選択しても前述のような未確定選択領域は表示されない。図２１Ａは、電子情報ボードの投げ縄による選択前の状態を示す図である。図２１Ｂは、電子情報ボードの投げ縄による選択中の状態を示す図である。図２１Ｂに示すように、ペンアップ後、投げ縄のバウンディングボックスが未確定選択領域として表示される。図２１Ｃは、電子情報ボードの投げ縄による選択後の状態を示す図である。 FIG. 21A, FIG. 21B, and FIG. 21C are diagrams showing the operation of another selection method of the electronic information board. In this case, the unconfirmed selection area as described above is not displayed even if the lasso is selected. FIG. 21A is a diagram showing a state before selection by the lasso of the electronic information board. FIG. 21B is a diagram showing a state during selection by a lasso of the electronic information board. As shown in FIG. 21B, after pen-up, the lasso bounding box is displayed as the unconfirmed selection area. FIG. 21C is a diagram showing a state after selection by the lasso of the electronic information board.
　その後、ペンアップイベントのレスポンスをサーバから受信したときに、図２１Ｃに示すように、未確定選択領域を確定選択領域に変更する。変更に際してはアニメーションを表示させても良い。なお、前述の選択の場合と同様に、未確定選択領域はペン操作で移動できるようにしても良い。図２２は、電子情報ボードの他の選択の動作を示すフローチャートである。 After that, when the response of the pen-up event is received from the server, as shown in FIG. 21C, the unconfirmed selected area is changed to the confirmed selected area. An animation may be displayed when changing. As in the case of the selection described above, the unconfirmed selection area may be moved by a pen operation. FIG. 22 is a flowchart showing another selection operation of the electronic information board.
　図２３Ａ、図２３Ｂは、電子情報ボードの移動の動作を示す図である。図２３Ａは、電子情報ボートの選択領域の移動中の状態を示す図である。図２３Ｂは、電子情報ボードの選択領域の移動後の状態を示す図である。図２３Ａに示すように、選択領域をペンでドラッグすることで、ストロークを移動させることができる。移動中はガイド枠を表示する。ガイド枠のみが移動するようにしても良いし、ガイド枠に映像のキャプチャー画像の選択領域部分を貼り付けるとようにしても良い。移動中であるため、半透明にする。 FIG. 23A and FIG. 23B are diagrams showing an operation of moving the electronic information board. FIG. 23A is a diagram showing a state in which the selected area of the electronic information boat is moving. FIG. 23B is a diagram showing a state after the selection area of the electronic information board is moved. As shown in FIG. 23A, the stroke can be moved by dragging the selection area with a pen. A guide frame is displayed during movement. Only the guide frame may be moved, or the selected area portion of the captured image of the video may be pasted on the guide frame. Make it translucent because it is moving.
　図２３Ｂに示すように、ペンアップすると、移動が確定し、選択領域自体を移動させる。同時に前記ガイドと貼り付けた画像を削除する。画像は単に削除するのではなく、一定時間経過後にフェードアウトさせるとよい。図２４は、電子情報ボードの選択領域の移動の動作を示すフローチャートである。 As shown in FIG. 23B, when the pen is up, the movement is confirmed and the selected area itself is moved. At the same time, the guide and the pasted image are deleted. The image should not be simply deleted, but faded out after a certain period of time. FIG. 24 is a flowchart showing an operation of moving the selection area of the electronic information board.
　図２５は、電子情報ボードの選択領域の移動直後の状態を示す図である。図２５に示すように、選択領域は移動しているが、遅延のため、映像上の文字はまだ移動していない。問題は、映像上の文字がもとの位置のままであるため、映像上で文字が移動するまで、ユーザを待たせてしまうことである。 FIG. 25 is a diagram showing a state immediately after the selection area of the electronic information board is moved. As shown in FIG. 25, the selected area has moved, but due to the delay, the characters on the video have not moved yet. The problem is that the characters on the video remain in their original positions, causing the user to wait until the characters move on the video.
　そこで、前述のように背景画像を取得できれば、選択領域のもとの位置に、背景画像の該当部分を貼り付けることで文字を隠すことができる。貼り付けた画像は一定時間経過後に消去する。図２６は、電子情報ボードの選択領域の移動直後の動作を示すフローチャートである。 Therefore, if the background image can be acquired as described above, the character can be hidden by pasting the corresponding part of the background image at the original position of the selected area. The pasted image is deleted after a predetermined time. FIG. 26 is a flowchart showing the operation immediately after the selection area of the electronic information board is moved.
　図２７Ａ、図２７Ｂは、電子情報ボードの拡大の動作を示す図である。図２７Ａは、電子情報ボードの拡大動作前の状態を示す図である。図２７Ｂは、電子情報ボードの拡大動作後の状態を示す図である。図２７Ａに示すように、選択領域四隅のつまみをペンでドラッグすることで、ストロークのサイズ変更、即ち拡大・縮小ができる。基本的動作は移動と同様である。例えば、拡大動作を行った後は図２７Ｂに示す状態となる。図２８は、電子情報ボードの拡大の動作を示すフローチャートである。 FIG. 27A and FIG. 27B are diagrams showing the operation of enlarging the electronic information board. FIG. 27A is a diagram illustrating a state before the electronic information board is enlarged. FIG. 27B is a diagram showing a state after the enlargement operation of the electronic information board. As shown in FIG. 27A, the size of the stroke, that is, enlargement / reduction can be changed by dragging the knobs at the four corners of the selection area with a pen. The basic operation is the same as moving. For example, after performing the enlargement operation, the state shown in FIG. 27B is obtained. FIG. 28 is a flowchart showing the operation of enlarging the electronic information board.
　つまみをペンでドラッグすることで、選択領域の拡大・縮小を行う。サイズの変更中はガイド枠を表示する。ガイド枠のみがサイズ変更されるようにしても良いし、ガイド枠には映像のキャプチャー画像の選択領域部分を貼り付けるようにしても良い。サイズの変更中であるため、半透明にする。 ド ラ ッ グ Enlarge / reduce the selected area by dragging the knob with the pen. A guide frame is displayed while changing the size. Only the guide frame may be resized, or a selection area portion of a captured video image may be pasted on the guide frame. Because the size is changing, make it translucent.
　図２９は、電子情報ボードの拡大直後の状態を示す図である。図２９に示すように、移動の場合と同様に、拡大・縮小でも、映像が拡大・縮小されるまで、遅延が生じる。図２５で示した移動の場合と同様の方法でこの問題を解決することが可能である。 FIG. 29 is a diagram showing a state immediately after the electronic information board is expanded. As shown in FIG. 29, as in the case of movement, even with enlargement / reduction, a delay occurs until the image is enlarged / reduced. This problem can be solved by a method similar to that of the movement shown in FIG.
　即ち、背景画像を取得し、選択領域の拡大・縮小前の位置に、背景画像の該当部分を貼り付けることで文字を隠すことができる。貼り付けた画像は一定時間経過後に消去する。図３０は、電子情報ボードの拡大・縮小直後の動作を示すフローチャートである。 That is, the character can be hidden by acquiring the background image and pasting the corresponding part of the background image at the position before the enlargement / reduction of the selected area. The pasted image is deleted after a predetermined time. FIG. 30 is a flowchart showing an operation immediately after enlargement / reduction of the electronic information board.
　図３１Ａ、図３１Ｂ、図３１Ｃ、図３１Ｄは、電子情報ボードのズームの動作を示す図である。図３１Ａは、電子情報ボードのズーム前の状態を示し、図３１Ｂは、電子情報ボードのズームの倍率を設定する様子を示している。画面をズームする場合は、まず映像をキャプチャーし、それをズームさせる。ズーム後にキャプチャー画像をフェードアウトさせる。図３１Ｃは、電子情報ボードのズームの動作を行った結果を示している。図３２は、電子情報ボードのズームの動作を示すフローチャートである。 FIG. 31A, FIG. 31B, FIG. 31C, and FIG. 31D are diagrams showing the zoom operation of the electronic information board. FIG. 31A shows a state of the electronic information board before zooming, and FIG. 31B shows a state where the zoom magnification of the electronic information board is set. To zoom the screen, first capture the video and zoom it. Fade out the captured image after zooming. FIG. 31C shows the result of the zoom operation of the electronic information board. FIG. 32 is a flowchart showing the zoom operation of the electronic information board.
　例えばズーム率を２００％から１００％に変化させる場合、２００％の映像は、端末で表示される部分だけ配信されるため、映像をキャプチャーしてズーム率を変更すると、図３１Ｄのように映像が表示できない領域が生じる。この領域はグレーで塗りつぶす。なお、ズーム中に現在のズーム率を示すために、画面中央にズーム率を表示してもよい。 For example, when the zoom rate is changed from 200% to 100%, the video of 200% is distributed only in the portion displayed on the terminal. Therefore, when the video is captured and the zoom rate is changed, the video is displayed as shown in FIG. 31D. An area that cannot be displayed is generated. This area is filled with gray. In order to show the current zoom rate during zooming, the zoom rate may be displayed at the center of the screen.
　図３３は、電子情報ボードの他のズームの動作を示す図である。前述の動作の場合、２００％から１００％にズームの倍率を変化させる場合に、グレーの部分が見えてしまうという問題があった。この問題は、拡大した場合もフルサイズの映像を配信することで解決できる。即ち、ズームの倍率が減少した場合に、グレーの部分としていた場所にフルサイズの映像を表示する。 FIG. 33 is a diagram showing another zoom operation of the electronic information board. In the case of the above-described operation, there is a problem that a gray portion is visible when the zoom magnification is changed from 200% to 100%. This problem can be solved by distributing full-size video even when enlarged. That is, when the zoom magnification is reduced, a full-size image is displayed in a place where the gray portion is present.
　しかし、例えば８００％のサイズの場合にフルサイズの映像をすべての端末に配信すると帯域を圧迫するため現実的ではない。そこで、図３３に示すように、１ｆｐｓ程度でフルサイズの映像を配信することで解決する。なお、このフルサイズ映像はローカルフィードバック用なので、８００％の場合でも８００％のサイズで配信する必要はなく、２００％程度にしてから配信するようにしても良い。図３４は、電子情報ボードの他のズーム動作を示すフローチャートである。 However, for example, in the case of 800% size, it is not realistic to distribute the full-size video to all terminals because the band is compressed. Therefore, as shown in FIG. 33, the problem is solved by distributing a full-size video at about 1 fps. Note that this full-size video is for local feedback, so even if it is 800%, it is not necessary to deliver it in 800% size, and it may be delivered after about 200%. FIG. 34 is a flowchart showing another zoom operation of the electronic information board.
　図３５Ａ、図３５Ｂ、図３５Ｃは、電子情報ボードのパンの動作を示す図である。図３５Ａは、電子情報ボードのパン動作前の状態を示す図である。図３５Ｂは、電子情報ボードのパン動作中の状態を示す図である。図３５Ｃは、電子情報ボードのパン動作後の状態を示す図である。ズームした場合は、希望の場所を閲覧するために、画面をパンすることができる。ズームと同様に、映像をキャプチャーして移動させることで実現できる。しかし、キャプチャー画像のサイズは電子情報ボードの画面サイズと同じなので、パンすると図３５Ｂに示すように、背後の領域が見えてしまうことになり、図３５Ｃの状態となるまでに遅延が生じる。 FIG. 35A, FIG. 35B, and FIG. 35C are diagrams illustrating the panning operation of the electronic information board. FIG. 35A is a diagram showing a state before the pan operation of the electronic information board. FIG. 35B is a diagram illustrating a state during a pan operation of the electronic information board. FIG. 35C is a diagram illustrating a state after the pan operation of the electronic information board. When zoomed, the screen can be panned to view the desired location. Similar to zooming, this can be achieved by capturing and moving the video. However, since the size of the captured image is the same as the screen size of the electronic information board, when panning, as shown in FIG. 35B, the area behind is visible, and a delay occurs until the state of FIG. 35C is reached.
　そこで、ズームの場合と同様の方法により、この問題を解決する。即ち、フルサイズ若しくは大きめのサイズの映像を配信しておき、パン直後の遅延が発生している間に表示する方法である。図３６及び図３７は、電子情報ボードのパンの動作を示すフローチャートである。 Therefore, this problem is solved by the same method as in the case of zooming. In other words, a full-size or larger-size video is distributed and displayed while a delay occurs immediately after panning. 36 and 37 are flowcharts showing the panning operation of the electronic information board.
　しかし、フルサイズの映像のキャプチャー画像は巨大であり、この画像をパン操作のために移動すると、処理遅延が発生する可能性がある。なお、パン中は、全体の中のどこにいるかを示すため、現在位置を表示するガイドを表示してもよい。図３８Ａ、図３８Ｂ、図３８Ｃは、電子情報ボードのその他のパンの動作を示す図である。図３８Ａは、電子情報ボードのパン動作前の状態を示す図である。図３８Ｂは、電子情報ボードのパン動作中の状態を示す図である。図３８Ｃは、電子情報ボードのパン動作後の状態を示す図である。 However, the captured image of the full-size video is huge, and if this image is moved for pan operation, processing delay may occur. During panning, a guide for displaying the current position may be displayed to indicate where in the whole. 38A, 38B, and 38C are diagrams illustrating other pan operations of the electronic information board. FIG. 38A is a diagram showing a state before the pan operation of the electronic information board. FIG. 38B is a diagram illustrating a state during a pan operation of the electronic information board. FIG. 38C is a diagram illustrating a state after the pan operation of the electronic information board.
　この問題を解決するため、図３８Ｂに示すように、キャプチャー画像をパンするのではなく、枠線を表示し、それを移動させることでパンしていることを示すことが考えられる。図３８Ｃに示すように、枠線は一定時間経過後に消去する。図３９は、電子情報ボードのその他のパンの動作を示すフローチャートである。 In order to solve this problem, as shown in FIG. 38B, instead of panning the captured image, it may be possible to display a frame line and move it to indicate that it is panning. As shown in FIG. 38C, the frame line is erased after a predetermined time. FIG. 39 is a flowchart showing another panning operation of the electronic information board.
　＜実施形態の効果＞
　これらの動作に共通して、手書き処理をサーバ側で行い、端末側ではサーバから配信された動画の表示と、手書き編集状態表示のみを行うので端末側に負荷をかけない。そのため、安価な電子情報ボードを提供できる。また、端末を交換することなく、処理の重い機能を提供することができる。さらに、安価でローパワーの電子情報ボードでありながら、手書き編集を遅延なく行うことができる。 <Effect of embodiment>
In common with these operations, handwriting processing is performed on the server side, and only the display of the moving image distributed from the server and the handwriting editing state display are performed on the terminal side, so that no load is imposed on the terminal side. Therefore, an inexpensive electronic information board can be provided. In addition, it is possible to provide a heavy processing function without replacing the terminal. Furthermore, handwriting editing can be performed without delay while being an inexpensive and low-power electronic information board.
　なお、上記の実施形態は、本発明の好適な実施形態であり、上記実施形態のみに本発明を限定するものではなく、本発明の要旨を逸脱しない範囲において種々の変更を施した形態での実施が可能である。 The above-described embodiment is a preferred embodiment of the present invention, and the present invention is not limited to the above-described embodiment alone, and various modifications are made without departing from the gist of the present invention. Implementation is possible.
　画像表示装置である電子情報ボード１と、画像処理装置であるサーバ２をネットワークで接続し画像処理システムとしても良い。また、これらの画像処理を実行させるためのプログラムを電子情報ボード１及びサーバ２にインストールして実行させるようにしても良い。 The electronic information board 1 that is an image display device and the server 2 that is an image processing device may be connected via a network to form an image processing system. A program for executing these image processes may be installed in the electronic information board 1 and the server 2 and executed.
　また、画像表示装置の例としてタッチパネルによって操作される電子情報ボードを挙げているが、画像表示装置としては電子情報ボードに限らず、ディスプレイ一体型のパーソナルコンピュータやタブレット端末等であっても良い。 Further, although an electronic information board operated by a touch panel is cited as an example of the image display device, the image display device is not limited to the electronic information board, and may be a display-integrated personal computer, a tablet terminal, or the like.
　１　　電子情報ボード
　２　　サーバ
　１０１　　タッチパネルディスプレイ
　１０１ａ　　映像表示レイヤ
　１０１ｂ　　ローカルフィードバックレイヤ
　１０１ｃ　　操作パネル表示レイヤ
　１０２　　操作イベント送信部
　１０３　　操作イベント受信部
　１０４　　イベント受信部
　１０５　　ストローク編集状態表示部
　１０６　　映像受信部
　１０７　　映像表示部
　２０１　　ストローク表示・編集部
　２０２　　操作イベントキュー
　２０３　　操作イベント受信部
　２０４　　イベント送信部
　２０５　　背景映像表示部
　２０６　　映像合成部
　２０７　　映像送信部
　５０２　　黒板消しアイコン
　５１１　　消しゴムアイコン DESCRIPTION OF SYMBOLS 1 Electronic information board 2 Server 101 Touch panel display 101a Video display layer 101b Local feedback layer 101c Operation panel display layer 102 Operation event transmission part 103 Operation event reception part 104 Event reception part 105 Stroke edit state display part 106 Video reception part 107 Video display part 201 Stroke Display / Editing Unit 202 Operation Event Queue 203 Operation Event Receiving Unit 204 Event Transmitting Unit 205 Background Video Display Unit 206 Video Composition Unit 207 Video Transmitting Unit 502 Blackboard Eraser Icon 511 Eraser Icon
特表２０１１－５２４５７２号公報Special table 2011-524572

Claims (20)

  1. 　画像を表示する画像表示手段と、
    　前記画像表示手段が表示している画像の上にストロークを描画し編集するための操作を行う操作手段と、
    を有する画像表示装置であって、
    　前記操作手段による操作の状態を表示する操作状態表示手段と、
    　前記操作手段によって発生する操作イベントを送信する操作イベント送信手段と、
    　前記操作イベントに基づくストロークを示す動画を受信する動画受信手段と、
    　前記動画受信手段によって受信した動画を表示する動画表示手段と、を有することを特徴とする画像表示装置。     Image display means for displaying an image;
    Operation means for performing an operation for drawing and editing a stroke on the image displayed by the image display means;
    An image display device comprising:
    Operation state display means for displaying the state of operation by the operation means;
    An operation event transmitting means for transmitting an operation event generated by the operation means;
    Moving image receiving means for receiving a moving image indicating a stroke based on the operation event;
    An image display device comprising: a moving image display unit that displays the moving image received by the moving image receiving unit.
  2. 　前記画像表示手段と、前記操作手段と、を機能として備えるタッチパネルディスプレイを有する電子情報ボードであることを特徴とする請求項１に記載の画像表示装置。 2. The image display device according to claim 1, wherein the image display device is an electronic information board having a touch panel display including the image display means and the operation means as functions.
  3. 　前記操作状態表示手段は、
    　前記操作手段によって投げ縄によって選択した領域を消去する操作が行われた場合、消去の対象となる領域に塗りつぶしを表示することで消去の操作状態を表示し、操作が確定した後、一定時間経過後に前記塗りつぶしを消去することを特徴とする請求項１に記載の画像表示装置。     The operation state display means includes
    When an operation for erasing the area selected by the lasso is performed by the operation means, the erasing operation state is displayed by displaying a fill in the area to be erased, and a certain time has elapsed after the operation is confirmed. The image display apparatus according to claim 1, wherein the fill is erased later.
  4. 　前記操作状態表示手段は、
    　前記操作手段によってアイコンが移動した範囲を消去する操作が行われた場合、消去の対象となる領域に塗りつぶしを表示することで消去の操作状態を表示し、操作が確定した後、一定時間経過後に前記塗りつぶしを消去することを特徴とする請求項１に記載の画像表示装置。     The operation state display means includes
    When an operation for erasing the range in which the icon has been moved by the operation means is performed, the erasure operation state is displayed by displaying a fill in the area to be erased, and after a certain time has elapsed after the operation is confirmed The image display apparatus according to claim 1, wherein the fill is erased.
  5. 　前記塗りつぶしは、消去の対象の領域の画像のキャプチャーにぼかし処理を行ったものであることを特徴とする請求項３または４に記載の画像表示装置。 The image display device according to claim 3 or 4, wherein the filling is obtained by performing blurring processing for capturing an image of a region to be erased.
  6. 　前記ぼかし処理は、ガウンシアンフィルターによるぼかし処理であることを特徴とする請求項５に記載の画像表示装置。 6. The image display device according to claim 5, wherein the blurring process is a blurring process using a Gaussian filter.
  7. 　前記ぼかし処理は、消去の対象の領域の画像のキャプチャーを縮小した縮小画像に対して行われ、
    　前記塗りつぶしは、前記ぼかし処理が行われた前記縮小画像を拡大したものであることを特徴とする請求項５に記載の画像表示装置。     The blurring process is performed on the reduced image obtained by reducing the capture of the image of the area to be erased,
    The image display apparatus according to claim 5, wherein the filling is an enlargement of the reduced image that has been subjected to the blurring process.
  8. 　前記消去する操作は、
    　消去の操作によりストロークの所定の長さの部分が消去されずに残った場合、残った部分も消去することを特徴とする請求項３に記載の画像表示装置。     The erasing operation is
    4. The image display device according to claim 3, wherein when a portion having a predetermined length of the stroke remains without being erased by the erasing operation, the remaining portion is also erased.
  9. 　前記消去する操作は、
    　消去の操作によりストロークの一定の割合の部分が消去された場合、残りの部分も消去することを特徴とする請求項３に記載の画像表示装置。     The erasing operation is
    4. The image display device according to claim 3, wherein when a portion of a certain proportion of the stroke is erased by the erasing operation, the remaining portion is also erased.
  10. 　前記操作状態表示手段は、
    　前記操作手段によって移動したアイコンが触れたストロークをストローク単位で消去する操作が行われた場合、アイコンが移動した軌跡を残像として表示し、一定時間経過後に前記残像を消去することを特徴とする請求項１に記載の画像表示装置。     The operation state display means includes
    When an operation for deleting the stroke touched by the icon moved by the operation means is performed in units of strokes, the locus of movement of the icon is displayed as an afterimage, and the afterimage is deleted after a predetermined time has elapsed. Item 4. The image display device according to Item 1.
  11. 　前記操作状態表示手段は、
    　前記操作手段によって投げ縄による領域の選択が行われた場合、選択されたストロークを含む領域を矩形の未確定選択領域として表示した後、矩形の確定選択領域になることを特徴とする請求項１に記載の画像表示装置。     The operation state display means includes
    2. When a region is selected by a lasso by the operation means, the region including the selected stroke is displayed as a rectangular unconfirmed selection region, and then becomes a rectangular fixed selection region. The image display device described in 1.
  12. 　前記確定選択領域の枠の色は、前記未確定選択領域の枠の色と異なる色で表示されることを特徴とする請求項１１に記載の画像表示装置。 12. The image display device according to claim 11, wherein the color of the frame of the confirmed selection region is displayed in a color different from the color of the frame of the unconfirmed selected region.
  13. 　前記操作状態表示手段は、
    　前記操作手段によって選択領域を移動する操作が行われた場合、移動の対象となる領域の画像のキャプチャーを移動の操作と共に移動させ、操作が確定した後、一定時間経過後に前記移動の対象となる領域の画像のキャプチャーを消去することを特徴とする請求項１に記載の画像表示装置。     The operation state display means includes
    When an operation to move the selected area is performed by the operation means, the image capture of the area to be moved is moved together with the movement operation, and after the operation is confirmed, the image is to be moved after a certain time has passed. The image display apparatus according to claim 1, wherein the image capture of the area is erased.
  14. 　前記画像のキャプチャーは、移動の操作が行われている間は半透明で表示されることを特徴とする請求項１３に記載の画像表示装置。 14. The image display device according to claim 13, wherein the image capture is displayed in a translucent manner while a movement operation is performed.
  15. 　前記操作状態表示手段は、
    　前記操作手段によって選択領域を移動する操作が行われた場合、移動の対象となる領域に表示されるガイド枠のみが移動し、操作が確定した後、一定時間経過後に前記ガイド枠を消去することを特徴とする請求項１に記載の画像表示装置。     The operation state display means includes
    When an operation for moving the selected area is performed by the operation means, only the guide frame displayed in the area to be moved is moved, and after the operation is confirmed, the guide frame is deleted after a predetermined time has elapsed. The image display apparatus according to claim 1.
  16. 　前記移動の対象となる領域は、移動の操作が行われている間は、背景画像の前記移動の対象となる領域に該当する部分が貼り付けられることを特徴とする請求項１３または１５に記載の画像表示装置。 16. The area to be moved is pasted in a portion corresponding to the area to be moved of a background image while a movement operation is being performed. Image display device.
  17. 　前記操作状態表示手段は、
    　前記操作手段によって選択領域のサイズを変更する操作が行われた場合、変更する対象の領域の画像のキャプチャーを変更の操作と共にサイズを変更し、操作が確定した後、一定時間経過後に前記変更する対象の領域の画像のキャプチャーを消去することを特徴とする請求項１に記載の画像表示装置。     The operation state display means includes
    When an operation for changing the size of the selected area is performed by the operation unit, the size of the image captured in the area to be changed is changed together with the change operation, and after the operation is confirmed, the change is performed after a predetermined time has elapsed. The image display apparatus according to claim 1, wherein the image capture of the target area is erased.
  18. 　前記操作状態表示手段は、ズーム後にパンの操作が行われた場合、枠線を表示し、前記枠線を移動させることによって前記パンの操作が行われていることを表すことを特徴とする請求項１に記載の画像表示装置。 The operation state display means displays a frame line when a pan operation is performed after zooming, and indicates that the pan operation is performed by moving the frame line. Item 4. The image display device according to Item 1.
  19. 　請求項１に記載の画像表示装置と、画像処理装置とを有する画像処理システムであって、
    　前記画像処理装置は、
    　前記操作イベント送信手段によって送信された操作イベントを受信する操作イベント受信手段と、
    　前記操作イベント受信手段が受信した操作イベントに基づいてストロークを示す動画を作成する動画作成手段と、
    　前記動画作成手段により作成された動画を前記動画受信手段に対して送信する動画送信手段と、を有することを特徴とする画像処理システム。     An image processing system comprising the image display device according to claim 1 and an image processing device,
    The image processing apparatus includes:
    An operation event receiving means for receiving the operation event transmitted by the operation event transmitting means;
    Movie creating means for creating a movie showing a stroke based on the operation event received by the operation event receiving means;
    An image processing system comprising: a moving image transmitting unit that transmits the moving image generated by the moving image generating unit to the moving image receiving unit.
  20. 　画像表示装置に表示されている画像の上にストロークを描画し編集するための操作を行う操作ステップと
    　前記操作ステップによる操作の状態を表示する操作状態表示ステップと、
    　前記操作ステップによって発生する操作イベントを送信する操作イベント送信ステップと、
    　前記操作イベントに基づくストロークを示す動画を受信する動画受信ステップと、
    　前記動画受信ステップによって受信した動画を表示する動画表示ステップと、を有することを特徴とする画像処理方法。     An operation step for performing an operation for drawing and editing a stroke on an image displayed on the image display device; an operation state display step for displaying an operation state by the operation step;
    An operation event transmission step of transmitting an operation event generated by the operation step;
    A moving image receiving step of receiving a moving image indicating a stroke based on the operation event;
    And a moving image display step for displaying the moving image received in the moving image receiving step.
PCT/JP2015/061290 2014-07-07 2015-04-10 Image display apparatus, image processing system, and image processing method WO2016006294A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2014139996A JP2017151491A (en) 2014-07-07 2014-07-07 Image display device, image processing system, image processing method, and image processing program
JP2014-139996 2014-07-07

Publications (1)

Publication Number Publication Date
WO2016006294A1 true WO2016006294A1 (en) 2016-01-14

Family

ID=55063939

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2015/061290 WO2016006294A1 (en) 2014-07-07 2015-04-10 Image display apparatus, image processing system, and image processing method

Country Status (2)

Country Link
JP (1) JP2017151491A (en)
WO (1) WO2016006294A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10997759B2 (en) 2017-05-08 2021-05-04 Lg Electronics Inc. User interface apparatus for vehicle and vehicle

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2020149633A (en) * 2019-03-15 2020-09-17 株式会社リコー Display device, display method, and display program
JP7406528B2 (en) 2021-07-28 2023-12-27 キヤノン株式会社 Image processing device, image processing method, and program

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH02288980A (en) * 1989-02-01 1990-11-28 Ricoh Co Ltd Image display control system
JPH03290726A (en) * 1990-04-06 1991-12-20 Seiko Epson Corp Cursor display method
JPH0683559A (en) * 1992-08-31 1994-03-25 Fujitsu General Ltd Menu selection method
JP2003036225A (en) * 2001-07-24 2003-02-07 Matsushita Electric Ind Co Ltd Electronic bulletin board system
JP2004118236A (en) * 2002-09-20 2004-04-15 Ricoh Co Ltd Device, system, method and program for managing picture data
JP2006350838A (en) * 2005-06-17 2006-12-28 Fujitsu Ltd Information processing apparatus and program
JP2013175189A (en) * 2012-02-24 2013-09-05 Samsung Electronics Co Ltd Method and device for generating captured image for display windows
JP2014016668A (en) * 2012-07-05 2014-01-30 Fujifilm Corp Display terminal device, erasure method and program

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH02288980A (en) * 1989-02-01 1990-11-28 Ricoh Co Ltd Image display control system
JPH03290726A (en) * 1990-04-06 1991-12-20 Seiko Epson Corp Cursor display method
JPH0683559A (en) * 1992-08-31 1994-03-25 Fujitsu General Ltd Menu selection method
JP2003036225A (en) * 2001-07-24 2003-02-07 Matsushita Electric Ind Co Ltd Electronic bulletin board system
JP2004118236A (en) * 2002-09-20 2004-04-15 Ricoh Co Ltd Device, system, method and program for managing picture data
JP2006350838A (en) * 2005-06-17 2006-12-28 Fujitsu Ltd Information processing apparatus and program
JP2013175189A (en) * 2012-02-24 2013-09-05 Samsung Electronics Co Ltd Method and device for generating captured image for display windows
JP2014016668A (en) * 2012-07-05 2014-01-30 Fujifilm Corp Display terminal device, erasure method and program

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10997759B2 (en) 2017-05-08 2021-05-04 Lg Electronics Inc. User interface apparatus for vehicle and vehicle
US11341700B2 (en) 2017-05-08 2022-05-24 Lg Electronics Inc. User interface apparatus for vehicle and vehicle

Also Published As

Publication number Publication date
JP2017151491A (en) 2017-08-31

Similar Documents

Publication Publication Date Title
US10250947B2 (en) Meeting system that interconnects group and personal devices across a network
EP3742280A1 (en) Display annotation method, device, apparatus, and storage medium
US7286141B2 (en) Systems and methods for generating and controlling temporary digital ink
EP2446619B1 (en) Method and device for modifying a composite video signal layout
US7324069B2 (en) Animation review methods and apparatus
US20130106888A1 (en) Interactively zooming content during a presentation
CN107145280B (en) Image data processing method and device
JP2003296012A (en) System for inputting and displaying graphic and method of using interface
CN112860163B (en) Image editing method and device
US9229679B2 (en) Image distribution apparatus, display apparatus, and image distribution system
US20180082663A1 (en) Information processing apparatus, image displaying method, and non-transitory computer readable medium
CN107766024B (en) PPT projection control method and system based on splicing wall
US10855481B2 (en) Live ink presence for real-time collaboration
EP2579588B1 (en) Collaborative meeting systems that enable parallel multi-user input to mark up screens
WO2016006294A1 (en) Image display apparatus, image processing system, and image processing method
JP2023038195A (en) Image processing method and information processing device
CN105743769B (en) Message image-text display method
CN113741775A (en) Image processing method and device and electronic equipment
JP2014149579A (en) Data control device, data sharing system, and program
JP2004325941A (en) Drawing processor, drawing method, drawing program, and electronic conference system provided with them
Ashdown et al. Remote collaboration on desk‐sized displays
KR101307192B1 (en) 3 Dimensional Electronic Writting System
KR101356639B1 (en) 3 Dimensional Electronic Writting Method using 3 Dimensional Electronic Writting System
JP2016212844A (en) Display device and display system
TW201616379A (en) Information transmission method capable of selecting preview information

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15818640

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: JP

122 Ep: pct application non-entry in european phase

Ref document number: 15818640

Country of ref document: EP

Kind code of ref document: A1