JP2020149634A - Image processing apparatus, method, and program - Google Patents

Abstract

To perform writing in display of a list of pages on an image processing apparatus, such as an electronic blackboard.SOLUTION: An image processing apparatus comprises a list control unit that creates and displays a screen of a list of a plurality of pages, wherein one page or two or more pages in the list are formed of a plurality of layers different in use, and changes the configuration of the plurality of layers to receive writing in the one page or the two or more pages.SELECTED DRAWING: Figure 3

Description

The present invention relates to image processing devices, methods, and programs.

Conventionally, at meetings in companies, educational institutions, government agencies, etc., an electronic blackboard has been used in which a background image is displayed on a large display and a user can draw a stroke image such as characters, numbers, and figures on the background image. ing.

It is known that such an image processing device such as an electronic blackboard has a function of writing by handwriting or the like and creating a page composed of captured images or the like. It is also known that the content of a page can be shared with an image processing device at a remote location by connecting a plurality of image processing devices. It is also known that the created pages can be displayed in a list to check the contents, and operations such as adding and duplicating pages can be performed.

For example, Patent Document 1 discloses that each information processing apparatus can display and browse data managed by a server in a list. In Patent Document 1, drawing by handwriting or the like can be added by an information processing device to the browsing display data created from the original data (paragraph [0046] of Patent Document 1).

However, in Patent Document 1, when displaying a list of browsing display data, it is not possible to add drawing by handwriting or the like.

Therefore, in one embodiment of the present invention, it is an object to write in the list display of pages of an image processing device such as an electronic blackboard.

In order to solve the above-mentioned problems, one embodiment of the present invention generates and displays a screen of a list of a plurality of pages, and one page or two or more pages in the list may have a plurality of different uses. It is composed of the above layers, and includes a list control unit that accepts writing to the one page or the two or more pages by changing the configuration of the plurality of layers.

According to one embodiment of the present invention, writing can be performed in a list display of pages of an image processing device such as an electronic blackboard.

It is an overall block diagram of the image processing system of this embodiment. This is the hardware configuration of the image processing device of the present embodiment. It is a functional block diagram of the electronic blackboard of this embodiment. It is a functional block diagram of the file processing part of this embodiment. It is a functional block diagram of the in-device server and transmission control unit of this embodiment. It is a conceptual diagram which shows the page data of this embodiment. It is a conceptual diagram which shows the stroke arrangement data of this embodiment. It is a conceptual diagram which shows the coordinate array data of this embodiment. It is a conceptual diagram which shows the media data of this embodiment. It is a conceptual diagram which shows the remote license management table of this embodiment. It is a conceptual diagram which shows the address book management table of this embodiment. It is a conceptual diagram which shows the backup data of this embodiment. It is a conceptual diagram which shows the connection destination management table of this embodiment. It is a conceptual diagram which shows the participation base management table of this embodiment. It is a conceptual diagram which shows the operation data of this embodiment. It is a block diagram of each image layer of this embodiment. It is a sequence diagram which showed the processing of each electronic blackboard of this embodiment. It is a sequence diagram which showed the processing of each electronic blackboard of this embodiment. It is a figure which shows an example of the layer structure of the item of the page to be displayed in the list of this embodiment. It is a flowchart of page loading of this embodiment. It is a flowchart which determines the layer composition of the item of the page to be displayed in the list of this embodiment. This is an example of a screen displayed on the display of the present embodiment. This is an example of a screen displayed on the display of the present embodiment.

Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

<System overview>
FIG. 1 is an overall configuration diagram of the image processing system 1 of the present embodiment. In addition, in FIG. 1, for simplification of the explanation, only two electronic blackboards 2a and 2b and the electronic pens 4a and 4b associated therewith are shown, and three or more electronic blackboards and electronic devices are shown. You may use a pen or the like.

As shown in FIG. 1, the image processing system 1 includes a plurality of electronic whiteboards 2a, 2b, a plurality of electronic pens 4a, 4b, a USB memory 5a, 5b, a notebook PC (Personal Computer) 6a, 6b, a television ( Video) It has conference terminals 7a, 7b, and a PC8. Further, the electronic blackboards 2a and 2b and the PC 8 are communicably connected via the communication network 9. Further, the plurality of electronic blackboards 2a and 2b are provided with displays 3a and 3b, respectively.

Further, on the electronic blackboard 2a, an image drawn by an event (touch of the pen tip of the electronic pen 4a or the pen tip of the electronic pen 4a on the display 3a) caused by the electronic pen 4a is displayed on the display 3a. Can be done. In addition to the electronic pen 4a, the image displayed on the display 3a can be changed based on an event (gestures such as enlargement, reduction, page turning, etc.) generated by the user's hand Ha or the like.

Further, a USB memory 5a can be connected to the electronic whiteboard 2a, the electronic whiteboard 2a can read an electronic file such as a PDF from the USB memory 5a, and the electronic blackboard 2a can record an electronic file in the USB memory 5a. .. Further, the electronic blackboard 2a is connected to the electronic blackboard 2a via a cable 10a1 capable of communicating according to standards such as DisplayPort, DVI (Digital Visual Interface), HDMI (registered trademark) (High-Definition Multimedia Interface) and VGA (Video Graphics Array). The notebook PC 6a is connected. Then, the electronic blackboard 2a generates an event by touching the display 3a, and transmits event information indicating the event to the notebook PC 6a in the same manner as an event from an input device such as a mouse or a keyboard. Similarly, a television (video) conference terminal 7a is connected to the electronic blackboard 2a via a cable 10a2 capable of communicating according to the above standard. The notebook PC 6a and the video conferencing terminal 7a may communicate with the electronic blackboard 2a by wireless communication compliant with various wireless communication protocols such as Bluetooth (registered trademark).

On the other hand, at other bases where the electronic blackboard 2b is installed, similarly to the above, the electronic blackboard 2b equipped with the display 3b, the electronic pen 4b, the USB memory 5b, the notebook PC 6b, the video conferencing terminal 7b, the cable 10b1, and the cable 10b2 Is used. Further, the image displayed on the display 3b can be changed based on the event generated by the user's hand Hb or the like.

As a result, the image drawn on the display 3a of the electronic whiteboard 2a at one base is also displayed on the display 3b of the electronic whiteboard 2b at the other base, and conversely, on the display 3b of the electronic whiteboard 2b at the other base. The image drawn on the screen is displayed on the display 3a of the electronic whiteboard 2a at one base. As described above, since the image processing system 1 can perform remote sharing processing for sharing the same image at a remote location, it is very convenient to use it for a conference or the like at a remote location.

In the following, when any electronic blackboard among a plurality of electronic blackboards is indicated, it is referred to as "electronic blackboard 2". When indicating an arbitrary display among a plurality of displays, it is indicated as "display 3". When indicating an arbitrary electronic pen among a plurality of electronic pens, it is indicated as "electronic pen 4". When indicating an arbitrary USB memory among a plurality of USB memories, it is indicated as "USB memory 5". When indicating an arbitrary notebook PC among a plurality of notebook PCs, it is indicated as "notebook PC 6". When indicating an arbitrary video conference terminal among a plurality of video conference terminals, it is indicated as "video conference terminal 7". Further, when indicating an arbitrary hand among the hands of a plurality of users, it is indicated as "hand H". When indicating an arbitrary cable among a plurality of cables, it is indicated as "cable 10".

Further, in the present embodiment, an electronic blackboard will be described as an example of the image processing device, but the present invention is not limited to this, and other examples of the image processing device include electronic signage (digital signage), sports, weather forecast, and the like. It may be a telestrator to be used, a remote image (video) diagnostic device, or the like. Further, the notebook PC 6 will be described as an example of the information processing terminal, but the present invention is not limited to this, and other examples of the information processing terminal include a desktop PC, a tablet PC, a PDA, a digital video camera, a digital camera, and a game. It may be a terminal capable of supplying an image frame such as a machine. Further, the communication network includes the Internet, a LAN (Local Area Network), a mobile phone communication network, and the like. Further, in the present embodiment, the USB memory will be described as an example of the recording medium, but the present invention is not limited to this, and various recording media such as an SD card may be used as another example of the recording medium.

<Hardware configuration of electronic blackboard>
Subsequently, the hardware configuration of the electronic blackboard of the present embodiment will be described with reference to FIG. Note that FIG. 2 is a hardware configuration diagram of the electronic blackboard.

As shown in FIG. 2, the electronic whiteboard 2 is a CPU 101 that controls the operation of the entire electronic whiteboard 2, a ROM 102 that stores a program used to drive the CPU 101 such as an IPL, and a RAM 103 that is used as a work area of the CPU 101. , SSD 104 that stores various data such as programs for the electronic blackboard 2, a network controller 105 that controls communication with the communication network 9, and an external storage controller 106 that controls communication with the USB memory 5.

Further, the electronic blackboard 2 displays a capture device 111 that displays video information as a still image or a moving image on the display of the notebook PC 6, a GPU (Graphics Processing Unit) 112 that specializes in graphics, and an output image from the GPU. A display controller 113 that controls and manages a screen display for output to 3 or a video conference terminal 7 is provided.

Further, the electronic blackboard 2 includes a sensor controller 114 that controls the processing of the contact sensor 115, and a contact sensor 115 that detects that the electronic pen 4 or the user's hand H or the like touches the display 3. The contact sensor 115 inputs coordinates and detects coordinates by an infrared blocking method. In the method of inputting the coordinates and detecting the coordinates, two light receiving and emitting devices (not shown) installed at both upper ends of the display 3 emit a plurality of infrared rays in parallel with the display 3 and surround the display 3. This is a method of receiving light that is reflected by a reflecting member provided in the light receiving element and returns on the same optical path as the light path emitted by the light receiving element. The contact sensor 115 outputs an infrared ID (Identification) emitted by two light emitting / receiving devices blocked by the object to the sensor controller 114, and the sensor controller 114 identifies a coordinate position which is a contact position of the object. All IDs shown below are examples of identification information.

Further, the contact sensor 115 is not limited to the infrared blocking method, but is a touch panel of the capacitance type that specifies the contact position by detecting a change in capacitance, and the contact position is determined by a voltage change of two opposing resistance films. Various detection means such as a touch panel of the resistance film type to be specified and a touch panel of the electromagnetic induction type which detects the electromagnetic induction generated by the contact object coming into contact with the display unit and specifies the contact position may be used.

Further, the electronic blackboard 2 includes an electronic pen controller 116. The electronic pen controller 116 communicates with the electronic pen 4 to determine whether or not there is a touch of the pen tip or a touch of the pen tip on the display 3. The electronic pen controller 116 may determine whether or not there is a touch not only on the pen tip and pen tip of the electronic pen 4 but also on the portion gripped by the user of the electronic pen 4 and other electronic pen portions.

Further, the electronic whiteboard 2 includes the CPU 101, ROM 102, RAM 103, SSD 104, network controller 105, external storage controller 106, capture device 111, GPU 112, sensor controller 114, and electronic pen controller 116 as shown in FIG. It is provided with a bus line 120 such as an address bus and a data bus for electrically connecting.

The program for the electronic blackboard 2 may be recorded on a computer-readable recording medium such as a CD-ROM and distributed.

<Functional configuration of electronic blackboard>
Subsequently, the functional configuration of the electronic blackboard will be described with reference to FIGS. 3 to 16. First, the overall functional configuration of the electronic blackboard 2 will be described with reference to FIG. FIG. 3 is a functional block diagram of the electronic blackboard.

The electronic blackboard 2 has each functional configuration shown in FIG. 3 according to the hardware configuration and program shown in FIG. The electronic blackboard 2 can be a "sponsoring device" that first starts the remote sharing process, and can also be a "participating device" that later participates in the remote sharing process that has already been started. Further, the electronic blackboard 2 is roughly divided into both a client unit 20 and a server unit 90. The client unit 20 and the server unit 90 are functions realized in one housing of the electronic blackboard 2. When the electronic blackboard 2 serves as a host device, the client unit 20 and the server unit 90 are realized in the electronic blackboard 2. Further, when the electronic blackboard 2 serves as a participating device, the client unit 20 is realized but the server unit 90 is not realized in the electronic blackboard 2. That is, in FIG. 1, when the electronic blackboard 2a is the host device and the electronic blackboard 2b is the participating device, the client unit 20 of the electronic blackboard 2a is the other via the server unit 90 realized in the same electronic blackboard 2a. Communicates with the client unit 20 of the electronic blackboard 2b of the above. On the other hand, the client unit 20 of the electronic blackboard 2b communicates with the client unit of the other electronic blackboard 2a via the server unit 90 realized in the other electronic blackboard 2a.

[Functional configuration of client unit 20]
Subsequently, the functional configuration of the client unit 20 will be described mainly with reference to FIGS. 3 to 5. The client unit 20 includes an image acquisition unit 21, a coordinate detection unit 22, an automatic adjustment unit 23, a contact detection unit 24, an event distribution unit 25, an operation processing unit 26, a gesture processing unit 27, an image superimposition unit 28, and a finger discrimination unit 29. , An image processing unit 30, and a communication control unit 60.

Of these, the video acquisition unit 21 acquires the output video of the video output device connected to the cable 10. When the image acquisition unit 21 receives an image signal from the image output device, the image acquisition unit 21 analyzes the image signal, updates the resolution of the image frame which is the display image of the image output device formed by the image signal, and updates the image frame. Image information such as frequency is derived and output to the image acquisition unit 31.

The coordinate detection unit 22 detects the coordinate position of an event (such as an action in which the user's hand H is touched on the display 3) generated by the user on the display 3. The coordinate detection unit 22 also detects the touched area.

The automatic adjustment unit 23 is activated when the electronic blackboard 2 is activated, and adjusts the image processing parameters of the sensor camera in the optical sensor method of the contact sensor 115 so that the contact sensor 115 can output an appropriate value to the coordinate detection unit 22. To do.

The contact detection unit 24 detects an event (such as an operation in which the pen tip of the electronic pen 4 or the pen tail of the electronic pen 4 is pressed (touched) on the display 3) caused by the user.

The event distribution unit 25 distributes the coordinate position of the event detected by the coordinate detection unit 22 and the detection result detected by the contact detection unit 24 into each event of stroke drawing, UI operation, and gesture operation.

Here, in the "stroke drawing", when the stroke image (B) shown in FIG. 16 is displayed on the display 3, the user presses the electronic pen 4 on the display 3, and this pressing is performed. This is an event in which the electronic pen 4 is moved in this state, and finally the electronic pen 4 is released from the display 3. By this stroke drawing, for example, the alphabets "S" and "T" are drawn on the display 3. Note that this "stroke drawing" includes not only drawing an image, but also an event of deleting an already drawn image or editing the drawn image.

The "UI operation" is an event in which the user presses a predetermined position with the electronic pen 4 or the hand H when the UI image (A) shown in FIG. 16 is displayed on the display 3. .. By this UI operation, for example, the color and width of the line drawn by the electronic pen 4 are set.

The “gesture operation” is an event in which the user touches or moves the display 3 with his / her hand H when the stroke image (B) shown in FIG. 16 is displayed on the display 3. By this gesture operation, for example, by moving the hand H while the user touches the hand H on the display 3, the image can be enlarged (or reduced), the display area can be changed, the page can be switched, and the like. it can.

The operation processing unit 26 executes various operations according to the elements of the UI in which the event has occurred from those determined to be UI operations by the event distribution unit 25. Elements of this UI include, for example, buttons, lists, checkboxes, and text boxes.

The gesture processing unit 27 executes an operation corresponding to the gesture operation determined by the event distribution unit 25.

The image superimposing unit 28 displays the image superimposed by the display superimposing unit 36, which will be described later, as an image on the image output device (display 3, etc.). Further, the video superimposing unit 28 pictures-in-pictures the video sent from another video output device (video conference terminal 7 or the like) with respect to the video from the video output device (notebook PC 6 or the like). Further, the image superimposing unit 28 switches to display the image that has been picture-in-pictured and displayed on a part of the display 3 on the entire display 3.

The image processing unit 30 performs overlay processing and the like of each image layer as shown in FIG. The image processing unit 30 includes an image acquisition unit 31, a stroke processing unit 32, a UI image generation unit 33, a background generation unit 34, a layout management unit 35, a display superimposition unit 36, a page processing unit 37, a marker processing unit 38, and context processing. It has a unit 39, a file processing unit 40, a list control unit 41, a page data storage unit 300, and a remote license management table 310.

Of these, the image acquisition unit 31 acquires each frame as an image from the image acquired by the image acquisition unit 21. The image acquisition unit 31 outputs the data of this image to the page processing unit 37. This image corresponds to the output image (C) from the video output device (notebook PC 6 or the like) shown in FIG.

The stroke processing unit 32 draws an image, deletes the drawn image, and edits the drawn image based on the event related to the stroke drawing assigned by the event distribution unit 25. The image obtained by drawing the stroke corresponds to the stroke image (B) shown in FIG. Further, each result of drawing, deleting, and editing the image based on the stroke drawing is stored in the operation data storage unit 840 as the operation data described later.

The UI image generation unit 33 generates a UI (user interface) image preset on the electronic blackboard 2. This UI image corresponds to the UI image (A) shown in FIG.

The background generation unit 34 receives the media data of the page data read from the page data storage unit 300 by the page processing unit 37 from the page processing unit 37. The background generation unit 34 outputs the received media data to the display superimposition unit 36. Further, the image based on this media data corresponds to the background image (D) shown in FIG. The pattern of the background image (D) is plain, grid display, or the like.

The layout management unit 35 provides the display superimposition unit 36 with layout information indicating the layout of each image output from the image acquisition unit 31, the stroke processing unit 32, and the UI image generation unit 33 (or the background generation unit 34). I manage it. As a result, the layout management unit 35 causes the display superimposition unit 36 to display the output image (C) and the stroke image (B) at which position in the UI image (A) and the background image (D). You can instruct whether to hide it.

Based on the layout information output from the layout management unit 35, the display superimposition unit 36 arranges the layout of each image output from the image acquisition unit 31, the stroke processing unit 32, and the UI image generation unit 33 (background generation unit 34). Do.

The page processing unit 37 collects the data of the stroke image (B) and the data of the output image (C) into one page data and stores it in the page data storage unit 300. The data of the stroke image (B) forms a part of the page data as the stroke array data (each stroke data) indicated by the stroke array data ID shown in FIG. The data of the output image (C) forms a part of the page data as the media data indicated by the media data ID shown in FIG. Then, when this media data is read from the page data storage unit 300, it is treated as the data of the background image (D).

Further, the page processing unit 37 transmits the media data of the page data once stored to the display superimposition unit 36 via the background generation unit 34, so that the video superimposition unit 28 displays the background image (D). It can be redisplayed in 3. Further, the page processing unit 37 can return the stroke array data (each stroke data) of the page data to the stroke processing unit 32 so that the stroke can be re-edited. Further, the page processing unit 37 can also delete or duplicate the page data.

That is, the data of the output image (C) displayed on the display 3 at the time when the page processing unit 37 stores the page data in the page data storage unit 300 is temporarily stored in the page data storage unit 300, and then stored in the page data storage unit 300. When it is read from the page data storage unit 300, it is read as media data indicating the background image (D). Then, the page processing unit 37 outputs the stroke array data indicating the stroke image (B) of the page data read from the page data storage unit 300 to the stroke processing unit 32. Further, the page processing unit 37 outputs the media data indicating the background image (D) among the page data read from the page data storage unit 300 to the background generation unit 34.

The display superimposition unit 36 is from the output image (C) from the image acquisition unit 31, the stroke image (B) from the stroke processing unit 32, the UI image (A) from the UI image generation unit 33, and the background generation unit 34. The background image (D) of the above is superimposed according to the layout specified by the layout management unit 35. As a result, as shown in FIG. 16, each of the UI image (A), the stroke image (B), the output image (C), and the background image (D) is in the order in which the user can see the images even if they overlap. It is composed of Leia.

Further, the display superimposition unit 36 can switch between the image (C) and the image (D) shown in FIG. 16 and superimpose the image (A) and the image (B) exclusively. .. For example, when the image (A), the image (B), and the image (C) are initially displayed and the cable 10 between the electronic blackboard 2 and the video output device (notebook PC 6 or the like) is disconnected, By the designation of the layout management unit 35, the image (C) can be excluded from the superimposition target and the image (D) can be displayed. In this case, the display superimposing unit 36 also performs display enlargement, display reduction, and display area movement processing.

The page data storage unit 300 stores page data as shown in FIG. FIG. 6 is a conceptual diagram showing page data. The page data is data for one page (stroke array data (each stroke data) and media data) displayed on the display 3. Since there are many types of parameters included in the page data, the contents of the page data will be described here by dividing them into FIGS. 6 to 9.

As shown in FIG. 6, the page data includes a page data ID for identifying an arbitrary page, a disclosure time indicating the time when the display of this page is started, and rewriting the contents of the page by strokes, gestures, or the like. The end time indicating the time when is no longer performed, the stroke array data ID for identifying the stroke array data generated by the stroke by the electronic pen 4 or the user's hand H, and the media data ID for identifying the media data are associated with each other. Is remembered. The stroke array data is data for displaying the stroke image (B) shown in FIG. 16 described later on the display 3. The media data is data for displaying the background image (D) shown in 16 below on the display 3.

With such page data, for example, when the user draws the alphabet "S" with the electronic pen 4, it is written in one stroke, so that the alphabet [S] of one character is indicated by one stroke data ID. However, when the user draws the alphabet "T" with the electronic pen 4, it is written in two strokes, so that the alphabet "T" of one character is indicated by two stroke data IDs.

In addition, the stroke array data shows detailed information as shown in FIG. FIG. 7 is a conceptual diagram showing stroke array data. As shown in FIG. 7, one stroke array data is represented by a plurality of stroke data. Then, one stroke data includes a stroke data ID for identifying the stroke data, a start time indicating the writing start time of one stroke, an end time indicating the writing end time of one stroke, and a stroke color. The width of the stroke and the coordinate array data ID for identifying the array of passing points of the stroke are shown.

Further, this coordinate array data shows detailed information as shown in FIG. FIG. 8 is a conceptual diagram showing coordinate array data. As shown in FIG. 8, the coordinate array data is one point (X coordinate value, Y coordinate value) on the display 3, and the time (ms) of the difference from the disclosure time of the stroke when passing through this one point. ), And each information of the pen pressure of the electronic pen 4 at this one point is shown. That is, a set of one point shown in FIG. 8 is shown by one coordinate array data shown in FIG. 7. For example, when the user draws the alphabet "S" with the electronic pen 4, it is written in one stroke, but since the user passes through a plurality of passing points by the time the drawing of the "S" is completed, the coordinate array data has these multiple passing points. Information is shown.

Further, the media data among the page data shown in FIG. 6 shows detailed information as shown in FIG. FIG. 9 is a conceptual diagram showing media data. As shown in FIG. 9, as for the media data, the media data ID in the page data shown in FIG. 6, the data type of the media data, and the page data are stored in the page data storage unit 300 from the page processing unit 37. The recorded time, the position (X coordinate value, Y coordinate value) of the image displayed on the display 3 by the page data, the size (width, height) of the image, and the data indicating the contents of the media data are shown in association with each other. Has been done. Of these, the position of the image displayed on the display 3 by the page data is displayed by the page data when the coordinates of the upper left corner of the display 3 are (X coordinate value, Y coordinate value) = (0,0). It shows the position of the upper left corner of the image to be displayed.

Further, returning to FIG. 3, the remote license management table 310 manages the license data necessary for executing the remote sharing process. In the remote license management table 310, as shown in FIG. 10, the product ID of the electronic blackboard 2, the license ID used for authentication, and the expiration date of the license are managed in association with each other.

(Functional configuration of file processing unit 40)
Subsequently, the functional configuration of the file processing unit 40 shown in FIG. 3 will be described with reference to FIG. Note that FIG. 4 is a functional block diagram of the file processing unit.

The file processing unit 40 includes a recovery unit 41, a file input unit 42a, a file output unit 42b, a file conversion unit 43, a file transmission unit 44, an address book input unit 45, a backup processing unit 46, a backup output unit 47, and a setting management unit 48. It has a setting file input unit 49a and a setting file output unit 49b. Further, the file processing unit 40 has an address book management table 410, a backup data storage unit 420, a setting file storage unit 430, and a connection destination management table 440.

Of these, the recovery processing unit 41 detects the abnormal termination after the electronic blackboard 2 has abnormally terminated, and recovers the unsaved page data. For example, in the case of normal termination, the page data is recorded as a PDF file in the USB 5 via the file processing unit 40, but in the case of abnormal termination such as when the power is turned off, the page data is recorded in the page data storage unit 300. It remains to be done. Therefore, when the power is turned on again, the recovery processing unit 41 recovers by reading the page data from the page data storage unit 300.

The file input unit 42a reads a PDF file from the USB memory 5 and stores each page as page data in the page data storage unit 300.

The file conversion unit 43 converts the page data stored in the page data storage unit 300 into a PDF format file.

The file output unit 42b records the PDF file output by the file conversion unit 42 in the USB memory 5.

The file transmission unit 44 attaches the PDF file generated by the file conversion unit 43 to an e-mail and transmits the PDF file. The destination of this file is that the display superimposition unit 36 displays the contents of the address book management table 410 on the display 3, and the file transmission unit 44 accepts the selection of the destination by the operation of an input device such as a touch panel by the user. It is determined. In the address book management table 410, as shown in FIG. 11, the name of the destination and the e-mail address of the e-mail of the destination are managed in association with each other. Further, the file transmission unit 44 can also accept the input of the e-mail address as the destination by the operation of the input device such as the touch panel by the user.

The address book input unit 45 reads a list file of e-mail addresses from the USB memory 5 and manages them in the address book management table 410.

The backup processing unit 46 backs up the file output by the file output unit 42b and the file transmitted by the file transmission unit 44 by storing it in the backup data storage unit 420. If the user does not set the backup, the backup process will not be performed. The backup data is stored in PDF format as shown in FIG.

The backup output unit 47 stores the backed up file in the USB memory 5. At the time of this storage, a password is input for security by the operation of an input device such as a touch panel by the user.

The setting management unit 48 manages the electronic blackboard 2 by storing and reading various setting information in the setting file storage unit 430. Examples of the various setting information include network settings, date and time settings, region and language settings, mail server settings, address book settings, connection destination list settings, and backup settings. The network settings include, for example, the IP address setting of the electronic blackboard 2, the netmask setting, the default gateway setting, the DNS (Domain Name System) setting, and the like.

The setting file output unit 49b records various setting information of the electronic blackboard 2 in the USB memory 5 as a setting file. Due to the security of the configuration file, the user cannot see the contents.

The setting file input unit 49a reads the setting file stored in the USB memory 5 and reflects various setting information in various settings of the electronic blackboard.

The address book input unit 50 reads a list file of connection destination IP addresses for remote sharing processing from the USB memory 5 and manages them in the connection destination management table 440. As shown in FIG. 13, when the electronic blackboard 2 is a participating device that intends to participate in the remote sharing process, the user of the participating device serves as a host device in the connection destination management table 440. This is a table for managing in advance in order to reduce the trouble of inputting the IP address of the electronic blackboard. In the connection destination management table 440, the name of the base where the electronic blackboard 2 as the host device that can participate is installed and the IP address of the electronic blackboard 2 as the host device are managed in association with each other.

The connection destination management table 440 may not be provided. However, in this case, the user of the participating device needs to input the IP address of the host device by an input device such as a touch panel in order to start remote request processing with the host device. Therefore, the user of the participating device obtains the IP address of the host device from the user of the host device by telephone, e-mail, or the like.

The list control unit 41 of the image processing unit 30 controls the list of pages. Specifically, the list control unit 41 generates and displays a page list screen when an icon or the like for displaying a page list is pressed by the user. The list control unit 41 receives data edited by the stroke processing unit 32 and the marker processing unit 38 (that is, writing data such as handwriting), and lists items (elements constituting the list, for example, one item). Performs the process of adding on a page (or two or more pages). Data related to the contents written from the other connected electronic blackboard 2 and the operation on the display page are sent from the communication control unit 60 and received by the image processing unit 30. The list control unit 41 receives the processed writing and operation information, and reflects the contents in the list items. As described above, in one embodiment of the present invention, the list control unit 41 can display a plurality of pages in a list, and the user can write on those pages by handwriting or the like.

(Functional configuration of communication control unit 60)
Next, the functional configuration of the communication control unit 60 will be described with reference to FIG.

The communication control unit 60 controls communication with another electronic blackboard 2 and communication with the communication control unit 70 described later in the server unit 90 via the communication network 9. Therefore, the communication control unit 60 includes a remote start processing unit 61, a remote participation processing unit 62, a remote image transmission unit 63, a remote image reception unit 64, a remote control transmission unit 65, a remote control reception unit 66, and a participation base management table 610. have.

Of these, the remote start processing unit 61 makes a new request to start the remote sharing process to the server unit 90 of the same electronic blackboard 2, and receives the result of the request from the server unit 90. In this case, the remote start processing unit 61 refers to the remote license management table 310 and requests to start the remote sharing process when the license information (product ID, license ID, and expiration date) is managed. be able to. However, if the license information is not managed, the request to start the remote sharing process cannot be made.

The participation base management table 610 is a table that manages the electronic blackboard as a participating device that is currently participating in the remote sharing process when the electronic blackboard is the host device. In the participating base management table 610, as shown in FIG. 14, the name of the base where the participating electronic blackboard 2 is installed and the IP address of the electronic blackboard 2 are managed in association with each other.

The remote participation processing unit 62 performs remote sharing processing for the remote connection request receiving unit 71 in the server unit 90 of the electronic blackboard 2 as a host device that has already started the remote sharing processing via the communication network 9. Request participation. In this case as well, the remote participation processing unit 62 refers to the remote license management table 310. Further, when the remote participation processing unit 62 participates in the remote sharing processing that has already been started, the remote participation processing unit 62 refers to the connection destination management table 440 and acquires the IP address of the electronic blackboard 2 of the participation destination. The remote participation processing unit 62 may not refer to the connection destination management table, and the IP address of the participation destination electronic blackboard 2 may be input by the user operating an input device such as a touch panel.

The remote image transmission unit 63 transmits the output image (C) sent from the video acquisition unit 21 via the image acquisition unit 31 to the server unit 90.

The remote image receiving unit 64 receives image data from a video output device connected to another electronic blackboard 2 from the server unit 90 and outputs the image data to the display superimposing unit 36, thereby enabling remote sharing processing.

The remote control transmission unit 65 transmits various operation data necessary for the remote sharing process to the server unit 90. These various operation data include, for example, adding strokes, deleting strokes, editing strokes (enlarging, reducing, moving), storing page data, creating page data, duplicating page data, deleting page data, and displaying them. There is data on switching pages. Further, the remote control receiving unit 66 receives the operation data input by the other electronic blackboard 2 from the server unit 90 and outputs the operation data to the image processing unit 30, thereby performing the remote sharing process.

[Functional configuration of server section]
Subsequently, the functional configuration of the server unit 90 will be described with reference to FIG. The server unit 90 is provided on each electronic blackboard 2, and any electronic blackboard 2 can serve as a server unit. Therefore, the server unit 90 has a communication control unit 70 and a data management unit 80.

(Functional configuration of communication control unit 70)
Next, the functional configuration of the communication control unit 70 will be described with reference to FIG.

The communication control unit 70 controls communication with the communication control unit 70 in the client unit 20 in the same electronic blackboard 2 and the communication control unit 70 in the client unit 20 in the other electronic blackboard 2 via the communication network 9. The data management unit 80 manages operation data, image data, and the like.

More specifically, the communication control unit 70 includes a remote connection request receiving unit 71, a remote connection result transmitting unit 72, a remote image receiving unit 73, a remote image transmitting unit 74, a remote controlled receiving unit 75, and a remote controlled transmitting unit 76. have.

Of these, the remote connection request receiving unit 71 receives a start request for remote sharing processing from the remote start processing unit 61, and receives a participation request for remote sharing processing from the remote participation processing unit 62. The remote connection result transmission unit 72 transmits the result of the start request for the remote sharing process to the remote start processing unit 61, or transmits the result of the participation request for the remote sharing process to the remote participation processing unit 62.

The remote image receiving unit 73 receives the image data (data of the output image (C)) from the remote image transmitting unit 63 and transmits it to the remote image processing unit 82 described later. The remote image transmission unit 74 receives image data from the remote image processing unit 82, and transmits this image data to the remote image reception unit 64.

The remote control receiving unit 75 receives the operation data (data such as the stroke image (B)) from the remote control transmitting unit 65 and transmits it to the remote control processing unit 83 described later. The remote control transmission unit 76 receives the operation data from the remote control processing unit 83, and transmits the operation data to the remote control reception unit 66.

(Functional configuration of data management department)
Next, the functional configuration of the data management unit 80 will be described with reference to FIG.

The data management unit 80 includes a remote connection processing unit 81, a remote image processing unit 82, a remote operation processing unit 83, an operation composition processing unit 84, and a page processing unit 85. Further, the server unit 90 has a passcode management unit 810, a participation base management table 820, an image data storage unit 830, an operation data storage unit 840, and a page data storage unit 850.

Of these, the remote connection processing unit 81 starts the remote sharing process and ends the remote sharing process. Further, the remote connection processing unit 81 includes the license information received by the remote connection request receiving unit 71 together with the remote sharing processing start request from the remote start processing unit 61, or the participation request for the remote sharing processing from the remote participation processing unit 62. Based on the received license information, check whether there is a license and whether it is within the license period. Further, the remote connection processing unit 81 confirms whether the participation request from the other electronic blackboard 2 as the client unit exceeds the predetermined number of possible participations.

Further, in the remote connection processing unit 81, the passcode sent when the other electronic blackboard 2 requests participation in the remote sharing processing is the same as the passcode managed by the passcode management unit 810. Judge whether or not there is, and if they are the same, allow participation in remote sharing processing. This passcode is issued by the remote connection processing unit 81 when a new remote sharing process is started, and the user of the electronic blackboard 2 as a participating device who intends to participate in the remote sharing process acts as a host device. It is transmitted from the user of the electronic blackboard 2 by telephone, e-mail, or the like. As a result, the user of the participating device who intends to participate in the remote sharing process is permitted to participate by inputting the passcode into the participating device using an input device such as a touch panel and requesting participation. It should be noted that the user's usability may be prioritized over security, and the passcode confirmation may be omitted only by confirming the license status.

When the electronic blackboard 2 is the host device, the remote connection processing unit 81 sends the participation base information included in the participation request sent from the remote participation processing unit 62 of the participating device via the communication network 9 to the server unit. It is stored in the participation base management table 820 of 90. Then, the remote connection processing unit 81 reads the remote base information stored in the participating base management table 820 and transmits it to the remote connection result transmission unit 72. The remote connection result transmission unit 72 transmits remote base information to the remote start processing unit 61 in the client unit 20 of the same host device. The remote start processing unit 61 stores remote base information in 610 in the participating base management table. As a result, in the host device, the remote base information is managed by both the client unit 20 and the server unit 90.

The remote image processing unit 82 is image data from a video output device (notebook PC 6 or the like) connected to a client unit (including a client unit of its own electronic whiteboard, which is a host device) of each electronic whiteboard 2 during remote sharing processing. (Output image (C)) is received and stored in the image data storage unit 830, and the display order of image data to be remotely shared in the order of time when it reaches the server unit 90 of its own electronic whiteboard 2 which is the host device. To judge. Further, the remote image processing unit 82 refers to the participating base management table 820 to the client unit 20 of all the electronic blackboards 2 participating in the remote sharing process (including the client unit of the own electronic blackboard which is the host device). , Image data is transmitted in the order determined above via the communication control unit 70 (remote image transmission unit 74).

The remote operation processing unit 83 includes various operation data (stroke image (B)) such as a stroke image drawn by the client unit (including the client unit of its own electronic whiteboard, which is the host device) of each electronic whiteboard 2 during remote sharing processing. ) Etc.) are received, and the display order of the images to be remotely shared is determined in the order of time when they arrive at the server unit 90 of the own electronic whiteboard 2 which is the host device. The various operation data are the same as the above-mentioned various operation data. In addition, the remote control processing unit 83 refers to the participating base management table 820, and operates data in the client unit 20 (including the client unit of its own electronic blackboard, which is the host device) of all the electronic blackboards 2 undergoing remote sharing processing. To send.

The operation synthesis processing unit 84 synthesizes the operation data of each electronic blackboard 2 output from the remote operation processing unit 83, stores the operation data as a result of the synthesis in the operation data storage unit 840, and also stores the operation data in the operation data storage unit 83. Return to. This operation data is transmitted from the remote operation transmission unit 76 to the client unit of the electronic blackboard which is the host device and the client unit of the electronic blackboard which is the participating device, so that the same operation data can be obtained on each electronic blackboard 2. The relevant image is displayed. Therefore, as shown in FIG. 15, the operation data includes SEQ (Sequence), the operation name of the operation data, the IP address of the electronic blackboard 2 which is the source of the operation data, and the port of the client unit (server unit). No., the IP address of the electronic blackboard 2 to which the operation data is sent, the Port No. of the client part (server part), the operation type of the operation data, the operation target of the operation data, and the data indicating the contents of the operation data are associated with each other. Is shown. For example, in SEQ1, when a stroke is drawn on the client part (Port No .: 50001) of the electronic blackboard (IP address: 192.0.0.1) which is the host device, the electronic blackboard (IP address: 192.0.0.1) which is the same host device. It is shown that the operation data was sent to the server section (Port No .: 50000) of 0.1). In this case, the operation type is "STROKE", the operation target is the page data ID "p005", and the data indicating the content of the operation data is the data indicating the stroke. In SEQ2, from the server section (Port No .: 50000) of the electronic blackboard (IP address: 192.0.0.1), which is the host device, to the client section of the other electronic blackboard (IP address: 192.0.0.1), which is the participating device. It is shown in (Port No .: 50001) that the operation data was sent.

Since the operation composition processing unit 84 synthesizes the operation data in the order in which the operation data is input to the operation composition processing unit 84, if the communication network 9 is not congested, the operation composition processing unit 84 is remotely shared in the stroke order by the user of each electronic blackboard 2. The stroke image (B) is displayed on the display 3 of all the electronic blackboards 2 being processed.

The page processing unit 85 has the same function as the page processing unit 37 in the image processing unit 30 of the client unit 20, and the server unit 90 also stores the page data shown in FIGS. 6 to 8 in the page data storage unit. Store in 850. Since the page data storage unit 850 has the same contents as the page data storage unit 300 in the image processing unit 30, the description thereof will be omitted.

<Processing or operation of the embodiment>
Subsequently, the process or operation of the present embodiment will be described with reference to FIGS. 17 and 18. 17 and 18 are sequence diagrams showing the processing of each electronic blackboard.

In the embodiments shown in FIGS. 17 and 18, the electronic blackboard 2a plays a role as a host device (server unit and client unit) for hosting the remote sharing process, and the electronic blackboards 2b and 2c participate in the remote sharing process. A case of playing a role as a device (client unit) will be described. Further, here, the displays 3a, 3b, and 3c are connected to the electronic blackboards 2a, 2b, and 2c, respectively, and the notebook PCs 6a, 6b, and 6c are connected, respectively. Further, in the electronic blackboards 2a, 2b, 2c, the electronic pens 4a, 4b, 4c are used, respectively.

(Participation processing)
First, the process for the electronic blackboards 2b and 2c to participate in the remote sharing process will be described with reference to FIG.

When the user turns on the power switch of the electronic blackboard 2a, the client unit 20 of the electronic blackboard 2a is activated. Then, when the user activates the server unit 90 by an input device such as a touch panel, the remote start processing unit 61 of the client unit 20 sends the server unit 90 to the remote connection request receiving unit 71 of the server unit 90 of the same electronic blackboard 2a. Outputs an instruction to start the processing of. As a result, in the electronic blackboard 2a, not only the client unit 20 but also the server unit 90 can start various processes (step S21).

Next, the UI image generation unit 33 in the client unit 20 of the electronic blackboard 2a generates connection information for establishing a connection with the electronic blackboard 2a, and the image superimposition unit 28 is displayed from the UI image generation unit 33 to the display superimposition unit 36. The connection information obtained through the above is displayed on the display 3a (step S22). This connection information includes the IP address of the host device and the passcode generated for this remote sharing process. In this case, the passcode stored in the passcode management unit 810 is read by the remote connection processing unit 81 shown in FIG. 5 and transmitted in the order of the remote connection result transmission unit 72 and the remote start processing unit 61. To. Further, the passcode is transmitted from the communication control unit 60 including the remote start processing unit 61 to the image processing unit 30 shown in FIG. 3, and finally input to the UI image generation unit 33. As a result, the connection information includes the passcode. Then, the connection information is transmitted to the users of the electronic blackboards 2b and 2c by the user of the electronic blackboard 2a by telephone or e-mail. If the connection destination management table 440 is provided, the participating device can make a participation request even if the connection information does not include the IP address of the host device.

Next, in the electronic whiteboards 2b and 2c, when the input of the connection information is received by each user by operating an input device such as a touch panel, the remote participation processing unit 62 in the client unit 20 of the electronic whiteboards 2a and 2b receives the connection information. Based on the IP address, a passcode is transmitted to the communication control unit 70 in the server unit 90 of the electronic blackboard 2a via the communication network 9 to request participation (steps S23 and S24). As a result, the remote connection request receiving unit 71 of the communication control unit 70 receives the participation request (including the passcode) from the electronic blackboards 2b and 2c, and outputs the passcode to the remote connection processing unit 81.

Next, the remote connection processing unit 81 authenticates the passcodes received from the electronic blackboards 2b and 2c using the passcodes managed by the passcode management unit 810 (step S25). Then, the remote connection result transmission unit 72 notifies the client unit 20 of each of the electronic blackboards 2b and 2c of the authentication result (steps S26 and S27). If it is determined by the authentication in step S25 that each of the electronic blackboards 2b and 2c is a legitimate electronic blackboard, the electronic blackboard 2a, which is the host device, and the electronic blackboards 2b, 2c, which are the participating devices, are remotely shared. Communication of processing is established, and the remote participation processing unit 62 in the client unit 20 of each of the electronic blackboards 2b and 2c enables the start of remote sharing processing with other electronic blackboards (steps S28 and S29).

(Display of output image)
Subsequently, the process of displaying the output image (C) in the remote sharing process will be described with reference to FIG.

First, the electronic blackboard 2b displays the output image (C) on the display 3b (step S30). Specifically, the image acquisition unit 31 of the electronic blackboard 2b receives the data of the output image (C) displayed on the notebook PC 6b from the notebook PC 6b via the image acquisition unit 21, and the display superimposition unit 36 and the image The display 3b displays the output image (C) by transmitting the data to the display 3b via the superimposing unit 28.

Next, the image processing unit 30 including the image acquisition unit 31 of the electronic blackboard 2b transmits the data of the output image (C) to the remote image transmission unit 63, so that the communication control unit 60 including the remote image transmission unit 63 The data of the output image (C) is transmitted to the communication control unit 70 of the electronic blackboard 2a, which is the host device, via the communication network 9 (step S31). As a result, the remote image receiving unit 73 of the electronic blackboard 2a receives the data of the output image (C) and outputs it to the remote image processing unit 82, so that the remote image processing unit 82 outputs the image to the image data storage unit 830. The data of (C) is stored.

Next, the electronic blackboard 2a, which is the host device, displays the output image (C) on the display 3a (step S32). Specifically, the remote image processing unit 82 of the electronic blackboard 2a outputs the data of the output image (C) received from the remote image receiving unit 73 to the remote image transmitting unit 74. The remote image transmission unit 74 outputs the data of the output image (C) to the remote image reception unit 64 in the client unit 20 of the electronic blackboard 2a, which is the same host device. The remote image receiving unit 64 outputs the data of the output image (C) to the display superimposing unit 36. The display superimposition unit 36 outputs the data of the output image (C) to the image superimposition unit 28. The image superimposing unit 28 outputs the data of the output image (C) to the display 3a. As a result, the display 3a displays the output image (C).

Next, the communication control unit 70 including the remote image transmission unit 74 in the server unit 90 of the electronic blackboard 2a as the host device uses the electronic blackboard 2b, which is the source of the data of the output image (C), via the communication network 9. The data of the output image (C) is transmitted to the communication control unit 60 of the electronic blackboard 2c other than the above (step S33). As a result, the remote image receiving unit 64 of the electronic blackboard 2c, which is the participating device, receives the data of the output image (C).

Next, the electronic blackboard 2c displays the output image (C) on the display 3c (step S34). Specifically, the remote image receiving unit 64 of the electronic blackboard 2c outputs the data of the output image (C) received in step S33 to the display superimposing unit 36 of the electronic blackboard 2c. The display superimposition unit 36 outputs the data of the output image (C) to the image superimposition unit 28. The image superimposing unit 28 outputs the data of the output image (C) to the display 3c. As a result, the display 3c displays the output image (C).

When not only the data of the output image (C) but also the data of the UI image (A) and the stroke image (B) are input to the display composition unit 28, they are superimposed by the display superimposition unit 36. Images (A, B, C) are generated, and the image superimposing unit 28 outputs the data of the superposed images (A, B, C) to the display 3c. Further, when the video (E) data for the video conference is sent from the video conferencing terminal 7 to the video superimposing unit 28, the video superimposing unit 28 is used for the superposed image (A, B, C). The video (E) data for video conferencing is superimposed by picture-in-picture and output to the display 3c.

(Display of superimposed image)
Subsequently, the process of displaying the superimposed image in the remote sharing process will be described with reference to FIG.

First, in the electronic blackboard 2b, the user draws a stroke image (B) on the electronic blackboard 2b using the electronic pen 4b (step S41).

Next, as shown in FIG. 16, the display superimposition unit 36 of the electronic blackboard 2b superimposes the stroke image (B) on the UI image (A) and the output image (C), and superimposes the image. The unit 28 displays the superimposed image (A, B, C) on the display 3b of the electronic blackboard 2b (step S42). Specifically, the stroke processing unit 32 of the electronic blackboard 2b receives the data of the stroke image (B) as the operation data from the coordinate detection unit 22 and the contact detection unit 24 via the event distribution unit 25, and displays and superimposes the data. It is transmitted to the unit 36. As a result, the display superimposing unit 36 can superimpose the stroke image (B) on the UI image (A) and the output image (C), and the image superimposing unit 28 is placed on the display 3b of the electronic blackboard 2b. The superimposed image (A, B, C) can be displayed.

Next, the image processing unit 30 including the stroke processing unit 32 of the electronic whiteboard 2b transmits the data of the stroke image (B) to the remote operation transmission unit 65, so that the remote operation transmission unit 65 of the electronic blackboard 2b communicates. The stroke image (B) data is transmitted to the communication control unit 70 of the electronic blackboard 2a, which is the host device, via the network 9 (step S43). As a result, the remote control receiving unit 75 of the electronic blackboard 2a receives the data of the stroke image (B) and outputs it to the remote control processing unit 83, so that the remote control processing unit 83 sends the stroke image to the operation composition processing unit 84. Output the data of (B). In this way, the data of the stroke image (B) drawn on the electronic blackboard 2b is sequentially transmitted to the remote control processing unit 83 of the electronic blackboard 2a, which is the host device, each time it is drawn. The data of the stroke image (B) is the data shown for each stroke data ID shown in FIG. 7. Therefore, for example, as described above, when the user draws the alphabet "T" with the electronic pen 4, it is written in two strokes, so that the data of the stroke image (B) indicated by each of the two stroke data IDs is sequentially transmitted. Will be done.

Next, the electronic blackboard 2a, which is the host device, displays the superimposed image (A, B, C) including the data of the stroke image (B) sent from the electronic blackboard 2b on the display 3a (step). S44). Specifically, the operation composition processing unit 84 of the electronic blackboard 2a synthesizes the data of a plurality of stroke images (B) sequentially sent via the remote operation processing unit 83, and causes the operation data storage unit 840. It is stored and returned to the remote control processing unit 83. As a result, the remote control processing unit 83 outputs the data of the stroke image (B) after synthesis received from the operation synthesis processing unit 84 to the remote control transmission unit 76. The remote control transmission unit 76 outputs the data of the stroke image (B) after synthesis to the remote control reception unit 66 in the client unit 20 of the electronic blackboard 2a, which is the same host device. The remote control receiving unit 66 outputs the data of the stroke image (B) after composition to the display superimposing unit 36 in the image processing unit 30. Therefore, the display superimposition unit 36 superimposes the combined stroke image (B) on the UI image (A) and the output image (C). Finally, the image superimposing unit 28 causes the superposed image (A, B, C) superimposed by the display superimposing unit 36 to be displayed on the display 3a.

Next, the communication control unit 70 including the remote control transmission unit 76 in the server unit 90 of the electronic blackboard 2a as the host device uses the electronic blackboard 2b which is the transmission source of the stroke image (B) data via the communication network 9. The data of the stroke image (B) after composition is transmitted to the communication control unit 60 of the electronic blackboard 2c other than the above (step S45). As a result, the remote control receiving unit 66 of the electronic blackboard 2c, which is the participating device, receives the data of the stroke image (B) after the composition.

Next, the electronic blackboard 2c displays the superimposed image (A, B, C) on the display 3c (step S34). Specifically, the remote control receiving unit 66 of the electronic blackboard 2c outputs the data of the stroke image (B) after the composition received in step S45 to the image processing unit 30 of the electronic blackboard 2c. The display superimposition unit 36 of the image processing unit 30 superimposes the data of the UI image (A) and the output image (C) and the data of the stroke image (B) after composition, and superimposes the superimposed image (on top of the image superimposing unit 28). Output the data of A, B, C). The image superimposing unit 28 outputs the data of the superposed image (A, B, C) to the display 3c. As a result, the display 3c displays the superimposed image (A, B, C).

In the above process, the output image (C) is displayed on the display 3, but the background image (D) may be displayed instead of the output image (C). Further, the exclusive relationship between the output image (C) and the background image (D) may be stopped, and both the output image (C) and the background image (D) may be displayed on the display 3 at the same time.

(End of participation)
Subsequently, the process of terminating the participation of the participating device in the remote sharing process will be described with reference to FIG. In the embodiment shown in FIG. 18, the process of terminating the participation of the electronic blackboard 2c is shown.

First, when the electronic blackboard 2c receives the participation end request by the operation of an input device such as a touch panel by the user, the remote participation processing unit 62 sends the communication control unit 70 in the server unit 90 of the electronic blackboard 2a as the host device to the communication control unit 70. Request the end of participation (step S47). As a result, the remote connection request receiving unit 71 of the communication control unit 70 receives the participation end request from the electronic blackboard 2c, and outputs the participation end request to the remote connection processing unit 81 together with the IP address of the electronic blackboard 2c. .. Then, the remote connection processing unit 81 of the electronic whiteboard 2a requests the end of participation from the participation base management table 820 based on the IP address sent from the remote connection request receiving unit 71, and the IP of the electronic whiteboard 2c. The address and the name of the base where the electronic whiteboard 2c is installed are deleted, and the IP address of the electronic whiteboard 2c and a notification to that effect are output to the remote connection result transmission unit 72.

Next, the communication control unit 70 including the remote connection result transmission unit 72 instructs the communication control unit 60 in the client unit 20 of the electronic blackboard 2c to end the participation via the communication network 9 (step S48). As a result, the remote participation processing unit 62 of the communication control unit 60 on the electronic blackboard 2c disconnects the communication of the remote sharing processing, thereby performing the participation termination processing, and the participation ends (step S49).

FIG. 19 is a diagram showing an example of the layer structure of the items of the page to be displayed in the list. As shown in FIG. 19, a transparent button layer for accepting operations on the page, a stroke information layer for reflecting both writings, and a background image layer for displaying the background information of the page. Consists of ,. Here, the stroke information layer is such that the stroke information can be directly edited, for example, InkCanvas, which is a function of Microsoft's WPF (Windows Presentation Foundation). Further, in order to allow the user to know which page in the list is shared with other electronic blackboards, the list control unit 41 acquires the information of the shared page from the page processing unit 37 and frames the item. Highlight.

FIG. 20 is a flowchart of page loading. At the start of displaying the page list screen, the flow of FIG. 20 is performed for all the pages to be retained. The transparent button layer and the stroke information layer are configured to be possessed by the items of all pages (steps S201 and S202), and the background image layer is configured to be possessed only by the items of the page for which the background is set on the page. There is (step S203). If there is a stroke written on the page, the stroke is drawn on the stroke information layer. When the background image layer is created, the background image is acquired from the page and drawn (step S204). By displaying the layered items on the screen in a list in this way, it is possible to display a list of all pages (step S205).

If there is a request to add a page from another electronic blackboard or PC while the page list is being displayed on the electronic blackboard, the flow in FIG. 20 is processed based on the request information to display the page list on the screen. Add the information of the newly added page to. Similarly, when a stroke is written or deleted from another electronic blackboard while the page list is displayed, the stroke information layer is updated in response to the stroke change. If a background image is set later for a page that does not have a background information layer, a new background image layer is created for the item on that page and placed at the back.

FIG. 21 is a flowchart for determining the layer configuration of the items on the page to be displayed in the list.
-In step 211 (S211), when the process of changing the state of the button is executed by using the trigger of the button or the like, it is determined whether or not it is in the state of confirming the page contents. If the page content is confirmed, the process proceeds to step 212. If the page content is not confirmed, the process proceeds to step 213.
-In step 212 (S212), when the page content is confirmed, the button layer is brought to the foreground. By doing so, you will be able to receive click events from users and perform operations on the page.
-In step 213 (S213), when the page content is not confirmed, the stroke information layer is brought to the foreground. By doing so, it becomes possible to directly write to the page from within the list display.

22 and 23 are examples of screens displayed on the display. It is assumed that the trigger for changing the layer configuration can be applied to each page as shown in FIG. 22 or can be applied to the entire list as shown in FIG. 23.

In one embodiment of the present invention, the items to be displayed on the list are configured by a plurality of layers having different uses, and the configuration is dynamically changed. Therefore, the user can browse, add, or duplicate a page as a list function. It is possible to write on the page without impairing the function of performing operations such as.

As described above, in one embodiment of the present invention, writing can be performed in the list display of pages of an image processing device such as an electronic blackboard. Therefore, the participants of the conference can write and discuss while checking the previous and next pages (that is, the discussion is continued even while the page list is displayed).

Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" in the present specification is a processor programmed to execute each function by software such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. It shall include devices such as ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array) and conventional circuit modules.

The present invention is not limited to the configurations shown here, such as combinations with other elements in the configurations and the like described in the above embodiments. These points can be changed without departing from the spirit of the present invention, and can be appropriately determined according to the application form thereof.

1 Image processing system 2 Electronic blackboard (image processing device)
3 Display 4 Electronic pen 5 USB memory 6 Notebook PC
7 Video conferencing terminal 8 PC
9 Communication network 10 Cable 20 Client unit 21 Image acquisition unit 22 Coordinate detection unit 23 Automatic adjustment unit 24 Contact detection unit 25 Event distribution unit 26 Operation processing unit 27 Gesture processing unit 28 Image superimposition unit 29 Finger discrimination unit 30 Image processing unit 31 Image acquisition unit 32 Stroke processing unit 33 UI image generation unit 34 Background generation unit 35 Layout management unit 36 Display superimposition unit 37 Page processing unit 38 Marker processing unit 39 Context processing unit 40 File processing unit 41 List control unit 300 Page data storage unit 310 Remote license management table 60 Communication control unit 70 Communication control unit 80 Data management unit 90 Server unit

Japanese Unexamined Patent Publication No. 2015-138446

Claims (6)

A screen of a list of a plurality of pages is generated and displayed, and one page or two or more pages in the list is composed of a plurality of layers having different uses.
An image processing device including a list control unit that accepts writing on the one page or the two or more pages by changing the configuration of the plurality of layers. The first aspect of claim 1, wherein the plurality of layers include a layer of transparent buttons for receiving operations on the one page or the two or more pages, and a layer of stroke information for reflecting the writing. Image processing equipment. The image processing apparatus according to claim 2, wherein changing the configuration of the plurality of layers brings the layer of stroke information for reflecting the writing to the foreground. The image processing apparatus according to any one of claims 1 to 3, wherein the plurality of layers include a layer of a background image for displaying background information of the one page or the two or more pages. A step of generating and displaying a screen of a list of a plurality of pages, wherein one page or two or more pages in the list is composed of a plurality of layers having different uses.
A method comprising a step of accepting a write on the one page or the two or more pages by changing the configuration of the plurality of layers. A computer is generated to display a screen of a list of a plurality of pages, and one page or two or more pages in the list are composed of a plurality of layers having different uses.
A program for functioning as a list control unit that accepts writing to the one page or the two or more pages by changing the configuration of the plurality of layers.

Images