JP2021193774A - Electronic blackboard, voice playback method, program, and image processing system - Google Patents

Abstract

To provide an electronic blackboard that can reduce time and effort to display input data input by a user and playback voice data related to the input data.SOLUTION: An electronic blackboard displays input data input by a user, and has: display processing means that reads out, from a data recording unit that stores user identification information for identifying a user, input data, and voice data related to the input data in association with each other, input data associated with user identification information on a user who has been successfully authenticated and causes a video display unit to display the input data; and voice processing means that reads out, from the data recording unit, voice data related to the input data associated with the user identification information on the user who has been successfully authenticated and causes a voice playback unit to playback the voice data.SELECTED DRAWING: Figure 4

Description

The present invention relates to an electronic blackboard, an audio reproduction method, a program and an image processing system.

In recent years, an electronic blackboard in which a touch panel is mounted on a large flat panel display has been known. The electronic blackboard displays the screen on the flat panel display, captures the locus of an electronic pen or a finger via the touch panel, and draws the locus on the screen of the flat panel display as handwriting input.

For example, in a lesson or the like, a data recording device and a data reproduction device capable of automatically associating and recording the board writing data displayed two-dimensionally on an electronic blackboard or the like and stream data including audio and reproducing the lesson contents are available. It has been known conventionally (see, for example, Patent Document 1).

However, in the conventional electronic blackboard, the user acquires the data of the lesson contents recorded in the past and displays it on the electronic blackboard, and then selects the desired lesson contents by himself and reproduces the writing on the board and the voice, so that the data is recorded. There is no mechanism for efficiently searching the lesson content that the user wants to reproduce from the lesson content, and there is a problem that it takes time and effort to reproduce the desired lesson content.

It is an object of the present invention to provide an electronic blackboard that can reduce the trouble of displaying input data input by a user and reproducing voice data related to the input data.

In order to achieve the above-mentioned problems, claim 1 of the present application is an electronic blackboard for displaying input data input by a user, and the user identification information for identifying the user, the input data, and the input data. A display processing means for reading out the input data associated with the user identification information of the user who has succeeded in authentication from the data recording unit that stores the related voice data in association with each other and displaying the data on the video display unit, and the above. It is characterized by having a voice processing means for reading voice data related to the input data associated with the user identification information of the user who has succeeded in authentication from the data recording unit and causing the voice reproduction unit to reproduce the voice data. ..

According to the embodiment of the present invention, in order to display the input data associated with the user and reproduce the voice data according to the user's authentication, the input data input by the user is displayed and the input data thereof is displayed. It is possible to reduce the trouble of playing back related audio data.

It is a block diagram of an example of the image processing system which concerns on this embodiment. It is a hardware block diagram of an example of the computer which concerns on this embodiment. It is a hardware block diagram of an example of the electronic blackboard which concerns on this embodiment. It is a functional block diagram of an example of the electronic blackboard which concerns on this embodiment. It is a block diagram of an example of a user table. It is a block diagram of an example of a whiteboard table. It is a block diagram of an example of a page data table. It is a block diagram of an example of a voice data table. It is a block diagram of an example of a handwritten data table. It is a flowchart of an example of the processing procedure of the page display of an electronic blackboard. It is an image diagram of an example of a user authentication screen. It is an image diagram of an example of a UI screen. It is an image diagram of an example of a summary page screen. It is a flowchart of an example of the processing procedure of inputting handwritten data of an electronic blackboard and recording voice data. It is a flowchart of an example of the processing procedure of reproduction of audio data of an electronic blackboard. It is a flowchart of an example of the processing procedure of reproduction of audio data of an electronic blackboard. It is an image diagram of an example of utterance data.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this embodiment, an example of a conference using an electronic blackboard device will be described, but the present invention is not limited to the conference and can be applied to various situations such as seminars and lectures.

[First Embodiment]
<System configuration>
FIG. 1 is a configuration diagram of an example of an image processing system according to the present embodiment. In the image processing system 10 of FIG. 1, a server 12 and one or more electronic blackboards 14 are communicably connected to each other via a communication network 18. The communication network 18 includes, for example, the Internet, a LAN (Local Area Network), a mobile phone communication network, and the like.

The server 12 operates in cooperation with the electronic blackboard 14 to provide the functions described below related to the electronic blackboard 14. The server 12 may have the function of the user authentication unit 42 of the electronic blackboard 14 described later and the function of the data recording unit 46. The server 12 may be operated on-premises or in the cloud. The server 12 is not limited to a PC (Personal Computer), and may be a mobile phone, a smartphone, a tablet terminal, a PDA (Personal Digital Assistant), or the like.

On the electronic blackboard 14, an image drawn by an event (touch of the pen tip of the electronic pen 690 or the pen end of the electronic pen 690 on the display 680 described later) caused by the electronic pen 690 described later is displayed on the display 680. Can be made to. The electronic blackboard 14 can change the image displayed on the display 680 based on not only the electronic pen 690 but also an event (gestures such as enlargement, reduction, page turning, etc.) caused by the user's hand H or the like. can.

Further, a USB (Universal Serial Bus) memory 630, which will be described later, can be connected to the electronic blackboard 14. The electronic blackboard 14 can read an electronic file such as a PDF from the USB memory 630 and record the electronic file in the USB memory 630. Further, the electronic blackboard 14 is connected to the electronic blackboard 14 via a cable capable of communication according to standards such as DisplayPort, DVI (Digital Visual Interface), HDMI (registered trademark) (High-Definition Multimedia Interface), and VGA (Video Graphics Array). A notebook PC 670 described later can be connected.

The electronic blackboard 14 causes an event by contact with the display 680. The electronic blackboard 14 can transmit event information indicating this event to the notebook PC 670 in the same manner as an event from an input device such as a mouse or a keyboard. Similarly, a television (video) conference terminal may be connected to the electronic blackboard 14 via a cable capable of communicating according to the above standard. The notebook PC 670 and the video conference terminal may communicate with the electronic blackboard 14 by wireless communication compliant with various wireless communication protocols such as Bluetooth (registered trademark).

Further, the electronic blackboard 14 installed at different bases or the like communicably connected via the communication network 18 is placed on the display 680 based on an event generated by the user's hand H or the like on another electronic blackboard 14. You can also change the displayed image.

As a result, for example, an image drawn on the display 680 of the electronic blackboard 14 at one base is also displayed on the display 680 of the electronic blackboard 14 at another base. On the contrary, the image drawn on the display 680 of the electronic blackboard 14 at another base is displayed on the display 680 of the electronic blackboard 14 at one base. As described above, since the image processing system 10 can perform remote sharing processing for sharing the same image at a remote location, it is very convenient to use it for a conference or the like at a remote location.

The electronic blackboard 14 is an example, and if it is a device having a function of recording and displaying an image and a function of recording and playing back a voice, it can be used for an electronic signboard (digital signage), sports, weather forecast, and the like. Telestrator, remote image (video) diagnostic device, HUD (Head Up Display) device, industrial machine, image pickup device, sound collector, medical device, network home appliance, car (Connected Car), PC, mobile phone, smartphone, It may be a tablet terminal, a game machine, a PDA, a digital camera, a digital video camera, or the like. The USB memory 630 is an example of a recording medium, and may be various recording media such as an SD card.

The configuration of the image processing system 10 shown in FIG. 1 is an example. For example, the server 12 may be configured by a plurality of units, or may be omitted by providing the electronic blackboard 14 with the functions of the server 12. Further, at least a part of the functions of the electronic blackboard 14 may be provided in the server 12. As described above, the functions of the server 12 and the electronic blackboard 14 may be provided by either the server 12 or the electronic blackboard 14. Further, the functions of the server 12 and the electronic blackboard 14 may be configured to be realized by a device other than the server 12 and the electronic blackboard 14.

<Hardware configuration>
"Computer"
The server 12 shown in FIG. 1 is realized by, for example, a computer 500 having a hardware configuration as shown in FIG. FIG. 2 is a hardware configuration diagram of an example of a computer according to the present embodiment.

As shown in FIG. 2, the computer 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, a RAM (Random Access Memory) 503, an HD504, an HDD (Hard Disk Drive) controller 505, and a display 506. , External device connection I / F (Interface) 508, network I / F 509, data bus 510, keyboard 511, pointing device 512, DVD-RW (Digital Versatile Disk Rewritable) drive 514, media I / F 516.

Of these, the CPU 501 controls the operation of the entire computer 500 according to a program. The ROM 502 stores a program used to drive the CPU 501 such as an IPL. The RAM 503 is used as a work area of the CPU 501. The HD504 stores various data such as programs. The HDD controller 505 controls reading or writing of various data to the HD 504 according to the control of the CPU 501.

The display 506 displays various information such as cursors, menus, windows, characters, or images. The external device connection I / F 508 is an interface for connecting various external devices. The external device is, for example, a USB memory, a printer, or the like. The network I / F 509 is an interface for performing data communication using the communication network 18. The data bus 510 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 501.

The keyboard 511 is a kind of input means including a plurality of keys for inputting characters, numerical values, various instructions, and the like. The pointing device 512 is a kind of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, and the like. The DVD-RW drive 514 controls reading or writing of various data to the DVD-RW 513 as an example of a detachable recording medium. In addition, it is not limited to DVD-RW, and may be DVD-R or the like. The media I / F 516 controls reading or writing (storage) of data to a recording medium 515 such as a flash memory.

《Electronic blackboard》
FIG. 3 is a hardware configuration diagram of an example of an electronic blackboard according to the present embodiment. As shown in FIG. 3, the electronic blackboard 14 includes a CPU 601, a ROM 202, a RAM 603, an SSD (Solid State Drive) 604, a network I / F605, and an external device connection I / F606.

The CPU 601 controls the operation of the entire electronic blackboard 14 according to a program. The ROM 602 stores a program used to drive the CPU 601 such as the CPU 601 and the IPL. The RAM 603 is used as a work area for the CPU 601. The SSD 604 stores various data such as a program for the electronic blackboard 14.

The network I / F605 controls communication with the communication network 18. The external device connection I / F606 is an interface for connecting various external devices. The external device in this case is, for example, a USB memory 630, an external device (microphone 640, speaker 650, camera 660, etc.).

The electronic blackboard 14 includes a capture device 611, a GPU (Graphics Processing Unit) 612, a display controller 613, a contact sensor 614, a sensor controller 615, an electronic pen controller 616, an IC card reader 617, a short-range communication circuit 619, and short-range communication. The circuit 619 includes an antenna 619a, a power switch 622, and selection switches 623.

The capture device 611 displays video information as a still image or a moving image on the display of the external PC 670. GPU612 is a semiconductor chip that specializes in graphics. The display controller 613 controls and manages the screen display in order to output the output image from the GPU 612 to the display 680 or the like.

The contact sensor 614 detects that the electronic pen 690, the user's hand H, or the like touches the display 680. The sensor controller 615 controls the processing of the contact sensor 614. The contact sensor 614 inputs coordinates and detects coordinates by an infrared blocking method. In this method of inputting coordinates and detecting coordinates, two light receiving and emitting devices installed at both upper ends of the display 680 radiate a plurality of infrared rays in parallel with the display 680 and are provided around the display 680. This is a method of receiving light that is reflected by a reflecting member and returns on the same optical path as the optical path of the light emitted by the light receiving element.

The contact sensor 614 outputs an infrared ID emitted by two light receiving / receiving devices blocked by the object to the sensor controller 615, and the sensor controller 615 specifies a coordinate position which is a contact position of the object. The electronic pen controller 616 determines whether or not there is a touch of the pen tip or a touch of the pen tail on the display 680 by communicating with the electronic pen 690.

The IC card reader 617 reads the identification information unique to the IC card 700 from the RF tag embedded in the IC card 700 by wireless communication. The IC card reader 617 may be built in the electronic blackboard 14 or externally attached to the electronic blackboard 14. The IC card 700 may be a built-in smart device such as a smartphone. Further, the electronic blackboard 14 may use a device other than the IC card reader 617 as long as it can acquire identification information that can identify the user, such as a biometric authentication device (fingerprint, palm print, iris, face, etc.), a barcode reading device, or the like. May be used.

Further, the short-range communication circuit 619 is a communication circuit such as NFC or Bluetooth. The power switch 622 is a switch for switching ON / OFF of the power of the electronic blackboard 14. The selection switches 623 are, for example, a group of switches for adjusting the brightness and hue of the display of the display 680.

Further, the electronic blackboard 14 is provided with a bus line 610. The bus line 610 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 601 shown in FIG.

The contact sensor 614 is not limited to the infrared blocking method, but is a capacitive touch panel that specifies a contact position by detecting a change in capacitance, and a contact position is specified by a voltage change between two opposing resistance films. Various detection means such as a resistance film type touch panel and an electromagnetic induction type touch panel that detects the electromagnetic induction generated by the contact object touching the display unit and specifies the contact position may be used. Further, the electronic pen controller 616 may determine whether or not there is a touch not only on the pen tip and pen tail of the electronic pen 690 but also on the portion gripped by the user of the electronic pen 690 and other electronic pen portions.

The program for the electronic blackboard 14 may be provided by recording it on a computer-readable recording medium such as a CD-ROM and distributing it, or it may be downloaded from a server operated on-premises or in the cloud. It may be provided at. The server 12 may have a function of downloading the program for the electronic blackboard 14 to the electronic blackboard 14.

<Functional configuration>
The electronic blackboard 14 according to the present embodiment is realized by, for example, the functional configuration shown in FIG. FIG. 4 is a functional configuration diagram of an example of an electronic blackboard according to the present embodiment. In the functional configuration of FIG. 4, configurations unnecessary for the description of the present embodiment are appropriately omitted.

The electronic blackboard 14 of FIG. 4 realizes each functional configuration shown in FIG. 4 by the hardware configuration shown in FIG. 3 and the program for the electronic blackboard. The electronic blackboard 14 includes a voice reproduction unit 20, a voice input unit 22, a voice processing unit 24, a data link unit 26, a video display unit 28, a display processing unit 30, a page processing unit 32, an operation input unit 34, and an operation processing unit 36. It has a stroke processing unit 38, a card information acquisition unit 40, a user authentication unit 42, a state management unit 44, and a data recording unit 46.

The voice reproduction unit 20, the voice input unit 22, and the voice processing unit 24 use the functions of the OS (operating system) in which the program for the electronic blackboard 14 operates. The voice reproduction unit 20 reproduces the audio data transmitted from the audio processing unit 24 on the speaker 650. The voice input unit 22 transmits the voice data input from the microphone 640 to the voice processing unit 24.

The voice processing unit 24 reads the voice data to be reproduced from the data recording unit 46, transmits the read voice data to the voice reproduction unit 20, and reproduces the read voice data on the speaker 650. Further, the voice processing unit 24 records the voice data received from the voice input unit 22 and stores it in the data recording unit 46.

The video display unit 28 displays the image data transmitted from the display processing unit 30 on the touch display 710. The touch display 710 corresponds to the display 680 and the contact sensor 614 in FIG. The operation input unit 34 detects an event generated by the user on the touch display 710 (such as an operation in which the user's hand H is touched on the touch display 710), and transmits the coordinate position of the detected event to the operation processing unit 36. ..

The operation processing unit 36 distributes the event for which the coordinate position is received from the operation input unit 34 to each event of stroke drawing, UI operation, and gesture operation. Stroke drawing is an event in which, for example, the electronic pen 690 is pressed on the touch display 710, the electronic pen 690 is moved in the pressed state, and finally the electronic pen 690 is released from the touch display 710.

By this stroke drawing, for example, the alphabets "S" and "T" for handwriting input are drawn on the touch display 710. The stroke drawing includes not only drawing an image of handwriting input, but also an event of deleting an already drawn image or editing a drawn image.

The UI operation is an event in which the user presses a predetermined coordinate position with, for example, an electronic pen 690 or a hand H when, for example, the UI screen 1100 described later is displayed on the touch display 710. By this UI operation, for example, the color and width of the line drawn by the electronic pen 690 are set.

The gesture operation is an event in which the user touches or moves the touch display 710 with his / her hand H when an image such as a stroke image is displayed on the touch display 710. By this gesture operation, the user can enlarge (or reduce) the image, change the display area, switch pages, etc. by moving the hand H while touching the hand H on the touch display 710. ..

The operation processing unit 36 executes various operations from those determined to be UI operations according to the elements (display components) such as the UI screen 1100 in which the event is generated. Examples of the elements of the UI screen 1100 and the like include buttons, lists, check boxes, text boxes, and the like. Further, the operation processing unit 36 executes an operation corresponding to the gesture operation.

The display processing unit 30 transmits image data such as the UI screen 1100 generated by the page processing unit 32 to the video display unit 28 and displays it on the touch display 710. The stroke processing unit 38 draws an image, deletes the drawn image, and edits the drawn image based on the stroke drawing event assigned by the operation processing unit 36.

The page processing unit 32 uses images of handwritten input from the stroke processing unit 38, various data stored in the data recording unit 46, and the like to obtain image data of screens such as the UI screen 1100 and the summary page screen 1200 described later. Generate. The card information acquisition unit 40 acquires a card ID, which is an example of identification information unique to the IC card 700, from the IC card 700 and transmits it to the user authentication unit 42.

The user authentication unit 42 performs user authentication based on the card ID received from the card information acquisition unit 40 and the user table described later. For example, the user authentication unit 42 determines that the authentication is successful if the card ID received from the card information acquisition unit 40 is registered in the user table described later. The user authentication unit 42 may perform user authentication based on the user ID and password input by the user. Further, the user authentication unit 42 can acquire authentication information or the like using a camera 660, a biometric authentication device, or the like, and perform user authentication based on the authentication information or the like. The electronic blackboard 14 may have its own device perform user authentication, or may have an external device perform user authentication.

The state management unit 44 manages the state based on the result of user authentication by the user authentication unit 42 (state before successful user authentication, state after successful user authentication, etc.), and provides the managed state to the operation processing unit 36. ..

The data link unit 26 associates the user data, the whiteboard data, the page data, the voice data, and the handwritten data, which will be described later, with the user table, the whiteboard table, the page data table, the voice data table, and the handwritten data table. It is stored in the data recording unit 46. The data recording unit 46 shows a user table as shown in FIG. 5, a whiteboard table as shown in FIG. 6, a page data table as shown in FIG. 7, an audio data table as shown in FIG. 8, and a voice data table as shown in FIG. I remember a handwritten data table like this. The tables of FIGS. 5 to 9 show an example of managing data by GUID (UUID). The GUID has a format such as {104110D9-7794-4C82-95C1-694A0F164F81}.

FIG. 5 is a configuration diagram of an example of a user table. The user table of FIG. 5 manages user data having a user ID, a card ID, and a password as items. The user ID is an example of identification information that identifies a user. The card ID is an example of identification information that identifies the IC card 700. The password is an example of authentication information used for user authentication.

FIG. 6 is a block diagram of an example of a whiteboard table. The whiteboard table of FIG. 6 manages whiteboard data having a whiteboard ID, a user ID, and an end time as items. The whiteboard ID is an example of identification information that identifies the electronic blackboard 14. The user ID is an example of identification information that identifies a user. The end time represents the end time of use when the user identified by the user ID last uses the electronic blackboard 14 identified by the whiteboard ID.

FIG. 7 is a block diagram of an example of a page data table. The page data table of FIG. 7 manages page data having a page data ID, a user ID, a whiteboard ID, a summary page flag, a recording flag, and a page data path as items.

The page data ID is an example of identification information that identifies the page data. The user ID is an example of identification information that identifies a user. The whiteboard ID is an example of identification information that identifies the electronic blackboard 14. The summary page Flag is an example of information indicating whether or not the summary page is described later. The summary page Flag is "TRUE" when it is a summary page, and "FALSE" when it is not a summary page.

The recording flag is an example of information indicating whether or not to record audio data. The recording flag is "TRUE" when the voice data is recorded, and "FALSE" when the voice data is not recorded. The page data path is an example of information indicating the location of page data.

FIG. 8 is a block diagram of an example of a voice data table. The voice data table of FIG. 8 manages voice data having voice data ID, page data ID, start time, end time, and voice data path as items.

The voice data ID is an example of identification information for identifying voice data. The page data ID is an example of identification information that identifies the page data. The start time represents the recording start time of the voice data. The end time represents the recording end time of the voice data. The voice data path is an example of information indicating the location of voice data. For example, audio data can be stored and played back in MP3 format. As an example, the voice data indicated by the voice data ID "GUID # 3001" will be described. This audio data is audio data recorded when the page data indicated by the page data ID "GUID # 2003" is displayed on the electronic blackboard 14. This audio data is audio data that started recording at 13:30 on January 11, 2020 and ended recording at 13:37:00.

FIG. 9 is a block diagram of an example of a handwritten data table. The handwritten data table of FIG. 9 manages handwritten data having a handwritten data ID, a page data ID, a voice data ID, an elapsed time, an area ID, and a handwritten data path as items.

The handwritten data ID is an example of identification information that identifies the handwritten data. The page data ID is an example of identification information that identifies the page data. The voice data ID is an example of identification information for identifying voice data. The elapsed time represents the time elapsed from the start of recording the voice data to the start of handwriting. The area ID represents an area to be described later in which the handwritten summary page is performed. The handwritten data path is an example of information indicating the location of handwritten data. For example, handwritten data can be stored and displayed in the ink serialization format (ISF).

As an example, the handwritten data indicated by the handwritten data ID "GUID # 5101" will be described. This handwritten data is data handwritten when the page data indicated by the page data ID "GUID # 2003" is displayed on the electronic blackboard 14. The voice data "GUID # 3001" is voice data that starts recording from 13:30. The handwritten data ID "GUID # 5101" is handwritten data in which handwriting is started from 13:35:15, which is 315 seconds after the start of recording, with respect to the area where the area ID is 1. Similarly, the handwritten data ID "GUID # 5102" is handwritten data in which handwriting is started from 13:35:16, which is 316 seconds after the start of recording, with respect to the area where the area ID is 1. be. Further, the handwritten data ID "GUID # 5115" is handwritten data in which handwriting is started from 13:35:22, which is 322 seconds after the start of recording, for the area where the area ID is 2.

<Processing>
The image processing system 10 according to the present embodiment processes the page display by the procedure as shown in FIG. 10, for example. FIG. 10 is a flowchart of an example of the processing procedure for displaying the page of the electronic blackboard. The page display process of FIG. 10 is an example of a process of displaying the page used last time by the user when the user authenticates the user and the user authentication is successful. If the user has created a summary page in the previous use of the electronic blackboard 14, the electronic blackboard 14 displays the summary page.

In step S10, the video display unit 28 of the electronic blackboard 14 displays the user authentication screen 1000 as shown in FIG. 11 on the touch display 710, and waits for login by the IC card 700. FIG. 11 is an image diagram of an example of the user authentication screen. The user who uses the electronic blackboard 14 holds his / her IC card 700 over the IC card reader 617.

In step S12, the card information acquisition unit 40 acquires the card ID of the IC card 700 and transmits it to the user authentication unit 42. The user authentication unit 42 performs user authentication based on the card ID received from the card information acquisition unit 40 and the user table of FIG. In step S14, the user authentication unit 42 determines whether the user authentication is successful or unsuccessful. If the card ID received from the card information acquisition unit 40 is registered in the user table of FIG. 5, the user authentication unit 42 determines that the authentication is successful. Further, the user authentication unit 42 determines that the authentication has failed unless the card ID received from the card information acquisition unit 40 is registered in the user table of FIG.

If it is determined that the authentication has failed, the electronic blackboard 14 returns to step S10 and returns to a state of waiting for login by the IC card 700. If it is determined that the authentication is successful, the page processing unit 32 of the electronic blackboard 14 proceeds to step S16 to acquire the user ID (GUID) of the user who has succeeded in the authentication from the user table of FIG.

In step S18, the page processing unit 32 acquires the whiteboard ID (GUID) of the electronic blackboard 14 last used by the user who succeeded in authentication based on the user ID acquired in step S16 and the whiteboard table of FIG. ..

In step S20, the page processing unit 32 was last used by the user who succeeded in authentication based on the user ID acquired in step S16, the whiteboard ID acquired in step S18, and the page data table of FIG. Acquire the page data ID (GUID) of the electronic blackboard 14.

In step S22, the page processing unit 32 restores the whiteboard page data based on the page data of the page data ID acquired in step S20 and the handwritten data of FIG. 9 corresponding to the page data ID acquired in step S20. ..

In step S24, the display processing unit 30 causes the UI screen 1100 of FIG. 12 or the summary page screen 1200 of FIG. 13 to be displayed on the touch display 710. For example, the display processing unit 30 causes the touch display 710 to display the UI screen 1100 as shown in FIG. 12 if the item “summary page Flag” of the page data acquired in step S20 is not “TRUE”. Further, for example, if the item "summary page Flag" of the page data acquired in step S20 is "TRUE", the display processing unit 30 displays the summary page screen 1200 as shown in FIG. 13 on the touch display 710.

FIG. 12 is an image diagram of an example of the UI screen. The UI screen 1100 of FIG. 12 is an example of a display component that receives instructions from the user, such as a page addition button 1102, a page deletion button 1104, a summary page creation button 1106, an all page deletion button 1108, a page selection button 1110, and a page list scroll button. 1112, a page read button 1114, a handwritten button 1116, an end button 1118, and a handwritten delete button 1120 are provided.

The page addition button 1102 receives a page addition instruction from the user. The page deletion button 1104 receives a page deletion instruction from the user. The summary page creation button 1106 receives a summary page creation instruction from the user. The all-page deletion button 1108 receives an all-page deletion instruction from the user. The page selection button 1110 receives a page selection instruction from the user. The page list scroll button 1112 receives a page list scroll instruction from the user.

The page load button 1114 receives a page load instruction from the user. The handwriting button 1116 receives a handwriting instruction from the user. The end button 1118 receives an end instruction from the user. Further, the handwriting deletion button 1120 receives a handwriting deletion instruction from the user.

FIG. 13 is an image diagram of an example of the summary page screen. In addition to the display components of FIG. 12, the summary page screen 1200 of FIG. 13 is provided with an area 1202, an audio reproduction button 1204, all audio reproduction buttons 1206, an audio deletion button 1208, and a page selection button 1210 of the summary page. ..

The image data of the summary page screen 1200 of FIG. 13 is created by the user pressing the summary page creation button 1106 of FIG. 12, and is added as a new record to the page data table of FIG. 7. One or more areas 1202 are areas on the screen that accept handwriting input from the user. The voice reproduction button 1204 is provided for each area 1202 and receives a voice reproduction instruction from the user.

All voice playback buttons 1206 receive voice playback instructions from the user to play back all voice data associated with the summary page screen 1200. The page selection button 1210 of the summary page receives a summary page selection instruction from the user.

For example, on the UI screen 1100 of FIG. 12 or the summary page screen 1200 of FIG. 13, when a page switching event such as pressing the page selection button 1110 is received, the page processing unit 32 of the electronic blackboard 14 is as shown in FIG. Input handwritten data and record audio data in the procedure. FIG. 14 is a flowchart of an example of a procedure for inputting handwritten data on an electronic blackboard and recording voice data.

In step S30, it is assumed that the page processing unit 32 has received an event from the user to switch pages. In step S32, the page processing unit 32 determines whether or not audio data is being recorded. When the voice data has already been recorded, the voice processing unit 24 of the electronic blackboard 14 stops the recording of the voice data in step S34, and sets the end time in the corresponding record of the voice data table of FIG. If no audio data has been recorded, the process of step S34 is skipped.

Further, the page processing unit 32 determines whether or not the summary page Flag of the page data table of the page to be switched is "TRUE". If the summary page Flag of the page data table of the page to be switched is not "TRUE", the page processing unit 32 ends the process of FIG. 14 because the screen displayed by the page switching is not the summary page screen 1200.

If the summary page Flag of the page data table of the page to be switched is "TRUE", the page processing unit 32 proceeds to step S38 because the screen displayed by switching pages is the summary page screen 1200, and the summary page screen 1200 is displayed. It is determined whether or not the recording flag of the page data table is "TRUE".

Summary If the recording flag of the page data table of the page screen 1200 is not "TRUE", the page processing unit 32 ends the process of FIG. 14 because the audio data has already been recorded. If the recording flag of the page data table of the summary page screen 1200 is "TRUE", the page processing unit 32 proceeds to step S42 because no audio data has been recorded, and presses the audio reproduction button 1204 and all audio reproduction buttons 1206. Disable it.

In step S44, the page processing unit 32 adds a new record to the voice data table, and sets the voice data ID, the page data ID, and the start time. In step S46, the voice processing unit 24 starts recording voice data. Further, the page processing unit 32 sets the storage destination of the voice data as the voice data path.

Summary When the handwriting input to the page screen 1200 is started, the process proceeds from step S48 to step S50, and the page processing unit 32 determines whether or not the handwriting start position where the handwriting input is started is in the area 1202. If the handwriting start position where the handwriting input is started is not within the area 1202, the page processing unit 32 returns to step S48 and waits until the next handwriting input.

If the handwriting start position where the handwriting input is started is within the area 1202, the page processing unit 32 proceeds to step S52, adds a new record to the handwriting table of FIG. 9, and has a handwriting data ID, a page data ID, and an area. Set the ID. Further, the page processing unit 32 records the recording start time.

In step S54, when the handwriting input to the summary page screen 1200 is completed, the page processing unit 32 proceeds from step S54 to step S56, saves the handwriting data, and puts the handwriting data in the handwriting data path of the handwriting data table of FIG. Set the save destination of. The page processing unit 32 returns to step S48 and waits until the next handwriting input.

Further, for example, when the voice reproduction button 1204 or all the voice reproduction buttons 1206 of FIG. 13 are pressed, the page processing unit 32 of the electronic blackboard 14 reproduces the voice data by the procedure as shown in FIG. 15 or FIG. 15 and 16 are flowcharts of an example of a processing procedure for reproducing audio data of an electronic blackboard. Note that FIG. 15 shows an example in which the voice reproduction button 1204 in the area of the area ID “1” of the summary page screen 1200 is pressed. FIG. 16 shows an example in which all the voice reproduction buttons 1206 on the summary page screen 1200 are pressed.

In step S60, the page processing unit 32 extracts a record whose area ID is "1" with the target page data ID from the handwritten data table of FIG. In step S62, the page processing unit 32 extracts the voice data ID of the record extracted in step S60, and extracts the record in which the extracted voice data ID is set from the voice data table of FIG.

Proceeding to step S64, the page processing unit 32 deletes the record of the handwritten data table of the area ID “1”. The page processing unit 32 sorts the records in the audio data table extracted in step S62 in descending order according to the recording start time, and reproduces the audio data for each record in steps S68 to S70. If there is handwritten data with an elapsed time of N after N seconds have elapsed from the start of reproduction of the audio data, the page processing unit 32 draws the handwritten data in the area of the area ID “1”.

For example, it is assumed that the voice data shown in the utterance data of FIG. 17 is recorded in the area of the area ID “1”. FIG. 17 is an image diagram of an example of utterance data. The utterance data shown in FIG. 17 represents the speaker, the utterance time, and the utterance content. For example, when the voice data shown in the utterance data of FIG. 17 is recorded, the area ID "1" is set during the voice reproduction of the utterance time "13:35:13-13:35:22" of the speaker "Sato". The handwritten data IDs "GUID # 5101", "GUID # 5102", and "GUID # 5103" of FIG. 9 are displayed in the area according to the elapsed times "315", "316", and "317", respectively.

In step S80 of FIG. 16, the page processing unit 32 extracts a record of the target page data ID from the handwritten data table of FIG. In step S82, the page processing unit 32 extracts the voice data ID of the record extracted in step S80, and extracts the record in which the extracted voice data ID is set from the voice data table of FIG.

The process proceeds to step S84, and the page processing unit 32 deletes the record in the handwritten data table of the target page data ID. The page processing unit 32 sorts the records in the audio data table extracted in step S82 in descending order according to the recording start time, and reproduces the audio data for each record in steps S88 to S90. If there is handwritten data with an elapsed time of N after N seconds have elapsed from the start of reproduction of the voice data, the page processing unit 32 draws the handwritten data in the area of the area ID.

[Other embodiments]
The image processing system 10 of the present embodiment automatically deletes the voice data and the handwritten data after reproducing the voice data by the procedure as shown in FIG. 15 or FIG. 16, and stores the server 12 or the electronic blackboard 14. You may try to avoid tight capacity. Further, after the voice data is reproduced by the procedure as shown in FIG. 15 or FIG. 16, the image processing system 10 of the present embodiment displays an inquiry to the user as to whether or not to delete the voice data and the handwritten data, and the user displays. The operation of deleting the voice data and the handwritten data may be accepted.

Further, the timing of deleting the voice data and the handwritten data is not limited to the time of reproduction. For example, when voice data and handwritten data are newly recorded / recorded, the remaining amount of the storage capacity of the electronic blackboard 14 is calculated. If it is determined that the remaining amount is less than the capacity of the voice data and handwritten data recorded / recorded this time (even if the same voice data and handwritten data are recorded / recorded again, it cannot be executed), the voice data and handwritten data You may display asking the user whether or not to delete.

The present invention is not limited to the above-described embodiment disclosed specifically, and various modifications and modifications can be made without departing from the scope of claims. It is needless to say that the image processing system 10 described in this embodiment is an example, and there are various system configuration examples depending on the intended use and purpose.

Each function of the embodiment described above can be realized by one or more processing circuits. Here, the "processing circuit" as used herein is a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, or a processor designed to execute each function described above. It shall include devices such as ASIC (Application Specific Integrated Circuit), DSP (digital signal processor), FPGA (field programmable gate array) and conventional circuit modules.

The group of devices described in the examples is only one of a plurality of computing environments for implementing the embodiments disclosed herein.

In one embodiment, the server 12 includes a plurality of computing devices such as a server cluster. The plurality of computing devices are configured to communicate with each other via any type of communication link, including a communication network 18, shared memory, and the like, and perform the processes disclosed herein. Similarly, the electronic blackboard 14 can include a plurality of computing devices configured to communicate with each other.

Further, the server 12 and the electronic blackboard 14 can be configured to share the disclosed processing steps in various combinations. For example, the process executed by the electronic blackboard 14 may be executed by the server 12. Similarly, the process executed by the server 12 can be executed by the electronic blackboard 14. Further, each element of the server 12 and the electronic blackboard 14 may be integrated into one device, or may be divided into a plurality of devices.

The handwritten data is an example of input data input by the user. The input data is data that is input by the user and displayed on the touch display 710, and is, for example, stroke data. Further, the voice data related to the input data is voice data stored in association with the input data, for example, voice data recorded at the time of inputting the input data. The display processing unit 30 is an example of display processing means. The voice processing unit 24 is an example of voice processing means. The page processing unit 32 is an example of page processing means. The user ID is an example of user identification information. The data recording unit 46 is an example of data recording means. The user authentication unit 42 is an example of a user authentication means.

10 Image processing system 12 Server 14 Electronic blackboard 18 Communication network 20 Audio playback unit 22 Audio input unit 24 Audio processing unit 26 Data link unit 28 Video display unit 30 Display processing unit 32 Page processing unit 34 Operation input unit 36 Operation processing unit 38 Strokes Processing unit 40 Card information acquisition unit 42 User authentication unit 44 Status management unit 46 Data recording unit 614 Contact sensor 617 IC card reader 640 Microphone 650 Speaker 680 Display 690 Electronic pen 700 IC card 710 Touch display 1106 Summary page creation button 1200 Summary page screen 1202 Area 1204 Audio playback button

Japanese Unexamined Patent Publication No. 2005-284490

Claims (8)

An electronic blackboard that displays input data entered by the user.
The data recording unit that stores the user identification information for identifying the user, the input data, and the voice data related to the input data in association with each other is associated with the user identification information of the user who has succeeded in authentication. A display processing means for reading out the input data and displaying it on the video display unit,
A voice processing means for reading voice data related to the input data associated with the user identification information of the user who has succeeded in authentication from the data recording unit and causing the voice reproduction unit to reproduce the voice data.
Electronic blackboard with. The voice processing means is characterized in that the voice data at the time of inputting the input data by the user is recorded and stored in the data recording unit in association with the user identification information of the user and the input data. Item 1 The electronic blackboard. Further having a page processing means for displaying a page on which one or more areas for accepting handwriting input from the user are displayed on the video display unit and generating a page for accepting handwriting input from the user for each area.
The voice processing means records the voice data at the time of the handwriting input for each area that receives handwriting input from the user, and associates the data with the user identification information of the user and the input data of the handwriting input. The electronic blackboard according to claim 2, wherein the electronic blackboard is stored in a recording unit. The page processing means is based on the input data of the handwriting input stored in the data recording unit in association with the user identification information of the user who has succeeded in authentication and the voice data at the time of the handwriting input. Generates a page displaying one or more areas for displaying handwriting input and display components for receiving instructions from the user to reproduce the voice data at the time of the handwriting input of the handwriting input displayed in the area. The electronic blackboard according to claim 3, wherein the image is displayed on the image display unit. The page processing means displays an inquiry to the user as to whether or not to delete the input data of the handwriting input and the voice data at the time of the handwriting input after the voice reproduction unit reproduces the voice data. The electronic blackboard according to claim 4, wherein the user accepts an operation of deleting the input data of the handwriting input and the voice data at the time of the handwriting input. It is a voice reproduction method of the electronic blackboard that displays the input data input by the user.
The data recording unit that stores the user identification information for identifying the user, the input data, and the voice data related to the input data in association with each other is associated with the user identification information of the user who has succeeded in authentication. A display processing step of reading out the input data and displaying it on the video display unit,
A voice processing step of reading voice data related to the input data associated with the user identification information of the user who has succeeded in authentication from the data recording unit and causing the voice reproduction unit to reproduce the voice data.
Audio reproduction method having. A computer that displays the input data entered by the user,
The data recording unit that stores the user identification information for identifying the user, the input data, and the voice data related to the input data in association with each other is associated with the user identification information of the user who has succeeded in authentication. Display processing means for reading the input data and displaying it on the video display unit,
A voice processing means for reading voice data related to the input data associated with the user identification information of the user who has succeeded in authentication from the data recording unit and causing the voice reproduction unit to reproduce the voice data.
A program to function as. An image processing system including an electronic blackboard that displays input data input by a user, and a server that can be connected to the electronic blackboard via a network.
A data recording means for storing user identification information for identifying the user, the input data, and voice data related to the input data in association with each other.
A user authentication means for authenticating the user and
A display processing means for reading the input data associated with the user identification information of the user who has succeeded in authentication from the data recording means and displaying the input data on the video display unit.
A voice processing means for reading voice data related to the input data associated with the user identification information of the user who has succeeded in authentication from the data recording means and causing the voice reproduction unit to reproduce the voice data.
Image processing system with.

Images