US20180095711A1 - Communication terminal, communication system, transmission method, and recording medium storing program - Google Patents
Communication terminal, communication system, transmission method, and recording medium storing program Download PDFInfo
- Publication number
- US20180095711A1 US20180095711A1 US15/663,943 US201715663943A US2018095711A1 US 20180095711 A1 US20180095711 A1 US 20180095711A1 US 201715663943 A US201715663943 A US 201715663943A US 2018095711 A1 US2018095711 A1 US 2018095711A1
- Authority
- US
- United States
- Prior art keywords
- image
- data
- display
- electronic whiteboard
- communication
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 238000004891 communication Methods 0.000 title claims abstract description 250
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000005540 biological transmission Effects 0.000 title abstract description 91
- 230000015654 memory Effects 0.000 claims description 75
- 239000002131 composite material Substances 0.000 claims description 9
- 238000009877 rendering Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 description 53
- 238000010586 diagram Methods 0.000 description 49
- 238000012545 processing Methods 0.000 description 30
- 230000006870 function Effects 0.000 description 21
- 230000004044 response Effects 0.000 description 5
- 238000005401 electroluminescence Methods 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 230000005236 sound signal Effects 0.000 description 4
- 238000012546 transfer Methods 0.000 description 3
- 230000000903 blocking effect Effects 0.000 description 2
- 238000006243 chemical reaction Methods 0.000 description 2
- 230000000295 complement effect Effects 0.000 description 2
- 238000011960 computer-aided design Methods 0.000 description 2
- 230000005674 electromagnetic induction Effects 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 229910044991 metal oxide Inorganic materials 0.000 description 2
- 150000004706 metal oxides Chemical class 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 239000007787 solid Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- YSGQGNQWBLYHPE-CFUSNLFHSA-N (7r,8r,9s,10r,13s,14s,17s)-17-hydroxy-7,13-dimethyl-2,6,7,8,9,10,11,12,14,15,16,17-dodecahydro-1h-cyclopenta[a]phenanthren-3-one Chemical compound C1C[C@]2(C)[C@@H](O)CC[C@H]2[C@@H]2[C@H](C)CC3=CC(=O)CC[C@@H]3[C@H]21 YSGQGNQWBLYHPE-CFUSNLFHSA-N 0.000 description 1
- 241000238876 Acari Species 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000003213 activating effect Effects 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 239000012528 membrane Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/20—Drawing from basic elements, e.g. lines or circles
- G06T11/203—Drawing of straight lines or curves
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/60—Editing figures and text; Combining figures or text
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/40—Support for services or applications
- H04L65/403—Arrangements for multi-party communication, e.g. for conferences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2200/00—Indexing scheme for image data processing or generation, in general
- G06T2200/24—Indexing scheme for image data processing or generation, in general involving graphical user interfaces [GUIs]
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
- G09G2340/00—Aspects of display data processing
- G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
Definitions
- Embodiments of the present disclosure relate to a communication terminal, a communication system, a transmission method, and a recording medium storing program.
- communication systems are widely used, which are capable of carrying out videoconferences among remotely located sites through a communication network such as the Internet or private lines.
- a communication network such as the Internet or private lines.
- communication terminals in a communication system for example, electronic whiteboards that transmit stroke data such as of a drawn text, or video conference terminals that transmit video data such as of a conference participant are known in the art.
- the communication terminal on one side can perform rendering on the screen image and transmit it to the communication terminal on the other side such that the screen will be shared among multiple communication terminals.
- Embodiments of the present disclosure described herein provide a communication terminal, a communication system, a transmission method, and a recording medium storing program.
- the communication terminal and the transmission method includes controlling a display to display a first superimposed image where a drawn image and a user interface image related to drawing operation are superimposed on a background image, and transmitting image data of a second superimposed image, where the drawn image is superimposed on the background image and the user interface image is excluded from images to be superimposed, to an external communication terminal as image data to be displayed on the display.
- the communication system includes the communication terminal and the external communication terminal.
- the recording medium stores a program for causing a computer to execute the transmission method.
- FIG. 1 is a diagram illustrating a schematic network configuration of a communication system, according to an embodiment of the present invention.
- FIG. 2 is a schematic block diagram illustrating a hardware configuration of an electronic whiteboard, according to an embodiment of the present disclosure.
- FIG. 3 is a schematic block diagram illustrating a hardware configuration of a management, system according to an embodiment of the present disclosure.
- FIG. 4 is a schematic block diagram illustrating a hardware configuration of a transmission terminal according to an embodiment of the present disclosure.
- FIG. 5A and FIG. 5B are schematic block diagrams each illustrating a software configuration of an electronic whiteboard or a transmission terminal, respectively, according to an embodiment of the present disclosure.
- FIG. 6 is a functional block diagram of a communication system, according to an embodiment of the present disclosure.
- FIG. 7 is a functional block diagram of a drawing controller according to an embodiment of the present disclosure.
- FIG. 8 is a sequence diagram illustrating the processes of establishing communication among a plurality of electronic whiteboards, according to an embodiment of the present disclosure.
- FIG. 9 is a sequence diagram illustrating the processes of establishing communication between an electronic whiteboard and a transmission terminal, according to an embodiment of the present disclosure.
- FIG. 10A and FIG. 10B are data sequence diagrams each illustrating the processes of generating an image, according to an embodiment of the present disclosure.
- FIG. 11A and FIG. 11B are diagrams each illustrating a superimposed image, according to an embodiment of the present disclosure.
- FIG. 12 is a sequence diagram illustrating the processes of sending the screen image on an electronic whiteboard side to a transmission terminal, according to an embodiment of the present disclosure.
- FIG. 13 is a diagram illustrating an example display according to an embodiment of the present disclosure.
- FIG. 14A and FIG. 14B are diagrams each illustrating an example display according to an embodiment of the present disclosure.
- FIG. 15 is a diagram illustrating a schematic configuration of a communication system, according to a fourth embodiment of the present disclosure.
- FIG. 16 is a schematic block diagram illustrating a hardware configuration of a telephone terminal, according to an embodiment of the present disclosure.
- FIG. 17A , FIG. 17B , and FIG. 17C are functional block diagrams of an electronic whiteboard, a data server, and a telephone terminal, respectively, according to a fourth embodiment of the present disclosure.
- FIG. 18 is a sequence diagram illustrating the processes of downloading the data of a background image by an electronic whiteboard, according to a fourth embodiment of the present disclosure.
- FIG. 19 is a diagram illustrating an example display according to an embodiment of the present disclosure.
- FIG. 20 is a sequence diagram illustrating the processes of downloading the data of a background image by an electronic whiteboard, according to a fifth embodiment of the present disclosure.
- FIG. 21 is a diagram illustrating stroke arrangement data according to an embodiment of the present disclosure.
- FIG. 22 is a diagram illustrating an example data structure of the coordinate arrangement data, according to an embodiment of the present disclosure.
- processors may be implemented as program modules or functional processes including routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types and may be implemented using existing hardware at existing network elements or control nodes.
- Such existing hardware may include one or more central processing units (CPUs), digital signal processors (DSPs), application-specific-integrated-circuits (ASICs), field programmable gate arrays (FPGAs), computers or the like. These terms in general may be collectively referred to as processors.
- terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or the like refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical, electronic quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
- a communication system 1 is described with reference to FIG. 1 .
- FIG. 1 is a diagram illustrating a schematic configuration of the communication system 1 , according to an embodiment of the present disclosure.
- the communication system 1 includes a plurality of electronic whiteboards 2 a , 2 b , and 2 c , a transmission terminal 8 , a relay device 30 , and a management system 50 .
- the electronic whiteboards 2 a , 2 b , and 2 c are connected to each other and can communicate with each other via a communication network 9 a . Moreover, the electronic whiteboards 2 a , 2 b , and 2 c are provided with displays 3 a , 3 b , and 3 c , respectively.
- the electronic whiteboards 2 a , 2 b , and 2 c can control the displays 3 a , 3 b , and 3 c to display images that are drawn by the events caused by electronic styluses 4 a , 4 b , and 4 c (i.e., the touches on the displays 3 a , 3 b , and 3 c by the tip or bottom of the electronic styluses 4 a , 4 b , and 4 c ).
- the electronic whiteboards 2 a , 2 b , and 2 c can control the displays 3 a , 3 b , and 3 c to modify the images being displayed on the displays 3 a , 3 b , and 3 c according to the events caused not only by the electronic styluses 4 a , 4 b , and 4 c but also by user's hands Ha, Hb, and He, or the like (i.e., gestures such as for zooming in, zooming out, and flipping pages).
- any one of the electronic whiteboards 2 a , 2 b , and 2 c may be referred to simply as the electronic whiteboard 2
- any one of the displays 3 a , 3 b , and 3 c may be referred to simply as the display 3
- any one of the electronic styluses 4 a , 4 b , and 4 c may be referred to simply as the electronic stylus 4
- any one of the user's hands Ha. Hb, and He may be referred to simply as the hand H.
- USB universal serial bus
- the electronic whiteboards 2 a , 2 b , and 2 c can read electronic files such as portable document format (PDF) files from the USB memories 5 a , 5 b , and 5 c , or can store such electronic files in the USB memories 5 a , 5 b , and 5 c .
- PDF portable document format
- laptop personal computers (PCs) 6 a , 6 b , and 6 c are connected to the electronic whiteboards 2 a , 2 b , and 2 c , respectively, through a cable that allows communication under the standard such as DisplayPort, digital visual interface (DVI), high-definition multimedia interface (HDMI), registered trademark), and video graphics array (VGA).
- a cable that allows communication under the standard such as DisplayPort, digital visual interface (DVI), high-definition multimedia interface (HDMI), registered trademark), and video graphics array (VGA).
- USB memories 5 a , 5 b , and 5 c may be referred to simply as the USB memory 5
- any one of the laptop PCs 6 a , 6 b , and 6 c may be referred to simply as the laptop PC 6 .
- the stroke data of a stroke that is drawn on the display 3 of the electronic whiteboard 2 at one site is transmitted to the electronic whiteboard 2 at another site through the communication network 9 a , and the stroke data can also be displayed on the display 3 at the other site.
- the stroke data that is drawn on the display 3 of the electronic whiteboard 2 at the other site is transmitted to the electronic whiteboard 2 at the one site through the communication network 9 a , and the stroke data can be displayed on the display 3 of the electronic whiteboard 2 at the one site.
- drawing images can be shared among remote sites, such that the communication system 1 with the multiple electronic whiteboards 2 can facilitate a video conference.
- any desired external communication terminal other than the electronic whiteboard 2 as above may additionally be connected to the communication network 9 a to share stroke data with the electronic whiteboard 2 .
- Such any desired external communication terminal includes a personal computer (PC) with a stroke sharing function, a video conference terminal, a tablet personal computer (PC), a smartphone, digital signage, a telestrator that is often used, for example, in sports and weather broadcasts, or a computer including, for example, an image processing device such as a remote image (video) diagnostic device, a mobile information terminal, a digital video camera, a digital camera, and a terminal such as a game machine capable of providing image frames
- the transmission terminal 8 is an example of a communication terminal for video conference capable of transmitting video data and audio data.
- the transmission terminal 8 may be replaced with a tablet personal computer (PC), a smartphone, digital signage, a telestrator, an image processing device, a mobile information terminal, a digital video camera, a digital camera, and a communication terminal capable of providing image frames such as a game machine.
- the electronic whiteboard 2 c out of the electronic whiteboards 2 a , 2 b , and 2 c is installed with a video conference application, and is capable of transmitting not only stroke data but also video data and audio data. Due to this configuration, the electronic whiteboard 2 c and the transmission terminal 8 can exchange the video data and audio data for video conference through the communication network 9 b.
- the relay device 30 relays the video data and audio data transmitted by one of the electronic whiteboard 2 c and the transmission terminal 8 to the other one of the electronic whiteboard 2 c and the transmission terminal 8 .
- the management system 50 centrally manages, for example, the communication status between the electronic whiteboard 2 c and the transmission terminal 8 .
- the communication network 9 a is an intranet
- the communication network 9 b is the Internet.
- the communication networks 9 a and 9 b are not limited to the above configuration, and may be any desired communication network selected from the Internet, the local area network (LAN), the intranet, the mobile phone network, or the like.
- the communication networks 9 a and 9 b may be the same network.
- both the communication networks 9 a and 9 b may be the Internet.
- the hardware configuration of the electronic whiteboard 2 according to the present embodiment is described with reference to FIG. 2 .
- FIG. 2 is a schematic block diagram illustrating a hardware configuration of the electronic whiteboard 2 according to the present embodiment.
- the electronic whiteboard 2 includes, for example, a central processing unit (CPU) 201 that controls the overall operation of the electronic whiteboard 2 , a read only memory (ROM) 202 that stores a control program used for controlling the CPU 201 such as an initial program loader (IPL), a random access memory (RAM) 203 that is used as a work area for the CPU 201 , a solid state disk (SSD) 204 that stores various data such as a control program for the electronic whiteboard 2 , a network interface (I/F) 205 that controls the communication with an external device through the communication network 9 , and an external memory controller 206 that controls the communication with an external device such as a universal serial bus (USB) memory 5 .
- CPU central processing unit
- ROM read only memory
- RAM random access memory
- SSD solid state disk
- I/F network interface
- USB universal serial bus
- the electronic whiteboard 2 includes, for example, a camera 207 that captures a target object under the control of the CPU 201 , an imaging device interface (I/F) 208 that controls the operation of the camera 207 , a capture device 211 that controls the display of a laptop personal computer (PC) 6 to display the video data as a still image or moving images, a graphics processing unit (GPU) 212 specializing in processing graphics, and a display controller 213 that controls the visual display to output the image generated by the GPU 212 to the display 3 .
- a camera 207 that captures a target object under the control of the CPU 201
- I/F imaging device interface
- a capture device 211 controls the display of a laptop personal computer (PC) 6 to display the video data as a still image or moving images
- GPU graphics processing unit
- display controller 213 that controls the visual display to output the image generated by the GPU 212 to the display 3 .
- the electronic whiteboard 2 includes, for example, a sensor controller 214 that controls operation of the contact sensor 215 , and a contact sensor 215 that detects a touch onto the display 3 , which is made by, for example, an electronic stylus 4 and a user's hand H.
- the contact sensor 215 senses a touch input to a specific coordinate on the display 3 using the infrared blocking system.
- the display 3 is provided with two light receivers and light emitters disposed on both upper ends of the display 3 , and a reflector frame disposed on the periphery of the display 3 .
- the light receivers and light emitters emit a plurality of infrared rays in parallel to a touch panel of the display 3 .
- the light receivers and light emitters receive lights passing in the direction that is the same as that of an optical path of the emitted infrared rays, which are reflected by the reflector frame. By so doing, a touch input to specific coordinates is sensed.
- the contact sensor 215 outputs to the sensor controller 214 the identification (ID) of the infrared light rays that are emitted from the two light receivers and light emitters and then are blocked by an object such as the hand H, and the sensor controller 214 specifies the coordinates of the position touched by the object. Note also that all sorts of ID as described below are an example of identification information.
- the contact sensor 215 uses the infrared blocking system.
- the contact sensor 215 may use various types of detectors such as a capacitive touch panel that detects changes in capacitance to specify a touched position, a resistive touch panel that senses changes in voltage between two facing resistive membranes to specify a touched position, and an electromagnetic induction type touch panel that detects electromagnetic induction caused when an object touches a display to specify a touched position.
- the contact sensor 215 may identify the contact position using the camera 207 .
- the electronic whiteboard 2 is provided with an electronic stylus controller 216 .
- the electronic stylus controller 216 communicates with the electronic stylus 4 to detect a touch by the tip or bottom of the electronic stylus 4 to the display 3 .
- the electronic stylus controller 216 may also detect a touch by another part of the electronic stylus 4 , such as a part held by a hand.
- the electronic whiteboard 2 includes, for example, an audio input and output interface (I/F) 224 that controls the input and output of an audio signal between an external microphone 222 and an external loudspeaker 223 under the control of the CPU 101 , an operation key 225 that accepts user's operation, and a power switch 226 that turns on or turns off the power of the electronic whiteboard 2 .
- I/F audio input and output interface
- any of the microphone 222 and the loudspeaker 223 may be built into the electronic whiteboard 2 .
- the electronic whiteboard 2 includes, for example, a bus line 230 , such as an address bus or a data bus, which electrically connects the elements illustrated in FIG. 2 to one another.
- a bus line 230 such as an address bus or a data bus, which electrically connects the elements illustrated in FIG. 2 to one another.
- the control program for the electronic whiteboard 2 may be stored on a computer-readable recording medium such as a CD-ROM for distribution, or stored on a server on any desired network to be downloaded.
- the hardware configuration of the management system 50 according to the present embodiment is described with reference to FIG. 3 .
- FIG. 3 is a schematic block diagram illustrating a hardware configuration of the management system 50 according to the present embodiment.
- the management system 50 includes a CPU 501 that controls the entire operation of the management system 50 , a ROM 502 storing a transmission control program, a RAM 503 that is used as a work area for the CPU 501 , a hard disk (HD) 504 storing various kinds of data, a hard disk drive (HDD) 505 that controls reading or writing of various kinds of data to or from the HI) 504 under control of the CPU 501 , and a medium interface (I/F) 507 that controls reading or writing of data to or from a recording medium 506 such as a flash memory.
- a CPU 501 that controls the entire operation of the management system 50
- a ROM 502 storing a transmission control program
- a RAM 503 that is used as a work area for the CPU 501
- a hard disk (HD) 504 storing various kinds of data
- HDD hard disk drive
- I/F medium interface
- the management system 50 includes a display interface (I/F) 508 that controls the display 516 to display various kinds of information such as a cursor, a menu, a window, a character, and an image, a network interface (I/F) 509 that transmits data using the communication network 9 , a keyboard 511 that is provided with a plurality of keys for allowing a user to input, for example, characters, numerical values, and various kinds of instructions, a mouse 512 that is one example of pointing device for allowing a user to select or execute various kinds of instructions, select a target to be processed, or to move a mouse cursor being displayed.
- a display interface (I/F) 508 that controls the display 516 to display various kinds of information such as a cursor, a menu, a window, a character, and an image
- a network interface (I/F) 509 that transmits data using the communication network 9
- a keyboard 511 that is provided with a plurality of keys for allowing a user to input, for
- the management system 50 includes a compact disc read only memory (CD-ROM) drive 514 that reads or writes various kinds of data to and from a CD-ROM 513 , which is one example of removable recording medium, an external device interface (I/F) 515 that exchanges data with an external device, and a bus line 510 , such as an address bus or a data bus, which electrically connects the elements illustrated in FIG. 2 to one another.
- CD-ROM compact disc read only memory
- I/F external device interface
- bus line 510 such as an address bus or a data bus, which electrically connects the elements illustrated in FIG. 2 to one another.
- the above transmission control program is in an installable or executable file format, and may be recorded on a computer-readable recording medium such as the recording medium 506 and the CD-ROM 513 for distribution.
- the above transmission control program may be stored on the HD 504 .
- the hardware configuration of the relay device 30 is equivalent to that of the management system 50 , and thus its description is omitted.
- FIG. 4 is a schematic block diagram illustrating a hardware configuration of the transmission terminal 8 operating as a video conference terminal, according to the present embodiment.
- the transmission terminal 8 includes, for example, a central processing unit (CPU) 801 , a read only memory (ROM) 802 , a random access memory (RAM) 803 , a flash memory 804 , a solid state drive (SSD) 805 , a medium interface (I/F) 807 , a power switch 809 , a bus line 810 , a network interface (I/F) 811 , a camera 812 , an imaging device interface (I/F) 813 , a microphone 814 , a loudspeaker 815 , an audio input and output interface (I/F) 816 , a display interface (I/F) 817 , an external device connection interface (I/F) 818 , a keyboard 821 , and a mouse 822 .
- CPU central processing unit
- ROM read only memory
- RAM random access memory
- SSD solid state drive
- I/F medium interface
- I/F medium interface
- I/F medium interface
- the CPU 801 controls the overall operation of the transmission terminal 8 .
- the ROM 802 stores a control program such as an initial program loader (IPL) used for operating the CPU 801 .
- the RAM 803 is used as a work area for the CPU 801 .
- the flash memory 804 stores various kinds of data such as a communication control program, display data, and audio data.
- the SSD 805 controls reading or writing of various kinds of data to or from the flash memory 804 under control of the CPU 801 .
- a hard disk drive (HDD) may be used.
- the medium interface 807 controls reading or writing of data to or from a recording medium 806 such as a flash memory.
- the network interface 811 controls data communication with an external device through the communication network 9 b .
- the camera 812 is an example of a built-in imaging device that captures an object under the control of the CPU 801 to obtain display data.
- the imaging device interface 813 is a circuit that controls operation of the camera 812 .
- the microphone 814 is an example of a built-in sound collector capable of receiving audio data.
- the audio input and output interface 816 is a circuit for controlling an input and output of an audio signal between the microphone 814 and the loudspeaker 815 , under control of the CPU 801 .
- the display interface 817 is a circuit for transmitting display data to the external display 820 under control of the CPU 801 .
- the external device connection interface 818 is an interface circuit that connects the transmission terminal 8 to various kinds of external devices.
- the keyboard 821 is one example of input device provided with a plurality of keys for allowing an user to input characters, numerical values, or various kinds of instructions.
- the mouse 822 is one example of pointing device for allowing a user to select or execute various kinds of instructions, select a target to be processed, or to move a mouse cursor being displayed.
- the bus line 810 is, for example, an address bus or a data bus, which electrically connects a plurality of elements as illustrated in FIG. 4 such as the CPU 801 .
- the display 820 is an example of a display unit composed of a liquid crystal or organic electroluminescence (EL) display that displays, for example, an image of an object and an operation icon. Moreover, the display 820 is connected to the display interface 817 by a cable 820 c .
- the cable 820 c may be an analog RGB (red, green, blue) (video graphic array (VGA)) signal cable, a component video cable, a high-definition multimedia interface (HDMI, registered trademark) signal cable, or a digital video interactive (DVI) signal cable.
- RGB red, green, blue
- HDMI high-definition multimedia interface
- DVI digital video interactive
- the camera 812 includes a lens, and a solid-state image sensing device that converts an image or moving images (video) of an object into electronic data through photoelectric conversion.
- a solid-state image sensing device for example, a complementary metal oxide semiconductor (CMOS) or a charge-coupled device (CCD) is used.
- CMOS complementary metal oxide semiconductor
- CCD charge-coupled device
- the external device connection interface 818 is capable of connecting an external device such as an external camera, an external microphone, or an external loudspeaker thereto through a universal serial bus (USB) cable or the like.
- USB universal serial bus
- the recording medium 806 is removable from the transmission terminal 8 .
- the flash memory 804 may be replaced with any nonvolatile memory such as an electrically erasable and programmable read only memory (EEPROM) that reads or writes data under control of the CPU 801 .
- EEPROM electrically erasable and programmable read only memory
- FIG. 5A is a schematic block diagram illustrating a software configuration of the electronic whiteboard 2 , according to the present embodiment.
- the electronic whiteboard 2 is installed with an operating system (OS) 2020 , a first communication application A 11 , and a second communication application B 11 , each of which operates on a work area 2010 of the RAM 203 .
- OS operating system
- the OS 2020 is basic software that controls entire operation of the electronic whiteboard 2 through providing basic functions.
- the communication application A 11 and B 11 each enable the electronic whiteboard 2 to communicate with another electronic whiteboard (or communication terminal), using different communication protocols.
- the communication application A 11 is a client application that provides the electronic whiteboard 2 with a communication control function of controlling transmission of stroke data to another communication terminal, and an image processing function of outputting the stroke data as an image.
- the communication application B 11 is a client application that provides the electronic whiteboard 2 with a communication control function of sending and receiving the video data or audio data of a conference or the like to or from another communication terminal, an image processing function of inputting and outputting the video data, and an audio processing function of inputting and outputting the audio data.
- the OS 2020 and the communication application A 11 are installed onto the electronic whiteboards 2 a , 2 b , and 2 c before shipment.
- the communication application B 11 may be installed onto the electronic whiteboard 2 as desired after shipment. In the following description, it is assumed that the electronic whiteboard 2 c is installed with the communication application B 11 , but the electronic whiteboards 2 a and 2 b are not installed with the communication application B 11 .
- FIG. 5B is a schematic block diagram illustrating a software configuration of the transmission terminal 8 , according to the present embodiment.
- the OS 8020 and the third communication application B 11 are deployed and operate on a work area 8010 of the RAM 803 .
- the OS 8020 is basic software that controls the entire operation of the transmission terminal 8 through providing basic functions.
- the communication application B 11 provides the transmission terminal 8 with a communication control function of sending and receiving video data of a conference or the like to or from another external communication terminal, and an image processing function of outputting the video data as an image.
- the OS 8020 and the communication application B 11 are installed onto the transmission terminal 8 before or after shipment.
- FIG. 6 is a schematic block diagram illustrating a functional configuration of the electronic whiteboards 2 , the transmission terminal 8 , and the management system 50 in the communication system 1 , according to the present embodiment.
- the electronic whiteboard 2 is provided with the hardware components as illustrated in FIG. 2 which operate in cooperation with the control programs, thereby implementing the functional configuration as illustrated in FIG. 6 .
- the electronic whiteboard 2 includes a memory 2000 implemented by the SSD 204 .
- the transmission terminal 8 is provided with the hardware components as illustrated in FIG. 4 that operate in cooperation with the control programs, thereby implementing the functional configuration as illustrated in FIG. 6 .
- the transmission terminal 8 includes a memory 8000 implemented by the SSD 805 .
- the electronic whiteboard 2 may serve as a “host device” that firstly requests to start communication to share images (remote sharing), or may serve as a “participant device” that later participates in the communication (remote sharing) already started by the host device.
- the electronic whiteboard 2 consists of a client unit 20 and a server unit 90 .
- the client unit 20 and the server unit 90 are functions implemented by one electronic whiteboard 2 , with activation of the communication application A 11 .
- the electronic whiteboard 2 may only include the client unit 20 , and a device such as a distribution controller other than the electronic whiteboard 2 may implement the server unit 90 .
- both the client unit 20 and the server unit 90 are implemented by the electronic whiteboard 2 .
- the electronic whiteboard 2 serves as a participant device, only the client unit 20 is implemented by the electronic whiteboard 2 , and the server unit 90 is not implemented.
- the client unit 20 of the electronic whiteboard 2 a communicates with the client unit 20 of each of the other electronic whiteboards 2 b and 2 c , through the server unit 90 implemented in the same electronic whiteboard 2 a .
- the client unit 20 of each one of the electronic whiteboards 2 b and 2 c communicates with the client unit 20 of each of the other electronic whiteboard 2 a , 2 b , or 2 c , through the server unit 90 implemented in the electronic whiteboard 2 a.
- the client unit 20 starts operating.
- the client unit 20 includes a communication controller 21 and a drawing controller 22 .
- the communication controller 21 is implemented by the network interface 205 or the instructions from the CPU 201 , and controls the communication performed with another electronic whiteboard 2 or the communication performed with the server unit 90 or the like of the same electronic whiteboard 2 .
- the drawing controller 22 is implemented by the instructions from the CPU 201 , and performs image processing based on the stroke data generated in response to operation made on the display 3 , or based on various kinds of data obtained from any desired device. Examples of such a device include, but not limited to, the USB memory 5 , the laptop PC 6 , the communication controller 21 , and the memory 2000 .
- the drawing controller 22 generates an image layer based on the processed image data, and controls output of a combined image where a plurality of image layers are superimposed on top of one another.
- FIG. 7 is a functional block diagram of the drawing controller 22 , according to the present embodiment.
- the drawing controller 22 includes a stroke processing unit 22 A, a superimposition unit 22 Y, and a page processor 22 Z.
- the stroke processing unit 22 A may be implemented by the instructions from the CPU 201 , and generates stroke data that reflects the user's drawing on the display 3 with the electronic stylus 4 or the user's hand H.
- the superimposition unit 22 Y may be implemented by the instructions from the CPU 201 , and generates a plurality of image layers based on each type of data and superimposes those image layers on top of one another.
- the page processor 22 Z associates a plurality of items of display data such as of stroke data with each other as one page of data, and stores the associated items of display data in a page data memory 220 of the memory 2000 .
- the page data memory 220 is a memory area of the memory 2000 , and stores page data as depicted in Table 1 given below.
- Table 1 is a diagram illustrating an example data structure of the page data.
- the page data includes one-page data to be displayed on the display 3 , which includes stroke arrangement data (items of stroke data for one character, etc.) and medium data.
- page data ID by which any desired one page is identified the start time at which the display of that page is started, the end time when rewriting of page is no longer performed, for example, by stroke and gesture
- stroke arrangement data ID that identifies the stroke arrangement data caused by the stroke of the electronic stylus 4 or the user's hand H
- medium data ID that identifies the medium data are stored in association with each other.
- the stroke arrangement data is used to display a set of stroke images on the display 3 .
- the medium data is used to display another image overlapping with the stroke image on the display 3 .
- the stroke arrangement data includes information in detail as depicted in FIG. 21 .
- FIG. 21 is a diagram illustrating stroke arrangement data according to the present embodiment.
- one stroke arrangement data includes a plurality of items of stroke data.
- one item of stroke data indicates stroke data ID that identifies that stroke data, the start time at which the writing of one item of stroke starts, the end time at which the writing of one item of stroke ends, the color of the stroke, the width of the stroke, and the coordinate arrangement data ID that identifies the arrangement of way points of the stroke.
- stroke data ID identifies that stroke data
- start time at which the writing of one item of stroke starts the end time at which the writing of one item of stroke ends
- the color of the stroke the width of the stroke
- the coordinate arrangement data ID that identifies the arrangement of way points of the stroke.
- the coordinate arrangement data includes information in detail as depicted in FIG. 22 .
- FIG. 22 is a diagram illustrating an example data structure of the coordinate arrangement data, according to the present embodiment.
- the coordinate arrangement data indicates several items of information including a point on the display 3 (X coordinate value, Y coordinate value), the difference in time (millisecond (ms)) when passing through this point since the start time of the stroke, and the writing pressure of the electronic stylus 4 on that point.
- a collection of several points depicted in FIG. 22 is indicated by one item of coordinate arrangement data as depicted in FIG. 21 .
- the coordinate arrangement data indicates those multiple way points in the stroke drawing.
- the medium data of the page data in Table 1 includes more detailed information as depicted in Table 2.
- Table 2 depicts an example data structure of medium data.
- the medium data ID included in the page data depicted in Table 1 the type of data of the medium data, the time at which the medium data is stored, the position (X coordinate value, Y coordinate value) of the image to be displayed on the display 3 according to the medium data, and the size (width, height) of the image, as well as the data indicating the contents of the medium data are associated with each other.
- the server unit 90 includes a communication controller 91 .
- the communication controller 91 is implemented by the instructions from the CPU 201 , and controls communication, through the communication network 9 a , between the communication controller 21 in the client unit 20 of the same electronic whiteboard 2 and the communication controller 21 in the client unit 20 of another one of the electronic whiteboards 2 .
- the client unit 80 As the communication application B 11 is started in the electronic whiteboard 2 or the transmission terminal 8 , the client unit 80 starts operating.
- the client unit 80 includes a communication controller 81 , a display controller 82 , and an input acceptance unit 83 .
- the communication controller 81 may be implemented by the instructions from the CPU 201 or the CPU 801 or the network interface 205 or the network interface 811 , and controls communication with another communication terminal or the like.
- the display controller 82 may be implemented by the instructions from the CPU 201 or the CPU 801 , and performs image processing based on data obtained from the memory 2000 , the memory 8000 , or the like, and controls output of processed image data.
- the input acceptance unit 83 may be implemented by the instructions from the CPU 201 or the CPU 801 , and receives various kinds of operation or input made by a user.
- the electronic whiteboard 2 c further includes a display order controller 24 .
- the display order controller 24 starts operating as the OS 2020 is started in the electronic whiteboard 2 c.
- the display order controller 24 may be implemented by the instructions from the CPU 201 , and activates one of the images generated by the client units 20 and 80 and deactivates the other one of the images. Due to this configuration, the display order of these images is controlled. More specifically, the processes of activating one of the images consist of the processes of displaying that image to the front compared with the other images, and the processes of deactivating one of the images consist of the processes of displaying the other images to the front compared with the deactivated image.
- the management system 50 is provided with the hardware components as illustrated in FIG. 3 that operate in cooperation with control programs, thereby implementing each functional configuration as illustrated in FIG. 6 .
- the management system 50 also includes a memory 5000 that is implemented by the HD 504 illustrated in FIG. 3 .
- the communication controller 51 of the management system 50 may be implemented by the instructions from the CPU 501 , and control communication with the electronic whiteboard 2 or the transmission terminal 8 through the communication network 9 b.
- FIG. 8 is a sequence diagram illustrating the processes of establishing communication among a plurality of electronic whiteboards, according to the present embodiment.
- step S 1 the communication application A 11 starts operating, and the client unit 20 of the electronic whiteboard 2 a starts processing.
- the communication controller 21 of the client unit 20 sends instructions to start the processing of the server unit 90 to the communication controller 91 in the server unit 90 of the same electronic whiteboard 2 a . Due to this configuration, not only the client unit 20 but also the server unit 90 of the electronic whiteboard 2 a starts various kinds of processing (step S 2 ).
- the communication controller 91 of the electronic whiteboard 2 a generates connection information to establish a connection with the electronic whiteboard 2 a , and controls the display 3 a to display the generated connection information (step S 3 ).
- connection information includes the IP address of the host device and a pass code generated for the present remote sharing, and the pass code is stored in the memory 2000 .
- a user of the electronic whiteboard 2 a notifies the users of the electronic whiteboards 2 b and 2 c of the connection data by means of, for example, telephone or electronic mail.
- step S 4 and S 5 the power switch 226 of each of the electronic whiteboards 2 b and 2 c is turned on.
- the communication controller 21 in the client unit 20 of each of the electronic whiteboards 2 b and 2 b transmits a participation request including a pass code to the communication controller 91 in the server unit 90 of the electronic whiteboard 2 a , based on the IP address included in the connection information, through the communication network 9 a (steps S 6 and S 7 ).
- the communication controller 91 of the electronic whiteboard 2 a receives the participation requests including the pass code from each of the electronic whiteboards 2 b and 2 c.
- the communication controller 91 determines whether or not to authenticate each one of the electronic whiteboards 2 b and 2 c using the pass codes received from the electronic whiteboards 2 b and 2 c with reference to the pass codes stored in the memory 2000 (step S 8 ). Then, the communication controller 91 notifies each of the client units 20 of the electronic whiteboard 2 b and 2 c of the authentication result (steps S 9 and S 10 ).
- the electronic whiteboard 2 a that serves as the host device establishes a communication for remote sharing with the electronic whiteboard 2 b and electronic whiteboard 2 c each of which serves as a participant device (steps S 11 and S 12 ).
- contents of data exchanged among the electronic whiteboards 2 a , 2 b , and 2 c include stroke data.
- the stroke processing unit 22 A of the corresponding one of the electronic whiteboards 2 a , 2 b , and 2 c When a user draws a stroke image on one of the electronic whiteboards 2 a , 2 b , and 2 c using the electronic stylus 4 a , 4 b , or 4 c , the stroke processing unit 22 A of the corresponding one of the electronic whiteboards 2 a , 2 b , and 2 c generates the stroke data of the drawn stroke image based on the coordinates of the contact portion between the electronic styluses 4 a , 4 b , or 4 c and the displays 3 a , 3 b , or 3 c .
- the communication controller 21 of the relevant one of the electronic whiteboards 2 a , 2 b , and 2 c sends the generated stroke data to the communication controller 91 of the electronic whiteboard 2 that serves as the host device.
- the communication controller 91 of the electronic whiteboard 2 a transmits the received stroke data to the client units 20 of the electronic whiteboards 2 a , 2 b , and 2 c .
- the communication controllers 21 in the electronic whiteboards 2 a , 2 b , and 2 c then receive the transmitted stroke data.
- the page processors 22 Z of the electronic whiteboards 2 a , 2 b , and 2 c controls the page data memory 220 to store the newly received stroke data in an added manner without overwriting the previously stored stroke data (steps S 13 , S 14 , and S 15 ).
- the communication among the electronic whiteboards 2 a , 2 b , and 2 c is established, and then the stroke data that is generated by any one of the electronic whiteboards 2 a , 2 b , and 2 c is accumulated in the page data memories 220 of the electronic whiteboards 2 a , 2 b , and 2 c.
- FIG. 9 is a sequence diagram illustrating the processes of establishing communication between the electronic whiteboard 2 c and the transmission terminal 8 , according to the present embodiment.
- the client unit 80 starts various kinds of processing (step S 21 ).
- the communication controller 81 of the client unit 80 sends a login request to the management system 50 (step S 22 ).
- This login request includes a user ID and a pass code. Accordingly, the communication controller 51 of the management system 50 receives the login request.
- the management system 50 authenticates the electronic whiteboard 2 c that has sent the login request (step S 23 ), based on whether a pair of user ID and a pass code included in the login request matches a pair of user ID and a pass code stored in the memory 5000 in advance.
- the communication controller 51 of the management system 50 sends an authentication result to the electronic whiteboard 2 c that has sent the login request (step S 24 ). Accordingly, the communication controller 81 of the electronic whiteboard 2 c receives the authentication result.
- the electronic whiteboard 2 c is determined to be an authorized login request sender are described.
- step S 31 Once a request to start the communication application B 11 is accepted by user's operation made on the transmission terminal 8 , the communication application B 11 starts operating, and the client unit 80 of the transmission terminal 8 starts various kinds of processing (step S 31 ).
- the communication controller 81 of the transmission terminal 8 sends a login request including the user ID and pass code to the management system 50 (step S 32 ). Then, the management system 50 receives this login request.
- the management system 50 authenticates the transmission terminal 8 that has sent the login request using the user ID and the pass code included in the login request (step S 33 ), and sends the authentication result to the transmission terminal 8 (step S 34 ).
- the transmission terminal 8 is determined to be an authorized login request sender are described.
- the input acceptance unit 83 of the electronic whiteboard 2 c accepts a request to start a phone conversation with the transmission terminal 8 (step S 41 ).
- the communication controller 81 of the electronic whiteboard 2 c sends a request to start a phone conversation to the management system 50 (step S 42 ).
- This request to start a phone conversation includes the user ID of the electronic whiteboard 2 c that is the request sender terminal and the user ID of the transmission terminal 8 that is the destination terminal.
- the communication controller 51 of the management system 50 sends the received request to start a phone conversation to the transmission terminal 8 that is the counterpart terminal (step S 43 ).
- the communication controller 81 of the transmission terminal 8 receives the request to start a phone conversation sent from the management system 50 . Once a user of the transmission terminal 8 operates the transmission terminal 8 to accept the request to start a phone conversation, the communication controller 81 sends acceptance data to the management system 50 (step S 44 ).
- the communication controller 51 of the management system 50 receives the acceptance data sent from the transmission terminal 8 , the communication controller 51 sends the acceptance data to the electronic whiteboard 2 c that has sent the start request (step S 45 ).
- the communication controller 81 of the electronic whiteboard 2 c receives the acceptance data sent from the management system 50 .
- the management system 50 establishes communication between the electronic whiteboard 2 c and the transmission terminal 8 (steps S 46 - 1 and S 46 - 2 ).
- the communication controller 51 of the management system 50 sends the IP address of the electronic whiteboard 2 c or the transmission terminal 8 that serves as the counterpart communication terminal, and the relay device ID of the relay device 30 , to the transmission terminal 8 and the electronic whiteboard 2 c .
- the communication controller 81 of each of the transmission terminal 8 and the electronic whiteboard 2 c sends the relay device ID and the IP address received from the management system 50 to the relay device 30 , and establishes a connection with the relay device 30 .
- the communication controller 81 of each of the transmission terminal 8 and the electronic whiteboard 2 c sends the video data of the video recorded by the camera 207 or the camera 812 on the local terminal side and the audio data of the sound recorded by the microphone 222 or the microphone 814 on the local communication terminal side to the relay device 30 .
- the relay device 30 relays the image data and audio data between the transmission terminal 8 and the electronic whiteboard 2 c (step S 47 ).
- the communication controller 81 of each of the transmission terminal 8 and the electronic whiteboard 2 c receives the video data and audio data on the counterpart communication terminal side.
- the above video data and audio data is streaming data and includes a plurality of pieces of image data or audio data that are divided by time.
- the communication controller 81 of the electronic whiteboard 2 c newly receives the image data and audio data, updates the image data and audio data stored in the memory 2000 to the newly received image data and audio data (steps S 48 and S 49 ). Due to this configuration, the latest image data and audio data are stored in the memory 2000 .
- FIG. 10A is a flowchart of the processes of generating an image using the communication application A 11 , according to the present embodiment.
- the image data of a background image is stored in the page data memory 220 .
- the page processor 22 Z of the electronic whiteboard 2 c obtains the image data of a background image stored on the page data memory 220 (step S 51 ).
- the background image has a nontransparent area of the resolution (size) of the screen of the display 3 , and is, for example, a white and plain image.
- the background image is used as a layer of the background image that makes up the screen in the later processes as will be described later.
- the page processor 22 Z of the electronic whiteboard 2 c obtains an external image input from the laptop PC 6 c (step S 52 ).
- the external image is used as a layer of the external image that makes up the screen in the later processes as will be described later.
- the page processor 22 Z of the electronic whiteboard 2 c skips the processes in the step S 52 .
- the image data (medium data) of object images is stored in the page data memory 220 .
- the object images are, for example, images of a predetermined object such as a circle, rectangular, and a triangle.
- the page processor 22 Z of the electronic whiteboard 2 c generates a layer of object images where specified ones of the object images stored in the page data memory 220 are arranged at positions specified by the user's operation (step S 53 ). In these processes, the page processor 22 Z superimposes object images on the transparent pixel area of the resolution (size) of the screen of the display 3 .
- the page processor 22 Z of the electronic whiteboard 2 c generates a layer of stroke images based on the stroke data accumulated in the page data memory 220 (see the step S 15 ) (step S 54 ). In these processes, the page processor 22 Z superimposes the stroke images indicated by the stroke data on the transparent pixel area of the resolution (size) of the screen of the display 3 . For example,
- the page processor 22 Z When stroke data indicating the text of “S” in black is stored in the page data memory 220 , the page processor 22 Z generates a layer of stroke images including the text “S” in black thereon.
- the page data memory 220 of the electronic whiteboard 2 c stores the image data of a user interface (UI) image.
- image data of a UI image is, for example, an icon or key related to the drawing operation.
- the page processor 22 Z of the electronic whiteboard 2 c generates a layer of UI images based on the image data of a UI image stored in the page data memory 220 (step S 55 ). In these processes, the page processor 22 Z superimposes the U images indicated by the image data of the UI images on the transparent pixel area of the resolution (size) of the screen of the display 3 .
- the superimposition unit 22 Y of the electronic whiteboard 2 c performs rendering and sequentially superimposes the layer of a background image obtained in the step S 51 , the layer of an external image obtained in the step S 52 , the layer of object images generated in the step S 53 , and the layer of stroke images generated in the step S 54 on top of one another to generate a layer of a superimposed image excluding the UI image (step S 56 ).
- the superimposition unit 22 Y When a layer of an external image is not obtained in the step S 52 , i.e., when the laptop PC 6 is not connected to the electronic whiteboard 2 c , the superimposition unit 22 Y generates the layer of a superimposed image excluding the layer of an external image and the UI image.
- the superimposition unit 22 Y of the electronic whiteboard 2 c performs rendering and sequentially superimposes the layer of a superimposed image, excluding the UI image, generated in the step S 56 , the layer of a background image obtained in the step S 51 , the layer of an external image obtained in the step S 52 , the layer of object images generated in the step S 53 , the layer of stroke images generated in the step S 54 , and the layer of UI images generated in the step S 55 on top of one another to generate a layer of a superimposed image including the UI images (step S 57 ).
- the superimposition unit 22 Y When a layer of an external image is not obtained in the step S 52 , i.e., when the laptop PC 6 is not connected to the electronic whiteboard 2 c , the superimposition unit 22 Y generates the layer of a superimposed image excluding the layer of an external image but including the UI image.
- the superimposition unit 22 Y of the electronic whiteboard 2 c outputs a layer of a superimposed image including the UI image to the display order controller 24 (step S 58 ).
- FIG. 11A is a diagram illustrating multiple layers of a superimposed image including a UI image, according to the present embodiment.
- FIG. 11B is a diagram illustrating a superimposed image including a UI image, according to the present embodiment.
- the superimposed image 140 including the UI images includes a layer 140 a of the UI images, a layer 140 b of the stroke images, a layer 140 c of the object images, a layer 140 d of the external image, a layer 140 e of the background image, and a layer 140 f of the superimposed image excluding the UI image.
- the layer 140 e of the background image is a full screen and nontransparent
- the layer 140 f of the superimposed image excluding the UI image is invisible on the superimposed image 140 .
- the client unit 20 of the electronic whiteboard 2 c repeats the processes in the steps S 51 to S 58 . Due to this configuration, a superimposed image that is updated with the latest background image, external image, object image, stroke image, and the UI image is repeatedly output to the display order controller 24 .
- FIG. 10B is a flowchart of the processes of generating an image using the communication application B 11 , according to the present embodiment.
- the display controller 82 of the electronic whiteboard 2 obtains the image data of an updated image stored in the memory 2000 (see step S 48 ) (step S 61 ).
- the image data of an updated image indicates the image data of the latest image in the video data received by the electronic whiteboard 2 c.
- the display controller 82 of the electronic whiteboard 2 outputs the image data of an updated image obtained in the step S 61 to the display order controller 24 (step S 62 ).
- the client unit 80 of the electronic whiteboard 2 c repeats the processes in the steps S 61 and S 62 . Accordingly, the latest images that together configure a video are repeatedly output to the display order controller 24 .
- the display order controller 24 of the electronic whiteboard 2 c controls the images output from the client unit 20 and the images output from the client unit 80 , according to operation made by a user or events caused due to the communication performed by the communication applications A 11 or B 11 , such that the images output from the client unit 20 are activated and the images output from the client unit 80 are deactivated or such that the images output from the client unit 20 are deactivated and the images output from the client unit 80 are activated.
- the display order controller 24 controls the images output from the client unit 20 and the images output from the client unit 80 such that the images output from the client unit 20 are displayed in the foreground of the screen of the display 3 c or such that the images output from the client unit 80 are displayed in the foreground of the screen of the display 3 c . Due to this configuration, one of the superimposed image output in the step S 58 and the video of a video conference output in the step S 62 is actively output from the display 3 c of the electronic whiteboard 2 c.
- FIG. 12 is a sequence diagram illustrating the processes of sending the screen image on the electronic whiteboard 2 c side to the transmission terminal 8 , according to the present embodiment.
- the electronic whiteboard 2 c accepts a request to share the screen of the display 3 c with the transmission terminal 8 . Such a request may be made by operation or input by a user of the electronic whiteboard 2 c , or may be made by the transmission terminal 8 . Once such a request is accepted, the communication controller 81 of the electronic whiteboard 2 c obtains the layer of a superimposed image excluding the UI image, which is generated in the step S 56 (step S 71 ).
- the communication controller 81 of the electronic whiteboard 2 c sends the obtained layer of a superimposed image, excluding the UI image, to the relay device 30 as the display data indicating the screen that is being displayed on the electronic whiteboard 2 c side (step S 72 ).
- the relay device 30 sends the received display data to the transmission terminal 8 (step S 73 ).
- the display data that is received from the electronic whiteboard 2 c may be sent to the transmission terminal 8 upon its quality being changed according to, for example, the bandwidth of the communication network 9 b.
- the communication controller 81 of the transmission terminal 8 receives the display data sent from the relay device 30 .
- the display controller 82 of the transmission terminal 8 sends the video data on the electronic whiteboard 2 c side sent from the relay device 30 in the step S 47 and the display data of the screen that is being displayed on the electronic whiteboard 2 c sent from the relay device 30 in the step S 73 to the display 820 of the transmission terminal 8 (step S 74 ).
- FIG. 13 is a diagram illustrating an example image displayed on the display 820 on the transmission terminal 8 side in the step S 74 , according to the present embodiment.
- a video 150 of a conference participant on the electronic whiteboard 2 c side and the layer 140 f of a superimposed image, excluding the UI image, as a display screen, are displayed on the display 3 .
- the communication controller 81 of the electronic whiteboard 2 c sends the display data newly obtained in the step S 71 to the relay device 30 . Accordingly, the transmission terminal 8 receives the updated display data and outputs the received display data to the display 820 .
- the page processor 22 Z of the electronic whiteboard 2 c superimposes an input external image on the nontransparent pixel area of the resolution (size) of the screen of the display 3 to generate a layer of the external image. Due to these processes, a full screen and nontransparent layer of the external image is generated.
- the superimposition unit 22 Y of the electronic whiteboard 2 c When an external image is input from the laptop PC 6 c and a full screen and nontransparent layer of the external image is generated in the step S 53 , in the step S 56 , the superimposition unit 22 Y of the electronic whiteboard 2 c generates a superimposed image, excluding the UI images, where a layer of object images and a layer of stroke images are superimposed on the layer of the external image in that order.
- a layer of the external image is a full screen and nontransparent, even if the background image is not included in the images to be superimposed, the same superimposed image as the one generated when the background image is included in the images to be superimposed is generated. Due to this configuration, a layer of the background image is excluded from the images to be superimposed, and the load of processes on the superimposition unit 22 Y can be reduced.
- the page data memory 220 of the electronic whiteboard 2 c stores a functional configuration screen and a videoconference configuration screen.
- the page processor 22 Z of the electronic whiteboard 2 c obtains the functional configuration screen stored in the page data memory 220 , as a layer of UI images the superimposition unit 22 Y outputs a superimposed image that includes the obtained functional configuration screen to the display order controller 24 .
- the display order controller 24 outputs the image data including the functional configuration screen to the display 3 c.
- FIG. 14A is a diagram illustrating a functional configuration screen, according to the present embodiment.
- the functional configuration screen includes a key B 1 thereon for shifting to a videoconference configuration screen.
- the page processor 22 Z of the electronic whiteboard 2 c obtains the videoconference configuration screen stored in the page data memory 220 .
- the superimposition unit 22 Y outputs a superimposed image that includes the obtained videoconference configuration screen to the display order controller 24 .
- the display order controller 24 outputs the image data including the videoconference configuration screen to the display 3 c.
- FIG. 14B is a diagram illustrating a videoconference configuration screen, according to the present embodiment.
- the communication application B 11 stores in the memory 2000 the data indicating that the management system 50 is to be used for a video conference.
- the superimposition unit 22 Y of the electronic whiteboard 2 c refers to the memory 2000 and determines whether the data indicating that the management system 50 is to be used is stored therein.
- the superimposition unit 22 Y when it is determined that the data indicating that the management system 50 is to be used is stored, the superimposition unit 22 Y generates a superimposed image excluding the UI images.
- the superimposition unit 22 Y when it is determined that the data indicating that the management system 50 is to be used is not stored, the superimposition unit 22 Y does not generate a superimposed image excluding the UI images.
- a communication terminal such as a PC may be connected to the communication network 9 a or 9 b , and such a communication terminal may be allowed to access a desired electronic whiteboard 2 by inputting the IP address of the desired electronic whiteboard 2 through a Web browser. Due to this configuration, a videoconference configuration screen of the electronic whiteboard 2 may be displayed on the communication terminal such as a PC, and the above items become configurable.
- FIG. 15 is a diagram illustrating a schematic configuration of a communication system 1 ′, according to a fourth embodiment of the present disclosure.
- the communication system 1 ′ further includes a data server 60 and a telephone terminal 7 in addition to the devices and apparatuses in the communication system 1 .
- the communication system 1 ′ includes a communication network 9 c that serves as a mobile phone network.
- the communication network 9 c is connected to the communication network 9 b , i.e., the Internet, through a gateway.
- a base station is installed at the end of the communication network 9 c .
- a wireless LAN router that serves as the access point of radio communication is installed.
- the telephone terminal 7 is, for example, a mobile communication device such as a smartphone, and is connected to the communication network 9 c through a base station to communicate with another telephone terminal 7 . Further, the telephone terminal 7 communicates with the data server 60 and the electronic whiteboard 2 by connecting to the communication network 9 b through a wireless LAN router or by connecting to the communication network 9 b through a base station, the communication network 9 c , and a gateway.
- the data server 60 is connected to the communication network 9 b .
- the data server 60 stores the data uploaded from any client communication device. Moreover, the data server 60 accepts a request to download data from any client communication device.
- FIG. 16 is a schematic block diagram illustrating a hardware configuration of a telephone terminal 7 , according to the present embodiment.
- the telephone terminal 7 is provided with a central processing unit (CPU) 101 that controls the overall operation of the telephone terminal 7 , a read-only memory (ROM) 102 storing a program, a random access memory (RAM) 103 that is used as a work area in which the CPU 101 executes a program, an electrically erasable and programmable read only memory (EEPROM) 104 that controls reading or writing of various kinds of data under control of the CPU 101 , and a medium drive 107 that controls reading or writing of data to or from a recording medium 106 such as a flash memory.
- CPU central processing unit
- ROM read-only memory
- RAM random access memory
- EEPROM electrically erasable and programmable read only memory
- medium drive 107 that controls reading or writing of data to or from a recording medium 106 such as a flash memory.
- the recording medium 106 is removable from the medium drive 107 , and data recorded in the recording medium 106 is read therefrom or data is newly written and stored on the recording medium 106 under control of the medium drive 107 .
- the EEPROM 104 stores an operating system (OS) to be executed by the CPU 101 , other various kinds of programs, and various types of data.
- OS operating system
- the telephone terminal 7 is provided with an antenna 111 a , and a communication circuit 111 that uses the antenna 111 a and radio communication signals to communicate with a nearest base station.
- the telephone terminal 7 is provided with a complementary metal oxide semiconductor (CMOS) sensor 112 that captures an object under the control of the CPU 101 to obtain captured-image data, a sound input 114 that converts the sound into an audio signal, and a sound output 115 that converts the audio signal into sound.
- CMOS sensor 112 may be a charge coupled device that converts an image of an object into electronic data through photoelectric conversion. In other words, as long as it can capture an object, the CMOS sensor may be replaced with a charge coupled device (CCD) sensor.
- CCD charge coupled device
- the sound input 114 includes a microphone for receiving sound
- the sound output 115 includes a loudspeaker to output sound.
- the telephone terminal 7 is provided with a display 120 such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display that display an image of an object, various kinds of icons, or the like, and a touch panel 121 that is made of a pressure-sensing or electrostatic panel and disposed on the display 120 .
- the touch panel 121 detects a touch point on the display 120 by a finger, an electronic stylus, or the like.
- the telephone terminal 7 is provided with a bus line 110 such as an address bus or a data bus that electrically connects the elements as illustrated in FIG. 16 .
- the hardware configuration of the data server 60 is equivalent to the hardware configuration of the management system 50 , and thus its description is omitted.
- FIG. 17A is a functional block diagram of the electronic whiteboard 2 c , according to the fourth embodiment.
- the client unit 20 of the electronic whiteboard 2 c includes an input acceptance unit 23 and a code processor 25 .
- the input acceptance unit 23 receives various kinds of operation or input made by a user according to the instructions from the CPU 201 , or operation of the sensor controller 214 and the contact sensor 215 .
- the code processor 25 generate a bar code used to download data according to operation or input made by a user. Such a bar code used to download data will be described later in detail.
- FIG. 17B is a functional block diagram of the data server 60 , according to the fourth embodiment.
- the data server 60 includes a communication controller 61 and a data processor 69 . Moreover, the data server 60 includes a memory 6000 that is configured by the HD 504 illustrated in FIG. 3 .
- the memory 6000 stores various kinds of data uploaded from any client communication device.
- Such various kinds of data includes, for example, image data such as of Joint Photographic Experts Group (JPEG), tagged image file fbrmat (TIFF), portable network graphics (PNG), or bitmap (BMP), video data such as of Moving Picture Experts Group (MPEG)-4, a computer aided design (CAD) file, a text, or a file used by a word processing program, a spreadsheet program, a presentation program, or a viewer.
- JPEG Joint Photographic Experts Group
- TIFF tagged image file fbrmat
- PNG portable network graphics
- BMP bitmap
- MPEG Moving Picture Experts Group
- MPEG computer aided design
- CAD computer aided design
- the communication controller 61 may be implemented by the instructions from the CPU 201 and the network interface 205 , and controls communication performed with the electronic whiteboard 2 or the telephone terminal 7 .
- the data processor 69 may be implemented by the instructions from the CPU 201 and the HDD 505 , and controls the memory 6000 to store various kinds of data or reads various kinds of data from the memory 6000 .
- FIG. 17C is a functional block diagram of the telephone terminal 7 , according to the fourth embodiment.
- the telephone terminal 7 includes a communication controller 71 , an imager 72 , an analyzer 73 , and a data processor 79 . Moreover, the telephone terminal 7 includes the ROM 102 , the RAM 103 , the EEPROM 104 , or the memory 7000 implemented by the recording medium 108 .
- the communication controller 61 may be implemented by the instructions from the CPU 101 and the communication circuit 11 l , and controls communication performed with the electronic whiteboard 2 or the data server 60 .
- the imager 72 may be implemented by the CMOS sensor 112 , and captures any desired object such as a bar code displayed on the display 3 c of the electronic whiteboard 2 c.
- the analyzer 73 may be implemented by the instructions from the CPU 101 , and analyzes a bar code captured by the imager 72 to extracts code data from the bar code. Such code data extracted from a bar code will be described later.
- FIG. 18 is a sequence diagram illustrating the processes of downloading the data of a background image by the electronic whiteboard 2 c , according to the fourth embodiment of the present disclosure.
- the input acceptance unit 23 of the electronic whiteboard 2 c accepts a request to download a background image (step S 101 ).
- a request to download a background image will be referred to simply as a downloading request.
- the code processor 25 generates a bar code to be used to download a background image (step S 102 ).
- the code processor 25 generates a bar code that indicates, based on predetermined standards, code data such as identification information indicating a downloading request, identification information of a background image as the data to be downloaded, the destination address of the background image, and a password.
- predetermined standards includes, for example, Japanese Article Number (JAN), European Article Number (EAN), Universal Product Code (UPC), Interleaved Two of Five (ITF), CODE 39, CODE 128, and NW-7.
- the identification information of a background image is, for example, a file name
- the destination address to which the image data is to be sent is, for example, a uniform resource locator (URL) that indicates the electronic whiteboard 2 c
- the password is, for example, prescribed in advance by the electronic whiteboard 2 c .
- a part of or the entirety of the code data may be input by a user of the electronic whiteboard 2 c.
- the page processor 22 Z of the electronic whiteboard 2 c generates a layer of UI images including the UI images stored in the page data memory 220 and the bar code generated in the step S 102 .
- the superimposition unit 22 Y of the electronic whiteboard 2 c outputs a superimposed image including the layer of UI images and the bar code to the display order controller 24 .
- the display order controller 24 controls the display 3 c to display the superimposed image including the layer of UI images and the bar code (step S 103 ).
- FIG. 19 is a diagram illustrating an example image on the display in the step S 103 , according to the present embodiment.
- the imager 72 of the telephone terminal 7 captures a bar code displayed on the display 3 c of the electronic whiteboard 2 c according to operation made by a user (step S 104 )
- the analyzer 73 of the telephone terminal 7 analyzes the captured bar code. More specifically, in accordance with the prescribed rules, the analyzer 73 extracts, from the bar code included in the captured image, code data such as identification information indicating a downloading request, identification information of a background image, the destination address of the background image, and a password (step S 105 ). Note also that the password out of the extracted code data is used later in the processes of authentication. For this reason, the password is displayed on the display 120 in accordance with the instructions given from the CPU 101 .
- a password is not included in the code data, and thus such a password does not need to be displayed on the electronic whiteboard 2 c .
- a password may be input, for example, by a user of the telephone terminal 7 , or may be stored in the memory 7000 of the telephone terminal 7 as setting data of an application.
- the communication controller 71 of the telephone terminal 7 sends the extracted code data to the data server 60 (step S 106 ).
- the password input by a user or the password stored in the memory 7000 is sent to the data server 60 .
- the communication controller 61 of the data server 60 receives the code data sent from the telephone terminal 7 .
- the data processor 69 of the data server 60 obtains the data of a background image identified by the identification information included in the code data from the memory 6000 (step S 107 ).
- the communication controller 61 of the data server 60 sends the obtained data of the background image and the password included in the code data to the electronic whiteboard 2 c that is identified by the destination address included in the code data (step S 108 ).
- the communication controller 21 of the electronic whiteboard 2 c receives the data of the background image sent from the data server 60 .
- the electronic whiteboard 2 c downloads the data of a background image from the data server 60 .
- the input acceptance unit 23 accepts the input of a password.
- the code processor 25 of the electronic whiteboard 2 c determines whether the password received by the input acceptance unit 23 matches the password included in the code data generated in the step S 102 . When these passwords match one another, the code processor 25 authenticates the user (step S 109 ). Once the user is authenticated, the page processor 22 Z of the electronic whiteboard 2 c stores the downloaded data of a background image in the page data memory 220 (step S 110 ).
- the page processor 22 Z of the electronic whiteboard 2 c obtains the downloaded data of a background image from the page data memory 220 in the step S 51 .
- the superimposition unit 22 Y of the electronic whiteboard 2 c performs rendering and generates a superimposed image including the downloaded data of a background image, and outputs the generated superimposed image to the display order controller 24 .
- the display order controller 24 of the electronic whiteboard 2 c controls the display 3 c to display a superimposed image including the downloaded data of a background image (step S 111 ).
- the superimposition unit 22 Y of the electronic whiteboard 2 c performs rendering and generates a superimposed image including the downloaded data of a background image but excluding the UI images.
- the communication controller 81 of the electronic whiteboard 2 c obtains the superimposed image including the downloaded data of a background image but excluding the UI images.
- the communication controller 81 of the electronic whiteboard 2 c sends the superimposed image including the downloaded data of a background image but excluding the UI images to the relay device 30 as display data.
- the relay device 30 relays the display data sent from the electronic whiteboard 2 c to the transmission terminal 8 , and the transmission terminal 8 receives the display data sent from the relay device 30 and controls the display 820 to display an image.
- the display data does not include the UI images, the bar code that is displayed on the electronic whiteboard 2 c side as a UI image is not displayed on the transmission terminal 8 side.
- the data server 60 in the communication system 1 ′ according to the fourth embodiment may be omitted.
- the memory 7000 of the telephone terminal 7 stores various kinds of data uploaded from any client communication device. Such various kinds of data is equivalent to the data that the data server 60 according to the fourth embodiment stores in the memory 6000 .
- FIG. 20 is a sequence diagram illustrating the processes of downloading the data of a background image by the electronic whiteboard 2 c , according to the fifth embodiment of the present disclosure.
- the data processor 79 obtains the data of a background image identified by the identification information of the background image included in the code data from the memory 7000 (step S 107 ′)
- the communication controller 71 of the telephone terminal 7 sends the data of the background image and the password included in the code data to the electronic whiteboard 2 c that is identified by the destination address included in the code data (step S 108 ′).
- the communication controller 21 of the electronic whiteboard 2 c receives the data of the background image sent from the telephone terminal 7 .
- the electronic whiteboard 2 c downloads the data of a background image from the telephone terminal 7 .
- the stroke processing unit 22 A of the electronic whiteboard 2 c accepts a drawing operation.
- the display order controller 24 (an example of a display controller) of the electronic whiteboard 2 c controls the display 3 c (an example of a display) to display on the screen of the display 3 c (an example of display controlling processes) an image drawn by operation accepted by the stroke processing unit 22 A and a first superimposed image where the UI images related to drawing operation are superimposed on the background image.
- the communication controller 81 (an example of a transmitter) of the electronic whiteboard 2 c sends the display data (an example of image data) of a second superimposed image where the UI images are excluded from the images to be superimposed and the drawn images are superimposed on the background image to the transmission terminal 8 (an example of an external communication terminal) (an example of transmitting processes) as the image data to be displayed on the screen of the display. Due to this configuration, when the electronic whiteboard 2 c capable of displaying the drawn images transmits its screen image to the transmission terminal 8 , the UI images that cannot be manipulated via the transmission terminal 8 is no longer output to the transmission terminal 8 , and reduction in viewability can be prevented.
- the electronic whiteboard 2 transmits an image where the drawn images are superimposed on the background image to the transmission terminal 8 , even if the background color set on the transmission terminal 8 side is the same as the color of a drawn image, a user of the transmission terminal 8 can read the drawn image.
- the superimposition unit 22 Y (i.e., an example of a generator) of the electronic whiteboard 2 c performs rendering and generates a layer of a second superimposed image (i.e., an example of a composite image) where a layer of drawn images are superimposed on a layer of the background image. Note that this generation of a layer of a second superimposed image is an example of a generating process.
- the communication controller 81 of the electronic whiteboard 2 c sends the display data of the second superimposed image generated by the superimposition unit 22 Y to the transmission terminal 8 . Due to this configuration, the communication controller 81 can obtain the image of a screen generated by the superimposition unit 22 Y and can send the obtained image to the transmission terminal 8 .
- the display order controller 24 of the electronic whiteboard 2 c controls the display 3 c to display a layer of a first superimposed image where a layer of the background image, a layer of drawn images, and a layer of UI images are superimposed on a layer of a second superimposed image in that order.
- the layer of the second superimposed image is arranged underneath the background image. Accordingly, the electronic whiteboard 2 c can make the layer of the second superimposed image invisible.
- the page data memory 220 (an example of a manager) of the electronic whiteboard 2 c manages the image data of the background image (an example of a first background image).
- the page processor 22 Z (an example of an input unit) of the electronic whiteboard 2 c inputs an external image (an example of a second background image) from the laptop PC 6 .
- the superimposition unit 22 Y of the electronic whiteboard 2 c combines a layer of the background image and a layer of drawn images to generate a layer of a second superimposed image.
- the superimposition unit 22 Y of the electronic whiteboard 2 c combines a layer of the external image and a layer of drawn images to generate a layer of a second superimposed image. Due to this configuration, the load of processes on the electronic whiteboard 2 c in generating a layer of a second superimposed image can be reduced.
- the superimposition unit 22 Y of the electronic whiteboard 2 c does not generate a second superimposed image. Due to this configuration, when the transmission of the display data to the transmission terminal 8 is not expected, the generation of the second superimposed image is omitted, and the load of processes on the electronic whiteboard 2 c can be reduced.
- a communication terminal such as a smartphone may be connected to the communication network 9 a , and may transfer the file of a background image stored in the local communication terminal to any one of the electronic whiteboards 2 , using, for example, a web application programming interface (API).
- the electronic whiteboard 2 receives the transferred file, and displays the background image in the received file as the background image of the screen.
- a file that is stored in a communication terminal may be a file that is generated by the scanning function of an image forming apparatus such as a multifunction peripheral (MFP) and then transferred from the image forming apparatus to the communication terminal.
- MFP multifunction peripheral
- the image forming apparatus may be connected to the communication network 9 a , and may directly transfer a file generated by scanning to the communication terminal, using an email sending function of the image forming apparatus.
- the image forming apparatus may transfer a file generated by scanning to a file server connected to the communication network 9 a , using a remote folder storing function of the image forming apparatus, and such transferred files may be accumulated in the file server.
- the communication terminal can store a desired file in the communication terminal by obtaining such a desired file from the files accumulated in the file server.
- the communication applications A 11 and B 11 may be recorded on a computer-readable recording medium for distribution in an installable or executable file format.
- a recording medium include, but not limited to, a compact disc-recordable (CD-R), a digital versatile disc (DVD), and a Blu-ray disc.
- a memory storing any one of the above-described control programs, such as a recording medium including a CD-ROM or a HDD, may be provided in the form of a program product to users within a certain country or outside that country.
- Any one of the electronic whiteboard 2 , the transmission terminal 8 , the relay device 30 , and the management system 50 according to the embodiments described above may be implemented by a single apparatus or a plurality of apparatuses to which divided elements (functions or units) are allocated.
- Processing circuitry includes a programmed processor, as a processor includes circuitry.
- a processing circuit also includes devices such as an application specific integrated circuit (ASIC), digital signal processor (DSP), field programmable gate array (FPGA), and conventional circuit components arranged to perform the recited functions.
- ASIC application specific integrated circuit
- DSP digital signal processor
- FPGA field programmable gate array
- processing circuit includes, for example, a device such as a processor that may be provided with an electronic circuit and is programmed to execute software to implement functions, an application specific integrated circuit (ASIC) that is designed to execute the above functions, and a conventional circuit module.
- any one of the above-described and other methods of the present invention may be embodied in the form of a computer program stored on any kind of storage medium.
- storage media include, but are not limited to, flexible disks, hard disks, optical discs, magneto-optical discs, magnetic tape, nonvolatile memory cards, ROM, etc.
- any one of the above-described and other methods of the present invention may be implemented by ASICs, prepared by interconnecting an appropriate network of conventional component circuits, or by a combination thereof with one or more conventional general-purpose microprocessors and/or signal processors programmed accordingly.
Abstract
A communication terminal, a communication system, a transmission method, and a recording medium storing program. The communication terminal and the transmission method includes controlling a display to display a first superimposed image where a drawn image and a user interface image related to drawing operation are superimposed on a background image, and transmitting image data of a second superimposed image, where the drawn image is superimposed on the background image and the user interface image is excluded from images to be superimposed, to an external communication terminal as image data to be displayed on the display. The communication system includes the communication terminal and the external communication terminal. The recording medium stores a program for causing a computer to execute the transmission method.
Description
- This patent application is based on and claims priority pursuant to 35 U.S.C. § 19(a) to Japanese Patent Application Nos. 2016-193251 and 2017-099616, filed on Sep. 30, 2016, and May 19, 2017, respectively, in the Japan Patent Office, the entire disclosures of which are hereby incorporated by reference herein.
- Embodiments of the present disclosure relate to a communication terminal, a communication system, a transmission method, and a recording medium storing program.
- With the need for reducing costs or times associated with business trips in recent years, communication systems are widely used, which are capable of carrying out videoconferences among remotely located sites through a communication network such as the Internet or private lines. As communication terminals in a communication system, for example, electronic whiteboards that transmit stroke data such as of a drawn text, or video conference terminals that transmit video data such as of a conference participant are known in the art. In such a communication system as above, as long as a communication terminal on one side is installed with an application that generates a screen image from certain contents of data, even if a communication terminal on the other side is not installed with the above application, the communication terminal on one side can perform rendering on the screen image and transmit it to the communication terminal on the other side such that the screen will be shared among multiple communication terminals.
- Embodiments of the present disclosure described herein provide a communication terminal, a communication system, a transmission method, and a recording medium storing program. The communication terminal and the transmission method includes controlling a display to display a first superimposed image where a drawn image and a user interface image related to drawing operation are superimposed on a background image, and transmitting image data of a second superimposed image, where the drawn image is superimposed on the background image and the user interface image is excluded from images to be superimposed, to an external communication terminal as image data to be displayed on the display. The communication system includes the communication terminal and the external communication terminal. The recording medium stores a program for causing a computer to execute the transmission method.
- A more complete appreciation of exemplary embodiments and the many attendant advantages thereof will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in connection with the accompanying drawings.
-
FIG. 1 is a diagram illustrating a schematic network configuration of a communication system, according to an embodiment of the present invention. -
FIG. 2 is a schematic block diagram illustrating a hardware configuration of an electronic whiteboard, according to an embodiment of the present disclosure. -
FIG. 3 is a schematic block diagram illustrating a hardware configuration of a management, system according to an embodiment of the present disclosure. -
FIG. 4 is a schematic block diagram illustrating a hardware configuration of a transmission terminal according to an embodiment of the present disclosure. -
FIG. 5A andFIG. 5B are schematic block diagrams each illustrating a software configuration of an electronic whiteboard or a transmission terminal, respectively, according to an embodiment of the present disclosure. -
FIG. 6 is a functional block diagram of a communication system, according to an embodiment of the present disclosure. -
FIG. 7 is a functional block diagram of a drawing controller according to an embodiment of the present disclosure. -
FIG. 8 is a sequence diagram illustrating the processes of establishing communication among a plurality of electronic whiteboards, according to an embodiment of the present disclosure. -
FIG. 9 is a sequence diagram illustrating the processes of establishing communication between an electronic whiteboard and a transmission terminal, according to an embodiment of the present disclosure. -
FIG. 10A andFIG. 10B are data sequence diagrams each illustrating the processes of generating an image, according to an embodiment of the present disclosure. -
FIG. 11A andFIG. 11B are diagrams each illustrating a superimposed image, according to an embodiment of the present disclosure. -
FIG. 12 is a sequence diagram illustrating the processes of sending the screen image on an electronic whiteboard side to a transmission terminal, according to an embodiment of the present disclosure. -
FIG. 13 is a diagram illustrating an example display according to an embodiment of the present disclosure. -
FIG. 14A andFIG. 14B are diagrams each illustrating an example display according to an embodiment of the present disclosure. -
FIG. 15 is a diagram illustrating a schematic configuration of a communication system, according to a fourth embodiment of the present disclosure. -
FIG. 16 is a schematic block diagram illustrating a hardware configuration of a telephone terminal, according to an embodiment of the present disclosure. -
FIG. 17A ,FIG. 17B , andFIG. 17C are functional block diagrams of an electronic whiteboard, a data server, and a telephone terminal, respectively, according to a fourth embodiment of the present disclosure. -
FIG. 18 is a sequence diagram illustrating the processes of downloading the data of a background image by an electronic whiteboard, according to a fourth embodiment of the present disclosure. -
FIG. 19 is a diagram illustrating an example display according to an embodiment of the present disclosure. -
FIG. 20 is a sequence diagram illustrating the processes of downloading the data of a background image by an electronic whiteboard, according to a fifth embodiment of the present disclosure. -
FIG. 21 is a diagram illustrating stroke arrangement data according to an embodiment of the present disclosure. -
FIG. 22 is a diagram illustrating an example data structure of the coordinate arrangement data, according to an embodiment of the present disclosure. - The accompanying drawings are intended to depict exemplary embodiments of the present disclosure and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted.
- The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “includes” and/or “including”, when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
- In describing example embodiments shown in the drawings, specific terminology is employed for the sake of clarity. However, the present disclosure is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have the same structure, operate in a similar manner, and achieve a similar result.
- In the following description, illustrative embodiments will be described with reference to acts and symbolic representations of operations (e.g., in the form of flowcharts) that may be implemented as program modules or functional processes including routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types and may be implemented using existing hardware at existing network elements or control nodes. Such existing hardware may include one or more central processing units (CPUs), digital signal processors (DSPs), application-specific-integrated-circuits (ASICs), field programmable gate arrays (FPGAs), computers or the like. These terms in general may be collectively referred to as processors.
- Unless specifically stated otherwise, or as is apparent from the discussion, terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical, electronic quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
- In the following description, a first embodiment of the present invention is described with reference to the drawings.
- <<Schematic Configuration of Communication System>>
- Firstly, a
communication system 1 is described with reference toFIG. 1 . -
FIG. 1 is a diagram illustrating a schematic configuration of thecommunication system 1, according to an embodiment of the present disclosure. - The
communication system 1 according to the present embodiment includes a plurality ofelectronic whiteboards transmission terminal 8, arelay device 30, and amanagement system 50. - The
electronic whiteboards communication network 9 a. Moreover, theelectronic whiteboards displays - The
electronic whiteboards displays electronic styluses displays electronic styluses electronic whiteboards displays displays electronic styluses electronic whiteboards electronic whiteboard 2, and any one of thedisplays display 3. Likewise, any one of theelectronic styluses electronic stylus 4, and any one of the user's hands Ha. Hb, and He may be referred to simply as the hand H. - To the
electronic whiteboards memories 5 a, 5 b, and 5 c may be connected. Theelectronic whiteboards USB memories 5 a, 5 b, and 5 c, or can store such electronic files in theUSB memories 5 a, 5 b, and 5 c. Moreover, laptop personal computers (PCs) 6 a, 6 b, and 6 c are connected to theelectronic whiteboards USB memories 5 a, 5 b, and 5 c may be referred to simply as the USB memory 5, and any one of thelaptop PCs - Due to the configuration as described above, the stroke data of a stroke that is drawn on the
display 3 of theelectronic whiteboard 2 at one site is transmitted to theelectronic whiteboard 2 at another site through thecommunication network 9 a, and the stroke data can also be displayed on thedisplay 3 at the other site. In a similar manner, the stroke data that is drawn on thedisplay 3 of theelectronic whiteboard 2 at the other site is transmitted to theelectronic whiteboard 2 at the one site through thecommunication network 9 a, and the stroke data can be displayed on thedisplay 3 of theelectronic whiteboard 2 at the one site. As described above, drawing images can be shared among remote sites, such that thecommunication system 1 with the multipleelectronic whiteboards 2 can facilitate a video conference. - Note also that any desired external communication terminal other than the
electronic whiteboard 2 as above may additionally be connected to thecommunication network 9 a to share stroke data with theelectronic whiteboard 2. Such any desired external communication terminal includes a personal computer (PC) with a stroke sharing function, a video conference terminal, a tablet personal computer (PC), a smartphone, digital signage, a telestrator that is often used, for example, in sports and weather broadcasts, or a computer including, for example, an image processing device such as a remote image (video) diagnostic device, a mobile information terminal, a digital video camera, a digital camera, and a terminal such as a game machine capable of providing image frames - The
transmission terminal 8 is an example of a communication terminal for video conference capable of transmitting video data and audio data. Thetransmission terminal 8 may be replaced with a tablet personal computer (PC), a smartphone, digital signage, a telestrator, an image processing device, a mobile information terminal, a digital video camera, a digital camera, and a communication terminal capable of providing image frames such as a game machine. Theelectronic whiteboard 2 c out of theelectronic whiteboards electronic whiteboard 2 c and thetransmission terminal 8 can exchange the video data and audio data for video conference through thecommunication network 9 b. - The
relay device 30 relays the video data and audio data transmitted by one of theelectronic whiteboard 2 c and thetransmission terminal 8 to the other one of theelectronic whiteboard 2 c and thetransmission terminal 8. - The
management system 50 centrally manages, for example, the communication status between theelectronic whiteboard 2 c and thetransmission terminal 8. - According to the present embodiment, the
communication network 9 a is an intranet, and thecommunication network 9 b is the Internet. However, thecommunication networks communication networks communication networks - <Hardware Configuration of Electronic Whiteboard>
- The hardware configuration of the
electronic whiteboard 2 according to the present embodiment is described with reference toFIG. 2 . -
FIG. 2 is a schematic block diagram illustrating a hardware configuration of theelectronic whiteboard 2 according to the present embodiment. - As illustrated in
FIG. 2 , theelectronic whiteboard 2 includes, for example, a central processing unit (CPU) 201 that controls the overall operation of theelectronic whiteboard 2, a read only memory (ROM) 202 that stores a control program used for controlling theCPU 201 such as an initial program loader (IPL), a random access memory (RAM) 203 that is used as a work area for theCPU 201, a solid state disk (SSD) 204 that stores various data such as a control program for theelectronic whiteboard 2, a network interface (I/F) 205 that controls the communication with an external device through the communication network 9, and an external memory controller 206 that controls the communication with an external device such as a universal serial bus (USB) memory 5. Further, theelectronic whiteboard 2 includes, for example, acamera 207 that captures a target object under the control of theCPU 201, an imaging device interface (I/F) 208 that controls the operation of thecamera 207, acapture device 211 that controls the display of a laptop personal computer (PC) 6 to display the video data as a still image or moving images, a graphics processing unit (GPU) 212 specializing in processing graphics, and adisplay controller 213 that controls the visual display to output the image generated by theGPU 212 to thedisplay 3. Further, theelectronic whiteboard 2 includes, for example, asensor controller 214 that controls operation of thecontact sensor 215, and acontact sensor 215 that detects a touch onto thedisplay 3, which is made by, for example, anelectronic stylus 4 and a user's hand H. Thecontact sensor 215 according to the present embodiment senses a touch input to a specific coordinate on thedisplay 3 using the infrared blocking system. More specifically, thedisplay 3 is provided with two light receivers and light emitters disposed on both upper ends of thedisplay 3, and a reflector frame disposed on the periphery of thedisplay 3. The light receivers and light emitters emit a plurality of infrared rays in parallel to a touch panel of thedisplay 3. The light receivers and light emitters receive lights passing in the direction that is the same as that of an optical path of the emitted infrared rays, which are reflected by the reflector frame. By so doing, a touch input to specific coordinates is sensed. Thecontact sensor 215 outputs to thesensor controller 214 the identification (ID) of the infrared light rays that are emitted from the two light receivers and light emitters and then are blocked by an object such as the hand H, and thesensor controller 214 specifies the coordinates of the position touched by the object. Note also that all sorts of ID as described below are an example of identification information. - In the present embodiment, the
contact sensor 215 uses the infrared blocking system. However, no limitation is intended therein, and thecontact sensor 215 may use various types of detectors such as a capacitive touch panel that detects changes in capacitance to specify a touched position, a resistive touch panel that senses changes in voltage between two facing resistive membranes to specify a touched position, and an electromagnetic induction type touch panel that detects electromagnetic induction caused when an object touches a display to specify a touched position. Alternatively, thecontact sensor 215 may identify the contact position using thecamera 207. - The
electronic whiteboard 2 is provided with anelectronic stylus controller 216. Theelectronic stylus controller 216 communicates with theelectronic stylus 4 to detect a touch by the tip or bottom of theelectronic stylus 4 to thedisplay 3. In addition or in alternative to detecting a touch by the tip or bottom of theelectronic stylus 4, theelectronic stylus controller 216 may also detect a touch by another part of theelectronic stylus 4, such as a part held by a hand. - Further, the
electronic whiteboard 2 includes, for example, an audio input and output interface (I/F) 224 that controls the input and output of an audio signal between anexternal microphone 222 and anexternal loudspeaker 223 under the control of theCPU 101, anoperation key 225 that accepts user's operation, and a power switch 226 that turns on or turns off the power of theelectronic whiteboard 2. Note also that any of themicrophone 222 and theloudspeaker 223 may be built into theelectronic whiteboard 2. - Further, the
electronic whiteboard 2 includes, for example, abus line 230, such as an address bus or a data bus, which electrically connects the elements illustrated inFIG. 2 to one another. - The control program for the
electronic whiteboard 2 may be stored on a computer-readable recording medium such as a CD-ROM for distribution, or stored on a server on any desired network to be downloaded. - <Hardware Configuration of Management System>
- The hardware configuration of the
management system 50 according to the present embodiment is described with reference toFIG. 3 . -
FIG. 3 is a schematic block diagram illustrating a hardware configuration of themanagement system 50 according to the present embodiment. - The
management system 50 according to the present embodiment includes aCPU 501 that controls the entire operation of themanagement system 50, aROM 502 storing a transmission control program, aRAM 503 that is used as a work area for theCPU 501, a hard disk (HD) 504 storing various kinds of data, a hard disk drive (HDD) 505 that controls reading or writing of various kinds of data to or from the HI) 504 under control of theCPU 501, and a medium interface (I/F) 507 that controls reading or writing of data to or from arecording medium 506 such as a flash memory. Moreover, themanagement system 50 according to the present embodiment includes a display interface (I/F) 508 that controls thedisplay 516 to display various kinds of information such as a cursor, a menu, a window, a character, and an image, a network interface (I/F) 509 that transmits data using the communication network 9, a keyboard 511 that is provided with a plurality of keys for allowing a user to input, for example, characters, numerical values, and various kinds of instructions, amouse 512 that is one example of pointing device for allowing a user to select or execute various kinds of instructions, select a target to be processed, or to move a mouse cursor being displayed. Further, themanagement system 50 according to the present embodiment includes a compact disc read only memory (CD-ROM) drive 514 that reads or writes various kinds of data to and from a CD-ROM 513, which is one example of removable recording medium, an external device interface (I/F) 515 that exchanges data with an external device, and abus line 510, such as an address bus or a data bus, which electrically connects the elements illustrated inFIG. 2 to one another. - The above transmission control program is in an installable or executable file format, and may be recorded on a computer-readable recording medium such as the
recording medium 506 and the CD-ROM 513 for distribution. The above transmission control program may be stored on theHD 504. - The hardware configuration of the
relay device 30 is equivalent to that of themanagement system 50, and thus its description is omitted. - <Hardware Configuration of Transmission Terminal>
-
FIG. 4 is a schematic block diagram illustrating a hardware configuration of thetransmission terminal 8 operating as a video conference terminal, according to the present embodiment. - The
transmission terminal 8 includes, for example, a central processing unit (CPU) 801, a read only memory (ROM) 802, a random access memory (RAM) 803, aflash memory 804, a solid state drive (SSD) 805, a medium interface (I/F) 807, apower switch 809, abus line 810, a network interface (I/F) 811, acamera 812, an imaging device interface (I/F) 813, amicrophone 814, aloudspeaker 815, an audio input and output interface (I/F) 816, a display interface (I/F) 817, an external device connection interface (I/F) 818, akeyboard 821, and amouse 822. TheCPU 801 controls the overall operation of thetransmission terminal 8. TheROM 802 stores a control program such as an initial program loader (IPL) used for operating theCPU 801. TheRAM 803 is used as a work area for theCPU 801. Theflash memory 804 stores various kinds of data such as a communication control program, display data, and audio data. TheSSD 805 controls reading or writing of various kinds of data to or from theflash memory 804 under control of theCPU 801. As an alternative to the SSD, a hard disk drive (HDD) may be used. Themedium interface 807 controls reading or writing of data to or from arecording medium 806 such as a flash memory. - The
network interface 811 controls data communication with an external device through thecommunication network 9 b. Thecamera 812 is an example of a built-in imaging device that captures an object under the control of theCPU 801 to obtain display data. Theimaging device interface 813 is a circuit that controls operation of thecamera 812. Themicrophone 814 is an example of a built-in sound collector capable of receiving audio data. The audio input andoutput interface 816 is a circuit for controlling an input and output of an audio signal between themicrophone 814 and theloudspeaker 815, under control of theCPU 801. Thedisplay interface 817 is a circuit for transmitting display data to theexternal display 820 under control of theCPU 801. The externaldevice connection interface 818 is an interface circuit that connects thetransmission terminal 8 to various kinds of external devices. Thekeyboard 821 is one example of input device provided with a plurality of keys for allowing an user to input characters, numerical values, or various kinds of instructions. Themouse 822 is one example of pointing device for allowing a user to select or execute various kinds of instructions, select a target to be processed, or to move a mouse cursor being displayed. - The
bus line 810 is, for example, an address bus or a data bus, which electrically connects a plurality of elements as illustrated inFIG. 4 such as theCPU 801. - The
display 820 is an example of a display unit composed of a liquid crystal or organic electroluminescence (EL) display that displays, for example, an image of an object and an operation icon. Moreover, thedisplay 820 is connected to thedisplay interface 817 by acable 820 c. Thecable 820 c may be an analog RGB (red, green, blue) (video graphic array (VGA)) signal cable, a component video cable, a high-definition multimedia interface (HDMI, registered trademark) signal cable, or a digital video interactive (DVI) signal cable. - The
camera 812 includes a lens, and a solid-state image sensing device that converts an image or moving images (video) of an object into electronic data through photoelectric conversion. As the solid-state image sensing device, for example, a complementary metal oxide semiconductor (CMOS) or a charge-coupled device (CCD) is used. The externaldevice connection interface 818 is capable of connecting an external device such as an external camera, an external microphone, or an external loudspeaker thereto through a universal serial bus (USB) cable or the like. - The
recording medium 806 is removable from thetransmission terminal 8. Theflash memory 804 may be replaced with any nonvolatile memory such as an electrically erasable and programmable read only memory (EEPROM) that reads or writes data under control of theCPU 801. - <Software Configuration of Communication Terminal>
-
FIG. 5A is a schematic block diagram illustrating a software configuration of theelectronic whiteboard 2, according to the present embodiment. - As illustrated in
FIG. 5A , theelectronic whiteboard 2 is installed with an operating system (OS) 2020, a first communication application A11, and a second communication application B11, each of which operates on awork area 2010 of theRAM 203. - The
OS 2020 is basic software that controls entire operation of theelectronic whiteboard 2 through providing basic functions. The communication application A11 and B11 each enable theelectronic whiteboard 2 to communicate with another electronic whiteboard (or communication terminal), using different communication protocols. The communication application A11 is a client application that provides theelectronic whiteboard 2 with a communication control function of controlling transmission of stroke data to another communication terminal, and an image processing function of outputting the stroke data as an image. The communication application B11 is a client application that provides theelectronic whiteboard 2 with a communication control function of sending and receiving the video data or audio data of a conference or the like to or from another communication terminal, an image processing function of inputting and outputting the video data, and an audio processing function of inputting and outputting the audio data. - The
OS 2020 and the communication application A11 are installed onto theelectronic whiteboards electronic whiteboard 2 as desired after shipment. In the following description, it is assumed that theelectronic whiteboard 2 c is installed with the communication application B11, but theelectronic whiteboards -
FIG. 5B is a schematic block diagram illustrating a software configuration of thetransmission terminal 8, according to the present embodiment. - As illustrated in
FIG. 5B , theOS 8020 and the third communication application B11 are deployed and operate on awork area 8010 of theRAM 803. - The
OS 8020 is basic software that controls the entire operation of thetransmission terminal 8 through providing basic functions. The communication application B11 provides thetransmission terminal 8 with a communication control function of sending and receiving video data of a conference or the like to or from another external communication terminal, and an image processing function of outputting the video data as an image. - The
OS 8020 and the communication application B11 are installed onto thetransmission terminal 8 before or after shipment. - <<Functional Configuration of Embodiment>>
- Next, a functional configuration of the
communication system 1 according to the present embodiment is described. -
FIG. 6 is a schematic block diagram illustrating a functional configuration of theelectronic whiteboards 2, thetransmission terminal 8, and themanagement system 50 in thecommunication system 1, according to the present embodiment. - The
electronic whiteboard 2 is provided with the hardware components as illustrated inFIG. 2 which operate in cooperation with the control programs, thereby implementing the functional configuration as illustrated inFIG. 6 . Moreover, theelectronic whiteboard 2 includes amemory 2000 implemented by theSSD 204. Thetransmission terminal 8 is provided with the hardware components as illustrated inFIG. 4 that operate in cooperation with the control programs, thereby implementing the functional configuration as illustrated inFIG. 6 . Moreover, thetransmission terminal 8 includes amemory 8000 implemented by theSSD 805. - The
electronic whiteboard 2 may serve as a “host device” that firstly requests to start communication to share images (remote sharing), or may serve as a “participant device” that later participates in the communication (remote sharing) already started by the host device. In broad outline, theelectronic whiteboard 2 consists of aclient unit 20 and aserver unit 90. Theclient unit 20 and theserver unit 90 are functions implemented by oneelectronic whiteboard 2, with activation of the communication application A11. Alternatively, theelectronic whiteboard 2 may only include theclient unit 20, and a device such as a distribution controller other than theelectronic whiteboard 2 may implement theserver unit 90. - When the
electronic whiteboard 2 serves as a host device, both theclient unit 20 and theserver unit 90 are implemented by theelectronic whiteboard 2. Moreover, when theelectronic whiteboard 2 serves as a participant device, only theclient unit 20 is implemented by theelectronic whiteboard 2, and theserver unit 90 is not implemented. In other words, when theelectronic whiteboard 2 a serves as a host device and theelectronic whiteboards client unit 20 of theelectronic whiteboard 2 a communicates with theclient unit 20 of each of the otherelectronic whiteboards server unit 90 implemented in the sameelectronic whiteboard 2 a. By contrast, theclient unit 20 of each one of theelectronic whiteboards client unit 20 of each of the otherelectronic whiteboard server unit 90 implemented in theelectronic whiteboard 2 a. - <Functional Configuration of
Client Unit 20> - As the communication application A11 is started in the
electronic whiteboard 2, theclient unit 20 starts operating. Theclient unit 20 includes acommunication controller 21 and a drawingcontroller 22. - The
communication controller 21 is implemented by thenetwork interface 205 or the instructions from theCPU 201, and controls the communication performed with anotherelectronic whiteboard 2 or the communication performed with theserver unit 90 or the like of the sameelectronic whiteboard 2. - The drawing
controller 22 is implemented by the instructions from theCPU 201, and performs image processing based on the stroke data generated in response to operation made on thedisplay 3, or based on various kinds of data obtained from any desired device. Examples of such a device include, but not limited to, the USB memory 5, the laptop PC 6, thecommunication controller 21, and thememory 2000. The drawingcontroller 22 generates an image layer based on the processed image data, and controls output of a combined image where a plurality of image layers are superimposed on top of one another. -
FIG. 7 is a functional block diagram of the drawingcontroller 22, according to the present embodiment. - The drawing
controller 22 includes astroke processing unit 22A, asuperimposition unit 22Y, and a page processor 22Z. - The
stroke processing unit 22A may be implemented by the instructions from theCPU 201, and generates stroke data that reflects the user's drawing on thedisplay 3 with theelectronic stylus 4 or the user's hand H. - The
superimposition unit 22Y may be implemented by the instructions from theCPU 201, and generates a plurality of image layers based on each type of data and superimposes those image layers on top of one another. - The page processor 22Z associates a plurality of items of display data such as of stroke data with each other as one page of data, and stores the associated items of display data in a
page data memory 220 of thememory 2000. - The
page data memory 220 is a memory area of thememory 2000, and stores page data as depicted in Table 1 given below. Table 1 is a diagram illustrating an example data structure of the page data. The page data includes one-page data to be displayed on thedisplay 3, which includes stroke arrangement data (items of stroke data for one character, etc.) and medium data. -
TABLE 1 STROKE PAGE ARRANGE- DATA MENT MEDIUM ID START TIME END TIME DATA ID DATA ID p001 20150522152034 20150522152402 st001 m001 p002 20150522152415 20150522152825 st002 m002 p003 20150522153345 20150522154033 st003 m003 . . . . . . . . . . . . . . . - In the page data, page data ID by which any desired one page is identified, the start time at which the display of that page is started, the end time when rewriting of page is no longer performed, for example, by stroke and gesture, stroke arrangement data ID that identifies the stroke arrangement data caused by the stroke of the
electronic stylus 4 or the user's hand H, and medium data ID that identifies the medium data are stored in association with each other. The stroke arrangement data is used to display a set of stroke images on thedisplay 3. The medium data is used to display another image overlapping with the stroke image on thedisplay 3. - The stroke arrangement data includes information in detail as depicted in
FIG. 21 . -
FIG. 21 is a diagram illustrating stroke arrangement data according to the present embodiment. - As illustrated in
FIG. 21 , one stroke arrangement data includes a plurality of items of stroke data. Further, one item of stroke data indicates stroke data ID that identifies that stroke data, the start time at which the writing of one item of stroke starts, the end time at which the writing of one item of stroke ends, the color of the stroke, the width of the stroke, and the coordinate arrangement data ID that identifies the arrangement of way points of the stroke. For example, when the user draws an alphabet “S” with theelectronic stylus 4 with a single stroke, one item of stroke data ID is assigned to the single character of alphabet “S”. However, when the user draws an alphabet “T” with theelectronic stylus 4 with two strokes, two items of stroke data ID are assigned to the single character of alphabet “T”. - Further, the coordinate arrangement data includes information in detail as depicted in
FIG. 22 . -
FIG. 22 is a diagram illustrating an example data structure of the coordinate arrangement data, according to the present embodiment. - The coordinate arrangement data indicates several items of information including a point on the display 3 (X coordinate value, Y coordinate value), the difference in time (millisecond (ms)) when passing through this point since the start time of the stroke, and the writing pressure of the
electronic stylus 4 on that point. In other words, a collection of several points depicted inFIG. 22 is indicated by one item of coordinate arrangement data as depicted inFIG. 21 . For example, when the user draws an alphabet “S” using theelectronic stylus 4 with a single stroke, a plurality of way points are passed through to complete the drawing of “S”. Due to this configuration, the coordinate arrangement data indicates those multiple way points in the stroke drawing. - The medium data of the page data in Table 1 includes more detailed information as depicted in Table 2. Table 2 depicts an example data structure of medium data. In the medium data, the medium data ID included in the page data depicted in Table 1, the type of data of the medium data, the time at which the medium data is stored, the position (X coordinate value, Y coordinate value) of the image to be displayed on the
display 3 according to the medium data, and the size (width, height) of the image, as well as the data indicating the contents of the medium data are associated with each other. Among these items of data, the position of the image to be displayed on thedisplay 3 according to the medium data indicates the upper-left corner of the image that is displayed according to the medium data assuming that the coordinates of upper-left corner of thedisplay 3 is (X coordinate value, Y coordinate value)=(0, 0). -
TABLE 2 MEDIUM TYPE OF RECORDED X COORDINATE Y COORDINATE DATA ID DATA TIME VALUE VALUE WIDTH HEIGHT DATA m001 image 20150522152632 0 0 1920 1080 abc.jpg m002 image 20150522153002 277 156 1366 768 bcd.jpg m003 image 20150522154421 277 156 1366 768 cde.jpg . . . . . . . . . . . . . . . . . . . . . . . . - <Functional Configuration of
Server Unit 90> - The
server unit 90 includes acommunication controller 91. - The
communication controller 91 is implemented by the instructions from theCPU 201, and controls communication, through thecommunication network 9 a, between thecommunication controller 21 in theclient unit 20 of the sameelectronic whiteboard 2 and thecommunication controller 21 in theclient unit 20 of another one of theelectronic whiteboards 2. - <Functional Configuration of
Client Unit 80> - As the communication application B11 is started in the
electronic whiteboard 2 or thetransmission terminal 8, theclient unit 80 starts operating. Theclient unit 80 includes acommunication controller 81, adisplay controller 82, and aninput acceptance unit 83. - The
communication controller 81 may be implemented by the instructions from theCPU 201 or theCPU 801 or thenetwork interface 205 or thenetwork interface 811, and controls communication with another communication terminal or the like. - The
display controller 82 may be implemented by the instructions from theCPU 201 or theCPU 801, and performs image processing based on data obtained from thememory 2000, thememory 8000, or the like, and controls output of processed image data. - The
input acceptance unit 83 may be implemented by the instructions from theCPU 201 or theCPU 801, and receives various kinds of operation or input made by a user. - <Display Order Controller>
- The
electronic whiteboard 2 c further includes adisplay order controller 24. Thedisplay order controller 24 starts operating as theOS 2020 is started in theelectronic whiteboard 2 c. - The
display order controller 24 may be implemented by the instructions from theCPU 201, and activates one of the images generated by theclient units - <Functional Configuration of
Management System 50> - The
management system 50 is provided with the hardware components as illustrated inFIG. 3 that operate in cooperation with control programs, thereby implementing each functional configuration as illustrated inFIG. 6 . Themanagement system 50 also includes amemory 5000 that is implemented by theHD 504 illustrated inFIG. 3 . - The
communication controller 51 of themanagement system 50 may be implemented by the instructions from theCPU 501, and control communication with theelectronic whiteboard 2 or thetransmission terminal 8 through thecommunication network 9 b. - <<Processes>>
- Next, the processes according to the present embodiment are described.
- <Establishment of Communication Among Electronic Whiteboards>
- With reference to
FIG. 8 , the processes of establishing communication among theelectronic whiteboards -
FIG. 8 is a sequence diagram illustrating the processes of establishing communication among a plurality of electronic whiteboards, according to the present embodiment. - Once the power switch 226 of the
electronic whiteboard 2 a is turned on, the communication application A11 starts operating, and theclient unit 20 of theelectronic whiteboard 2 a starts processing (step S1). Then, once a request to hold a conference is accepted by operation made on thedisplay 3 a, thecommunication controller 21 of theclient unit 20 sends instructions to start the processing of theserver unit 90 to thecommunication controller 91 in theserver unit 90 of the sameelectronic whiteboard 2 a. Due to this configuration, not only theclient unit 20 but also theserver unit 90 of theelectronic whiteboard 2 a starts various kinds of processing (step S2). - The
communication controller 91 of theelectronic whiteboard 2 a generates connection information to establish a connection with theelectronic whiteboard 2 a, and controls thedisplay 3 a to display the generated connection information (step S3). Such connection information includes the IP address of the host device and a pass code generated for the present remote sharing, and the pass code is stored in thememory 2000. Moreover, a user of theelectronic whiteboard 2 a notifies the users of theelectronic whiteboards - Next, once the power switch 226 of each of the
electronic whiteboards electronic whiteboards client unit 20 of each of theelectronic whiteboards displays electronic whiteboards communication controller 21 in theclient unit 20 of each of theelectronic whiteboards communication controller 91 in theserver unit 90 of theelectronic whiteboard 2 a, based on the IP address included in the connection information, through thecommunication network 9 a (steps S6 and S7). Thecommunication controller 91 of theelectronic whiteboard 2 a receives the participation requests including the pass code from each of theelectronic whiteboards - Next, the
communication controller 91 determines whether or not to authenticate each one of theelectronic whiteboards electronic whiteboards communication controller 91 notifies each of theclient units 20 of theelectronic whiteboard electronic whiteboards electronic whiteboard 2 a that serves as the host device establishes a communication for remote sharing with theelectronic whiteboard 2 b andelectronic whiteboard 2 c each of which serves as a participant device (steps S11 and S12). In the description of the present embodiment, it is assumed that contents of data exchanged among theelectronic whiteboards - When a user draws a stroke image on one of the
electronic whiteboards electronic stylus stroke processing unit 22A of the corresponding one of theelectronic whiteboards electronic styluses displays communication controller 21 of the relevant one of theelectronic whiteboards communication controller 91 of theelectronic whiteboard 2 that serves as the host device. - As the stroked data is received from the
electronic whiteboards communication controller 91 of theelectronic whiteboard 2 a transmits the received stroke data to theclient units 20 of theelectronic whiteboards communication controllers 21 in theelectronic whiteboards electronic whiteboards page data memory 220 to store the newly received stroke data in an added manner without overwriting the previously stored stroke data (steps S13, S14, and S15). Due to this configuration, the communication among theelectronic whiteboards electronic whiteboards page data memories 220 of theelectronic whiteboards - <Establishment of Communication Between Electronic Whiteboard and Transmission Terminal>
- With reference to
FIG. 9 , the processes of establishing communication between theelectronic whiteboard 2 c and thetransmission terminal 8, using the communication application B11, are described. -
FIG. 9 is a sequence diagram illustrating the processes of establishing communication between theelectronic whiteboard 2 c and thetransmission terminal 8, according to the present embodiment. - As the communication application B11 is started in the
electronic whiteboard 2 c in response to a user's operation, theclient unit 80 starts various kinds of processing (step S21). - The
communication controller 81 of theclient unit 80 sends a login request to the management system 50 (step S22). This login request includes a user ID and a pass code. Accordingly, thecommunication controller 51 of themanagement system 50 receives the login request. - The
management system 50 authenticates theelectronic whiteboard 2 c that has sent the login request (step S23), based on whether a pair of user ID and a pass code included in the login request matches a pair of user ID and a pass code stored in thememory 5000 in advance. Thecommunication controller 51 of themanagement system 50 sends an authentication result to theelectronic whiteboard 2 c that has sent the login request (step S24). Accordingly, thecommunication controller 81 of theelectronic whiteboard 2 c receives the authentication result. In the following description, cases in which theelectronic whiteboard 2 c is determined to be an authorized login request sender are described. - Once a request to start the communication application B11 is accepted by user's operation made on the
transmission terminal 8, the communication application B11 starts operating, and theclient unit 80 of thetransmission terminal 8 starts various kinds of processing (step S31). - The
communication controller 81 of thetransmission terminal 8 sends a login request including the user ID and pass code to the management system 50 (step S32). Then, themanagement system 50 receives this login request. - The
management system 50 authenticates thetransmission terminal 8 that has sent the login request using the user ID and the pass code included in the login request (step S33), and sends the authentication result to the transmission terminal 8 (step S34). In the following description, cases in which thetransmission terminal 8 is determined to be an authorized login request sender are described. - In response to operation made by a user of the
electronic whiteboard 2 c, theinput acceptance unit 83 of theelectronic whiteboard 2 c accepts a request to start a phone conversation with the transmission terminal 8 (step S41). In response to the acceptance of the request to start a phone conversation, thecommunication controller 81 of theelectronic whiteboard 2 c sends a request to start a phone conversation to the management system 50 (step S42). This request to start a phone conversation includes the user ID of theelectronic whiteboard 2 c that is the request sender terminal and the user ID of thetransmission terminal 8 that is the destination terminal. - When the request to start a phone conversation sent from the
electronic whiteboard 2 c is received, thecommunication controller 51 of themanagement system 50 sends the received request to start a phone conversation to thetransmission terminal 8 that is the counterpart terminal (step S43). - The
communication controller 81 of thetransmission terminal 8 receives the request to start a phone conversation sent from themanagement system 50. Once a user of thetransmission terminal 8 operates thetransmission terminal 8 to accept the request to start a phone conversation, thecommunication controller 81 sends acceptance data to the management system 50 (step S44). - Once the
communication controller 51 of themanagement system 50 receives the acceptance data sent from thetransmission terminal 8, thecommunication controller 51 sends the acceptance data to theelectronic whiteboard 2 c that has sent the start request (step S45). Thecommunication controller 81 of theelectronic whiteboard 2 c receives the acceptance data sent from themanagement system 50. - Then, the
management system 50 establishes communication between theelectronic whiteboard 2 c and the transmission terminal 8 (steps S46-1 and S46-2). In these processes, thecommunication controller 51 of themanagement system 50 sends the IP address of theelectronic whiteboard 2 c or thetransmission terminal 8 that serves as the counterpart communication terminal, and the relay device ID of therelay device 30, to thetransmission terminal 8 and theelectronic whiteboard 2 c. Thecommunication controller 81 of each of thetransmission terminal 8 and theelectronic whiteboard 2 c sends the relay device ID and the IP address received from themanagement system 50 to therelay device 30, and establishes a connection with therelay device 30. - The
communication controller 81 of each of thetransmission terminal 8 and theelectronic whiteboard 2 c sends the video data of the video recorded by thecamera 207 or thecamera 812 on the local terminal side and the audio data of the sound recorded by themicrophone 222 or themicrophone 814 on the local communication terminal side to therelay device 30. Therelay device 30 relays the image data and audio data between thetransmission terminal 8 and theelectronic whiteboard 2 c (step S47). Thecommunication controller 81 of each of thetransmission terminal 8 and theelectronic whiteboard 2 c receives the video data and audio data on the counterpart communication terminal side. Note that the above video data and audio data is streaming data and includes a plurality of pieces of image data or audio data that are divided by time. - Once the
communication controller 81 of theelectronic whiteboard 2 c newly receives the image data and audio data, updates the image data and audio data stored in thememory 2000 to the newly received image data and audio data (steps S48 and S49). Due to this configuration, the latest image data and audio data are stored in thememory 2000. - <Generation of Image by Communication Application A11>
- Next, the processes of generating an image based on the stroke data are described with reference to
FIG. 10A . -
FIG. 10A is a flowchart of the processes of generating an image using the communication application A11, according to the present embodiment. - In the following description, the processes of generating an image using the communication application A11 of the
electronic whiteboard 2 c are described. - The image data of a background image is stored in the
page data memory 220. The page processor 22Z of theelectronic whiteboard 2 c obtains the image data of a background image stored on the page data memory 220 (step S51). The background image has a nontransparent area of the resolution (size) of the screen of thedisplay 3, and is, for example, a white and plain image. The background image is used as a layer of the background image that makes up the screen in the later processes as will be described later. - Once the laptop PC 6 is connected to the
electronic whiteboard 2 c, the same image as the one to be output to the display of thelaptop PC 6 c to theelectronic whiteboard 2 c. The page processor 22Z of theelectronic whiteboard 2 c obtains an external image input from thelaptop PC 6 c (step S52). The external image is used as a layer of the external image that makes up the screen in the later processes as will be described later. When no external image is input, the page processor 22Z of theelectronic whiteboard 2 c skips the processes in the step S52. - The image data (medium data) of object images is stored in the
page data memory 220. The object images are, for example, images of a predetermined object such as a circle, rectangular, and a triangle. The page processor 22Z of theelectronic whiteboard 2 c generates a layer of object images where specified ones of the object images stored in thepage data memory 220 are arranged at positions specified by the user's operation (step S53). In these processes, the page processor 22Z superimposes object images on the transparent pixel area of the resolution (size) of the screen of thedisplay 3. - The page processor 22Z of the
electronic whiteboard 2 c generates a layer of stroke images based on the stroke data accumulated in the page data memory 220 (see the step S15) (step S54). In these processes, the page processor 22Z superimposes the stroke images indicated by the stroke data on the transparent pixel area of the resolution (size) of the screen of thedisplay 3. For example, - When stroke data indicating the text of “S” in black is stored in the
page data memory 220, the page processor 22Z generates a layer of stroke images including the text “S” in black thereon. - The
page data memory 220 of theelectronic whiteboard 2 c stores the image data of a user interface (UI) image. Such image data of a UI image is, for example, an icon or key related to the drawing operation. The page processor 22Z of theelectronic whiteboard 2 c generates a layer of UI images based on the image data of a UI image stored in the page data memory 220 (step S55). In these processes, the page processor 22Z superimposes the U images indicated by the image data of the UI images on the transparent pixel area of the resolution (size) of the screen of thedisplay 3. - The
superimposition unit 22Y of theelectronic whiteboard 2 c performs rendering and sequentially superimposes the layer of a background image obtained in the step S51, the layer of an external image obtained in the step S52, the layer of object images generated in the step S53, and the layer of stroke images generated in the step S54 on top of one another to generate a layer of a superimposed image excluding the UI image (step S56). When a layer of an external image is not obtained in the step S52, i.e., when the laptop PC 6 is not connected to theelectronic whiteboard 2 c, thesuperimposition unit 22Y generates the layer of a superimposed image excluding the layer of an external image and the UI image. - Further, the
superimposition unit 22Y of theelectronic whiteboard 2 c performs rendering and sequentially superimposes the layer of a superimposed image, excluding the UI image, generated in the step S56, the layer of a background image obtained in the step S51, the layer of an external image obtained in the step S52, the layer of object images generated in the step S53, the layer of stroke images generated in the step S54, and the layer of UI images generated in the step S55 on top of one another to generate a layer of a superimposed image including the UI images (step S57). When a layer of an external image is not obtained in the step S52, i.e., when the laptop PC 6 is not connected to theelectronic whiteboard 2 c, thesuperimposition unit 22Y generates the layer of a superimposed image excluding the layer of an external image but including the UI image. - The
superimposition unit 22Y of theelectronic whiteboard 2 c outputs a layer of a superimposed image including the UI image to the display order controller 24 (step S58). -
FIG. 11A is a diagram illustrating multiple layers of a superimposed image including a UI image, according to the present embodiment. -
FIG. 11B is a diagram illustrating a superimposed image including a UI image, according to the present embodiment. - As illustrated in
FIG. 11A andFIG. 11B , thesuperimposed image 140 including the UI images includes alayer 140 a of the UI images, alayer 140 b of the stroke images, alayer 140 c of the object images, alayer 140 d of the external image, alayer 140 e of the background image, and a layer 140 f of the superimposed image excluding the UI image. As thelayer 140 e of the background image is a full screen and nontransparent, the layer 140 f of the superimposed image excluding the UI image is invisible on thesuperimposed image 140. - While the communication application A11 is operating, the
client unit 20 of theelectronic whiteboard 2 c repeats the processes in the steps S51 to S58. Due to this configuration, a superimposed image that is updated with the latest background image, external image, object image, stroke image, and the UI image is repeatedly output to thedisplay order controller 24. - <Generation of Image by Communication Application B11>
- Next, the processes of generating an image based on the video data exchanged between the
electronic whiteboard 2 c and thetransmission terminal 8 are described with reference toFIG. 10B . -
FIG. 10B is a flowchart of the processes of generating an image using the communication application B11, according to the present embodiment. - In the following description, the processes of generating an image using the communication application B11 of the
electronic whiteboard 2 c are described. - The
display controller 82 of theelectronic whiteboard 2 obtains the image data of an updated image stored in the memory 2000 (see step S48) (step S61). Note that the image data of an updated image indicates the image data of the latest image in the video data received by theelectronic whiteboard 2 c. - The
display controller 82 of theelectronic whiteboard 2 outputs the image data of an updated image obtained in the step S61 to the display order controller 24 (step S62). - While the communication application B11 is operating, the
client unit 80 of theelectronic whiteboard 2 c repeats the processes in the steps S61 and S62. Accordingly, the latest images that together configure a video are repeatedly output to thedisplay order controller 24. - The
display order controller 24 of theelectronic whiteboard 2 c controls the images output from theclient unit 20 and the images output from theclient unit 80, according to operation made by a user or events caused due to the communication performed by the communication applications A11 or B11, such that the images output from theclient unit 20 are activated and the images output from theclient unit 80 are deactivated or such that the images output from theclient unit 20 are deactivated and the images output from theclient unit 80 are activated. In these processes, thedisplay order controller 24 controls the images output from theclient unit 20 and the images output from theclient unit 80 such that the images output from theclient unit 20 are displayed in the foreground of the screen of thedisplay 3 c or such that the images output from theclient unit 80 are displayed in the foreground of the screen of thedisplay 3 c. Due to this configuration, one of the superimposed image output in the step S58 and the video of a video conference output in the step S62 is actively output from thedisplay 3 c of theelectronic whiteboard 2 c. - Next, the processes of sending the screen image of the
display 3 c of theelectronic whiteboard 2 c to thetransmission terminal 8 are described with reference toFIG. 12 . -
FIG. 12 is a sequence diagram illustrating the processes of sending the screen image on theelectronic whiteboard 2 c side to thetransmission terminal 8, according to the present embodiment. - The
electronic whiteboard 2 c accepts a request to share the screen of thedisplay 3 c with thetransmission terminal 8. Such a request may be made by operation or input by a user of theelectronic whiteboard 2 c, or may be made by thetransmission terminal 8. Once such a request is accepted, thecommunication controller 81 of theelectronic whiteboard 2 c obtains the layer of a superimposed image excluding the UI image, which is generated in the step S56 (step S71). - The
communication controller 81 of theelectronic whiteboard 2 c sends the obtained layer of a superimposed image, excluding the UI image, to therelay device 30 as the display data indicating the screen that is being displayed on theelectronic whiteboard 2 c side (step S72). - Once the display data sent from the
electronic whiteboard 2 c is received, therelay device 30 sends the received display data to the transmission terminal 8 (step S73). When scalable communication is established between therelay device 30 and one of theelectronic whiteboard 2 c and thetransmission terminal 8, the display data that is received from theelectronic whiteboard 2 c may be sent to thetransmission terminal 8 upon its quality being changed according to, for example, the bandwidth of thecommunication network 9 b. - The
communication controller 81 of thetransmission terminal 8 receives the display data sent from therelay device 30. Thedisplay controller 82 of thetransmission terminal 8 sends the video data on theelectronic whiteboard 2 c side sent from therelay device 30 in the step S47 and the display data of the screen that is being displayed on theelectronic whiteboard 2 c sent from therelay device 30 in the step S73 to thedisplay 820 of the transmission terminal 8 (step S74). -
FIG. 13 is a diagram illustrating an example image displayed on thedisplay 820 on thetransmission terminal 8 side in the step S74, according to the present embodiment. - As illustrated in
FIG. 13 , avideo 150 of a conference participant on theelectronic whiteboard 2 c side and the layer 140 f of a superimposed image, excluding the UI image, as a display screen, are displayed on thedisplay 3. - Every time the layer of a superimposed image, excluding the UI image, is generated in the step S56, the
communication controller 81 of theelectronic whiteboard 2 c sends the display data newly obtained in the step S71 to therelay device 30. Accordingly, thetransmission terminal 8 receives the updated display data and outputs the received display data to thedisplay 820. - Next, a second embodiment of the present disclosure is described. In particular, differences in configuration from the embodiment as described above are described.
- In the processes in the step S53, the page processor 22Z of the
electronic whiteboard 2 c superimposes an input external image on the nontransparent pixel area of the resolution (size) of the screen of thedisplay 3 to generate a layer of the external image. Due to these processes, a full screen and nontransparent layer of the external image is generated. - When an external image is input from the
laptop PC 6 c and a full screen and nontransparent layer of the external image is generated in the step S53, in the step S56, thesuperimposition unit 22Y of theelectronic whiteboard 2 c generates a superimposed image, excluding the UI images, where a layer of object images and a layer of stroke images are superimposed on the layer of the external image in that order. As a layer of the external image is a full screen and nontransparent, even if the background image is not included in the images to be superimposed, the same superimposed image as the one generated when the background image is included in the images to be superimposed is generated. Due to this configuration, a layer of the background image is excluded from the images to be superimposed, and the load of processes on thesuperimposition unit 22Y can be reduced. - Next, a third embodiment of the present disclosure is described. In particular, differences in configuration from the embodiments as described above are described.
- The
page data memory 220 of theelectronic whiteboard 2 c stores a functional configuration screen and a videoconference configuration screen. When activated or according to a request made by a user, the page processor 22Z of theelectronic whiteboard 2 c obtains the functional configuration screen stored in thepage data memory 220, as a layer of UI images thesuperimposition unit 22Y outputs a superimposed image that includes the obtained functional configuration screen to thedisplay order controller 24. Thedisplay order controller 24 outputs the image data including the functional configuration screen to thedisplay 3 c. -
FIG. 14A is a diagram illustrating a functional configuration screen, according to the present embodiment. - The functional configuration screen includes a key B1 thereon for shifting to a videoconference configuration screen.
- When a user touches the key B1 being output to the
display 3 c, using theelectronic stylus 4 c or the like, the page processor 22Z of theelectronic whiteboard 2 c obtains the videoconference configuration screen stored in thepage data memory 220. In the step S57, thesuperimposition unit 22Y outputs a superimposed image that includes the obtained videoconference configuration screen to thedisplay order controller 24. Thedisplay order controller 24 outputs the image data including the videoconference configuration screen to thedisplay 3 c. -
FIG. 14B is a diagram illustrating a videoconference configuration screen, according to the present embodiment. - When a user ticks a checkbox for using the video conference function within an input field B2 on the videoconference configuration screen, using the
electronic stylus 4 c or the like, the communication application B11 stores in thememory 2000 the data indicating that themanagement system 50 is to be used for a video conference. - In the processes of the step S56, the
superimposition unit 22Y of theelectronic whiteboard 2 c refers to thememory 2000 and determines whether the data indicating that themanagement system 50 is to be used is stored therein. Here, when it is determined that the data indicating that themanagement system 50 is to be used is stored, thesuperimposition unit 22Y generates a superimposed image excluding the UI images. On the other hand, when it is determined that the data indicating that themanagement system 50 is to be used is not stored, thesuperimposition unit 22Y does not generate a superimposed image excluding the UI images. Due to the determination made as above, when it is assumed that the screen sharing function by the communication application B11 is not to be used, the processes of generating a superimposed image excluding the UI images can be omitted in theelectronic whiteboard 2 c. Accordingly, the load of processes can be reduced. - Alternatively, a communication terminal such as a PC may be connected to the
communication network electronic whiteboard 2 by inputting the IP address of the desiredelectronic whiteboard 2 through a Web browser. Due to this configuration, a videoconference configuration screen of theelectronic whiteboard 2 may be displayed on the communication terminal such as a PC, and the above items become configurable. - Next, a fourth embodiment of the present disclosure is described. In particular, differences in configuration from the embodiments as described above are described.
-
FIG. 15 is a diagram illustrating a schematic configuration of acommunication system 1′, according to a fourth embodiment of the present disclosure. - As illustrated in
FIG. 15 , thecommunication system 1′ further includes adata server 60 and atelephone terminal 7 in addition to the devices and apparatuses in thecommunication system 1. - The
communication system 1′ includes acommunication network 9 c that serves as a mobile phone network. Thecommunication network 9 c is connected to thecommunication network 9 b, i.e., the Internet, through a gateway. At the end of thecommunication network 9 c, a base station is installed. At the end of thecommunication network 9 b, a wireless LAN router that serves as the access point of radio communication is installed. - The
telephone terminal 7 is, for example, a mobile communication device such as a smartphone, and is connected to thecommunication network 9 c through a base station to communicate with anothertelephone terminal 7. Further, thetelephone terminal 7 communicates with thedata server 60 and theelectronic whiteboard 2 by connecting to thecommunication network 9 b through a wireless LAN router or by connecting to thecommunication network 9 b through a base station, thecommunication network 9 c, and a gateway. - The
data server 60 is connected to thecommunication network 9 b. Thedata server 60 stores the data uploaded from any client communication device. Moreover, thedata server 60 accepts a request to download data from any client communication device. -
FIG. 16 is a schematic block diagram illustrating a hardware configuration of atelephone terminal 7, according to the present embodiment. - As illustrated in
FIG. 16 , thetelephone terminal 7 is provided with a central processing unit (CPU) 101 that controls the overall operation of thetelephone terminal 7, a read-only memory (ROM) 102 storing a program, a random access memory (RAM) 103 that is used as a work area in which theCPU 101 executes a program, an electrically erasable and programmable read only memory (EEPROM) 104 that controls reading or writing of various kinds of data under control of theCPU 101, and amedium drive 107 that controls reading or writing of data to or from arecording medium 106 such as a flash memory. Note that therecording medium 106 is removable from themedium drive 107, and data recorded in therecording medium 106 is read therefrom or data is newly written and stored on therecording medium 106 under control of themedium drive 107. Note also that theEEPROM 104 stores an operating system (OS) to be executed by theCPU 101, other various kinds of programs, and various types of data. - Further, the
telephone terminal 7 is provided with anantenna 111 a, and acommunication circuit 111 that uses theantenna 111 a and radio communication signals to communicate with a nearest base station. Further, thetelephone terminal 7 is provided with a complementary metal oxide semiconductor (CMOS)sensor 112 that captures an object under the control of theCPU 101 to obtain captured-image data, asound input 114 that converts the sound into an audio signal, and asound output 115 that converts the audio signal into sound. Note also that theCMOS sensor 112 may be a charge coupled device that converts an image of an object into electronic data through photoelectric conversion. In other words, as long as it can capture an object, the CMOS sensor may be replaced with a charge coupled device (CCD) sensor. Moreover, thesound input 114 includes a microphone for receiving sound, and thesound output 115 includes a loudspeaker to output sound. Further, thetelephone terminal 7 is provided with adisplay 120 such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display that display an image of an object, various kinds of icons, or the like, and atouch panel 121 that is made of a pressure-sensing or electrostatic panel and disposed on thedisplay 120. Thetouch panel 121 detects a touch point on thedisplay 120 by a finger, an electronic stylus, or the like. Further, thetelephone terminal 7 is provided with abus line 110 such as an address bus or a data bus that electrically connects the elements as illustrated inFIG. 16 . - The hardware configuration of the
data server 60 is equivalent to the hardware configuration of themanagement system 50, and thus its description is omitted. -
FIG. 17A is a functional block diagram of theelectronic whiteboard 2 c, according to the fourth embodiment. - In addition to the functional configuration according to the embodiments described above, the
client unit 20 of theelectronic whiteboard 2 c according to the fourth embodiment includes aninput acceptance unit 23 and acode processor 25. - The
input acceptance unit 23 receives various kinds of operation or input made by a user according to the instructions from theCPU 201, or operation of thesensor controller 214 and thecontact sensor 215. - The
code processor 25 generate a bar code used to download data according to operation or input made by a user. Such a bar code used to download data will be described later in detail. -
FIG. 17B is a functional block diagram of thedata server 60, according to the fourth embodiment. - The
data server 60 includes acommunication controller 61 and adata processor 69. Moreover, thedata server 60 includes amemory 6000 that is configured by theHD 504 illustrated inFIG. 3 . - The
memory 6000 stores various kinds of data uploaded from any client communication device. Such various kinds of data includes, for example, image data such as of Joint Photographic Experts Group (JPEG), tagged image file fbrmat (TIFF), portable network graphics (PNG), or bitmap (BMP), video data such as of Moving Picture Experts Group (MPEG)-4, a computer aided design (CAD) file, a text, or a file used by a word processing program, a spreadsheet program, a presentation program, or a viewer. - The
communication controller 61 may be implemented by the instructions from theCPU 201 and thenetwork interface 205, and controls communication performed with theelectronic whiteboard 2 or thetelephone terminal 7. - The
data processor 69 may be implemented by the instructions from theCPU 201 and theHDD 505, and controls thememory 6000 to store various kinds of data or reads various kinds of data from thememory 6000. -
FIG. 17C is a functional block diagram of thetelephone terminal 7, according to the fourth embodiment. - The
telephone terminal 7 includes a communication controller 71, animager 72, ananalyzer 73, and a data processor 79. Moreover, thetelephone terminal 7 includes theROM 102, theRAM 103, theEEPROM 104, or thememory 7000 implemented by therecording medium 108. - The
communication controller 61 may be implemented by the instructions from theCPU 101 and the communication circuit 11 l, and controls communication performed with theelectronic whiteboard 2 or thedata server 60. - The
imager 72 may be implemented by theCMOS sensor 112, and captures any desired object such as a bar code displayed on thedisplay 3 c of theelectronic whiteboard 2 c. - The
analyzer 73 may be implemented by the instructions from theCPU 101, and analyzes a bar code captured by theimager 72 to extracts code data from the bar code. Such code data extracted from a bar code will be described later. -
FIG. 18 is a sequence diagram illustrating the processes of downloading the data of a background image by theelectronic whiteboard 2 c, according to the fourth embodiment of the present disclosure. - With reference to
FIG. 18 , how theelectronic whiteboard 2 c downloads the data of a background image is described. - In response to operation or input made by a user of the
electronic whiteboard 2 c to download the background image of thedisplay 3 c, theinput acceptance unit 23 of theelectronic whiteboard 2 c accepts a request to download a background image (step S101). Hereinafter, a request to download a background image will be referred to simply as a downloading request. - Once input of a downloading request is accepted by the
electronic whiteboard 2 c, thecode processor 25 generates a bar code to be used to download a background image (step S102). In these processes, thecode processor 25 generates a bar code that indicates, based on predetermined standards, code data such as identification information indicating a downloading request, identification information of a background image as the data to be downloaded, the destination address of the background image, and a password. Such predetermined standards includes, for example, Japanese Article Number (JAN), European Article Number (EAN), Universal Product Code (UPC), Interleaved Two of Five (ITF), CODE 39, CODE 128, and NW-7. The identification information of a background image is, for example, a file name, and the destination address to which the image data is to be sent is, for example, a uniform resource locator (URL) that indicates theelectronic whiteboard 2 c Moreover, the password is, for example, prescribed in advance by theelectronic whiteboard 2 c. A part of or the entirety of the code data may be input by a user of theelectronic whiteboard 2 c. - In the step S55, the page processor 22Z of the
electronic whiteboard 2 c generates a layer of UI images including the UI images stored in thepage data memory 220 and the bar code generated in the step S102. In the step S58, thesuperimposition unit 22Y of theelectronic whiteboard 2 c outputs a superimposed image including the layer of UI images and the bar code to thedisplay order controller 24. Thedisplay order controller 24 controls thedisplay 3 c to display the superimposed image including the layer of UI images and the bar code (step S103). -
FIG. 19 is a diagram illustrating an example image on the display in the step S103, according to the present embodiment. - The
imager 72 of thetelephone terminal 7 captures a bar code displayed on thedisplay 3 c of theelectronic whiteboard 2 c according to operation made by a user (step S104) Theanalyzer 73 of thetelephone terminal 7 analyzes the captured bar code. More specifically, in accordance with the prescribed rules, theanalyzer 73 extracts, from the bar code included in the captured image, code data such as identification information indicating a downloading request, identification information of a background image, the destination address of the background image, and a password (step S105). Note also that the password out of the extracted code data is used later in the processes of authentication. For this reason, the password is displayed on thedisplay 120 in accordance with the instructions given from theCPU 101. In an alternative embodiment of the present disclosure, a password is not included in the code data, and thus such a password does not need to be displayed on theelectronic whiteboard 2 c. In such a configuration, a password may be input, for example, by a user of thetelephone terminal 7, or may be stored in thememory 7000 of thetelephone terminal 7 as setting data of an application. The communication controller 71 of thetelephone terminal 7 sends the extracted code data to the data server 60 (step S106). In an alternative embodiment of the present disclosure, the password input by a user or the password stored in thememory 7000 is sent to thedata server 60. - The
communication controller 61 of thedata server 60 receives the code data sent from thetelephone terminal 7. Thedata processor 69 of thedata server 60 obtains the data of a background image identified by the identification information included in the code data from the memory 6000 (step S107). Thecommunication controller 61 of thedata server 60 sends the obtained data of the background image and the password included in the code data to theelectronic whiteboard 2 c that is identified by the destination address included in the code data (step S108). - The
communication controller 21 of theelectronic whiteboard 2 c receives the data of the background image sent from thedata server 60. In other words, theelectronic whiteboard 2 c downloads the data of a background image from thedata server 60. Once theelectronic whiteboard 2 c downloads the data of a background image, theinput acceptance unit 23 accepts the input of a password. Thecode processor 25 of theelectronic whiteboard 2 c determines whether the password received by theinput acceptance unit 23 matches the password included in the code data generated in the step S102. When these passwords match one another, thecode processor 25 authenticates the user (step S109). Once the user is authenticated, the page processor 22Z of theelectronic whiteboard 2 c stores the downloaded data of a background image in the page data memory 220 (step S110). - After the downloaded data of a background image is stored on the
page data memory 220, the page processor 22Z of theelectronic whiteboard 2 c obtains the downloaded data of a background image from thepage data memory 220 in the step S51. Once the page processor 22Z obtains the downloaded data of a background image, in the steps S57 and S58, thesuperimposition unit 22Y of theelectronic whiteboard 2 c performs rendering and generates a superimposed image including the downloaded data of a background image, and outputs the generated superimposed image to thedisplay order controller 24. Once thesuperimposition unit 22Y outputs the superimposed image including the downloaded data of a background image, thedisplay order controller 24 of theelectronic whiteboard 2 c controls thedisplay 3 c to display a superimposed image including the downloaded data of a background image (step S111). - Once the page processor 22Z obtains the downloaded data of a background image, in the step S56, the
superimposition unit 22Y of theelectronic whiteboard 2 c performs rendering and generates a superimposed image including the downloaded data of a background image but excluding the UI images. Once thesuperimposition unit 22Y generates the superimposed image including the downloaded data of a background image but excluding the UI images, in the step S71, thecommunication controller 81 of theelectronic whiteboard 2 c obtains the superimposed image including the downloaded data of a background image but excluding the UI images. - In the step S72, the
communication controller 81 of theelectronic whiteboard 2 c sends the superimposed image including the downloaded data of a background image but excluding the UI images to therelay device 30 as display data. - Subsequently, in a similar manner to the processes in the steps S73 and S74 of the embodiments as described above, the
relay device 30 relays the display data sent from theelectronic whiteboard 2 c to thetransmission terminal 8, and thetransmission terminal 8 receives the display data sent from therelay device 30 and controls thedisplay 820 to display an image. Note that as the display data does not include the UI images, the bar code that is displayed on theelectronic whiteboard 2 c side as a UI image is not displayed on thetransmission terminal 8 side. - Next, a fifth embodiment of the present disclosure is described. In particular, differences in configuration from the fourth embodiment as described above are described.
- In the fifth embodiment, the
data server 60 in thecommunication system 1′ according to the fourth embodiment may be omitted. In the fifth embodiment, thememory 7000 of thetelephone terminal 7 stores various kinds of data uploaded from any client communication device. Such various kinds of data is equivalent to the data that thedata server 60 according to the fourth embodiment stores in thememory 6000. -
FIG. 20 is a sequence diagram illustrating the processes of downloading the data of a background image by theelectronic whiteboard 2 c, according to the fifth embodiment of the present disclosure. - With reference to
FIG. 20 , differences in how theelectronic whiteboard 2 c downloads the data of a background image compared with that of the fourth embodiment is described. - Once the code data is extracted by the
analyzer 73 of thetelephone terminal 7, the data processor 79 obtains the data of a background image identified by the identification information of the background image included in the code data from the memory 7000 (step S107′) The communication controller 71 of thetelephone terminal 7 sends the data of the background image and the password included in the code data to theelectronic whiteboard 2 c that is identified by the destination address included in the code data (step S108′). - The
communication controller 21 of theelectronic whiteboard 2 c receives the data of the background image sent from thetelephone terminal 7. In other words, theelectronic whiteboard 2 c downloads the data of a background image from thetelephone terminal 7. The following processes are equivalent to those of the fourth embodiment, and thus their description is omitted. - With the data transmission method according to the embodiments described above, the
stroke processing unit 22A of theelectronic whiteboard 2 c (an example of a communication terminal) accepts a drawing operation. the display order controller 24 (an example of a display controller) of theelectronic whiteboard 2 c controls thedisplay 3 c (an example of a display) to display on the screen of thedisplay 3 c (an example of display controlling processes) an image drawn by operation accepted by thestroke processing unit 22A and a first superimposed image where the UI images related to drawing operation are superimposed on the background image. The communication controller 81 (an example of a transmitter) of theelectronic whiteboard 2 c sends the display data (an example of image data) of a second superimposed image where the UI images are excluded from the images to be superimposed and the drawn images are superimposed on the background image to the transmission terminal 8 (an example of an external communication terminal) (an example of transmitting processes) as the image data to be displayed on the screen of the display. Due to this configuration, when theelectronic whiteboard 2 c capable of displaying the drawn images transmits its screen image to thetransmission terminal 8, the UI images that cannot be manipulated via thetransmission terminal 8 is no longer output to thetransmission terminal 8, and reduction in viewability can be prevented. Moreover, as theelectronic whiteboard 2 transmits an image where the drawn images are superimposed on the background image to thetransmission terminal 8, even if the background color set on thetransmission terminal 8 side is the same as the color of a drawn image, a user of thetransmission terminal 8 can read the drawn image. - The
superimposition unit 22Y (i.e., an example of a generator) of theelectronic whiteboard 2 c performs rendering and generates a layer of a second superimposed image (i.e., an example of a composite image) where a layer of drawn images are superimposed on a layer of the background image. Note that this generation of a layer of a second superimposed image is an example of a generating process. Thecommunication controller 81 of theelectronic whiteboard 2 c sends the display data of the second superimposed image generated by thesuperimposition unit 22Y to thetransmission terminal 8. Due to this configuration, thecommunication controller 81 can obtain the image of a screen generated by thesuperimposition unit 22Y and can send the obtained image to thetransmission terminal 8. - The
display order controller 24 of theelectronic whiteboard 2 c controls thedisplay 3 c to display a layer of a first superimposed image where a layer of the background image, a layer of drawn images, and a layer of UI images are superimposed on a layer of a second superimposed image in that order. In such a configuration, the layer of the second superimposed image is arranged underneath the background image. Accordingly, theelectronic whiteboard 2 c can make the layer of the second superimposed image invisible. - The page data memory 220 (an example of a manager) of the
electronic whiteboard 2 c manages the image data of the background image (an example of a first background image). The page processor 22Z (an example of an input unit) of theelectronic whiteboard 2 c inputs an external image (an example of a second background image) from the laptop PC 6. When the second background image is not input by the page processor 22Z, thesuperimposition unit 22Y of theelectronic whiteboard 2 c combines a layer of the background image and a layer of drawn images to generate a layer of a second superimposed image. On the other hand, when the second background image is input by the page processor 22Z, thesuperimposition unit 22Y of theelectronic whiteboard 2 c combines a layer of the external image and a layer of drawn images to generate a layer of a second superimposed image. Due to this configuration, the load of processes on theelectronic whiteboard 2 c in generating a layer of a second superimposed image can be reduced. - When the use of the
management system 50 is not configured in theelectronic whiteboard 2 c, thesuperimposition unit 22Y of theelectronic whiteboard 2 c does not generate a second superimposed image. Due to this configuration, when the transmission of the display data to thetransmission terminal 8 is not expected, the generation of the second superimposed image is omitted, and the load of processes on theelectronic whiteboard 2 c can be reduced. - A communication terminal such as a smartphone may be connected to the
communication network 9 a, and may transfer the file of a background image stored in the local communication terminal to any one of theelectronic whiteboards 2, using, for example, a web application programming interface (API). Theelectronic whiteboard 2 receives the transferred file, and displays the background image in the received file as the background image of the screen. A file that is stored in a communication terminal may be a file that is generated by the scanning function of an image forming apparatus such as a multifunction peripheral (MFP) and then transferred from the image forming apparatus to the communication terminal. In such a configuration, the image forming apparatus may be connected to thecommunication network 9 a, and may directly transfer a file generated by scanning to the communication terminal, using an email sending function of the image forming apparatus. Alternatively, the image forming apparatus may transfer a file generated by scanning to a file server connected to thecommunication network 9 a, using a remote folder storing function of the image forming apparatus, and such transferred files may be accumulated in the file server. In this configuration, the communication terminal can store a desired file in the communication terminal by obtaining such a desired file from the files accumulated in the file server. - The communication applications A11 and B11 may be recorded on a computer-readable recording medium for distribution in an installable or executable file format. Examples of such a recording medium include, but not limited to, a compact disc-recordable (CD-R), a digital versatile disc (DVD), and a Blu-ray disc. Moreover, a memory storing any one of the above-described control programs, such as a recording medium including a CD-ROM or a HDD, may be provided in the form of a program product to users within a certain country or outside that country.
- Any one of the
electronic whiteboard 2, thetransmission terminal 8, therelay device 30, and themanagement system 50 according to the embodiments described above may be implemented by a single apparatus or a plurality of apparatuses to which divided elements (functions or units) are allocated. - Each of the functions of the described embodiments may be implemented by one or more processing circuits or circuitry. Processing circuitry includes a programmed processor, as a processor includes circuitry. A processing circuit also includes devices such as an application specific integrated circuit (ASIC), digital signal processor (DSP), field programmable gate array (FPGA), and conventional circuit components arranged to perform the recited functions. The term “processing circuit” herein includes, for example, a device such as a processor that may be provided with an electronic circuit and is programmed to execute software to implement functions, an application specific integrated circuit (ASIC) that is designed to execute the above functions, and a conventional circuit module.
- Numerous additional modifications and variations are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims, the disclosure of the present invention may be practiced otherwise than as specifically described herein. For example, elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of this disclosure and appended claims.
- Further, as described above, any one of the above-described and other methods of the present invention may be embodied in the form of a computer program stored on any kind of storage medium. Examples of storage media include, but are not limited to, flexible disks, hard disks, optical discs, magneto-optical discs, magnetic tape, nonvolatile memory cards, ROM, etc. Alternatively, any one of the above-described and other methods of the present invention may be implemented by ASICs, prepared by interconnecting an appropriate network of conventional component circuits, or by a combination thereof with one or more conventional general-purpose microprocessors and/or signal processors programmed accordingly.
Claims (9)
1. A communication terminal comprising circuitry configured to control a display to display a first superimposed image where a drawn image and a user interface image related to drawing operation are superimposed on a background image, and
transmit image data of a second superimposed image, where the drawn image is superimposed on the background image and the user interface image is excluded from images to be superimposed, to an external communication terminal as image data to be displayed on the display.
2. The communication terminal according to claim 1 , wherein
the circuitry performs rendering to generate a composite image of the background image and the drawn image, and
the circuitry transmits the composite image to the external communication terminal as the image data.
3. The communication terminal according to claim 2 , wherein the circuitry controls the display to display the first superimposed image where the background image, the drawn image, and the user interface image are superimposed on the composite image in that order.
4. The communication terminal according to claim 2 , further comprising
a memory to store a first background image,
wherein the circuitry obtains a second background image from an external device, and
wherein
when the second background image is not obtained by the circuitry, the circuitry generates a composite image of the first background image and the drawn image, as the composite image of the background image and the drawn image, and
when the second background image is obtained by the circuitry, the circuitry generates a composite image of the second background image and the drawn image, as the composite image of the background image and the drawn image.
5. The communication terminal according to claim 2 , wherein when the circuitry determines that use of a management system that manages communication with the external communication terminal is not configured in the communication terminal, the circuitry does not generate the composite image.
6. A communication system comprising:
the communication terminal according to claim 1 ; and
the external communication terminal.
7. A communication system comprising:
the communication terminal according to claim 5 ;
the external communication terminal; and
the management system.
8. A method of transmitting data, the method comprising:
controlling a display to display a first superimposed image where a drawn image and a user interface image related to drawing operation are superimposed on a background image; and
transmitting image data of a second superimposed image, where the drawn image is superimposed on the background image and the user interface image is excluded from images to be superimposed, to an external communication terminal as image data to be displayed on the display.
9. A computer-readable non-transitory recording medium storing a program for causing a computer to execute a method, the method comprising:
controlling a display to display a first superimposed image where a drawn image and a user interface image related to drawing operation are superimposed on a background image; and
transmitting image data of a second superimposed image, where the drawn image is superimposed on the background image and the user interface image is excluded from images to be superimposed, to an external communication terminal as image data to be displayed on the display.
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2016193251 | 2016-09-30 | ||
JP2016-193251 | 2016-09-30 | ||
JP2017099616A JP2018060513A (en) | 2016-09-30 | 2017-05-19 | Communication terminal, communication system, transmission method, and program |
JP2017-099616 | 2017-05-19 |
Publications (1)
Publication Number | Publication Date |
---|---|
US20180095711A1 true US20180095711A1 (en) | 2018-04-05 |
Family
ID=61758782
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/663,943 Abandoned US20180095711A1 (en) | 2016-09-30 | 2017-07-31 | Communication terminal, communication system, transmission method, and recording medium storing program |
Country Status (2)
Country | Link |
---|---|
US (1) | US20180095711A1 (en) |
JP (1) | JP7255728B2 (en) |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170300285A1 (en) * | 2016-04-13 | 2017-10-19 | Seiko Epson Corporation | Display system, display device, and method of controlling display system |
CN110070585A (en) * | 2019-01-31 | 2019-07-30 | 北京字节跳动网络技术有限公司 | Image generating method, device and computer readable storage medium |
US10528317B2 (en) | 2017-07-28 | 2020-01-07 | Ricoh Company, Ltd. | Communication system, communication method, and electronic apparatus |
US10867585B2 (en) * | 2017-05-12 | 2020-12-15 | Samsung Electronics Co., Ltd. | Electronic apparatus and method for displaying a content screen on the electronic apparatus thereof |
US11061641B2 (en) | 2019-02-28 | 2021-07-13 | Ricoh Company, Ltd. | Screen sharing system, and information processing apparatus |
US11061639B2 (en) * | 2018-03-14 | 2021-07-13 | Ricoh Company, Ltd. | Electronic whiteboard system, electronic whiteboard, and method of displaying content data |
CN114025217A (en) * | 2020-07-15 | 2022-02-08 | 华为技术有限公司 | Image display method, equipment and storage medium |
US20220083306A1 (en) * | 2020-09-16 | 2022-03-17 | Ricoh Company, Ltd. | Information processing device, non-transitory recording medium, and information processing system |
US11418628B2 (en) * | 2020-03-02 | 2022-08-16 | Ricoh Company, Ltd. | Communication terminal, communication system, and communication method |
US11778137B2 (en) | 2021-07-26 | 2023-10-03 | Ricoh Company, Ltd. | Information processing device, information processing method, and non-transitory recording medium |
US11799925B2 (en) | 2020-09-16 | 2023-10-24 | Ricoh Company, Ltd. | Communication system, communication terminal, and screen sharing method |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20070121146A1 (en) * | 2005-11-28 | 2007-05-31 | Steve Nesbit | Image processing system |
US20120271915A1 (en) * | 2005-01-05 | 2012-10-25 | Microsoft Corporation | Processing files from a mobile device |
US20160098172A1 (en) * | 2014-10-03 | 2016-04-07 | Radim BACINSCHI | User-driven evolving user interfaces |
US20170078877A1 (en) * | 2015-09-11 | 2017-03-16 | Lenovo Enterprise Solutions (Singapore) Pte. Ltd. | Peer-to-peer wireless connection between server computing device and mobile client computing device for authentication and server management |
US20170278549A1 (en) * | 2016-03-24 | 2017-09-28 | Fujitsu Limited | Drawing processing device and method |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6343897B2 (en) * | 2013-09-30 | 2018-06-20 | 株式会社リコー | Transmission terminal, transmission method, and program |
JP6260201B2 (en) * | 2013-10-31 | 2018-01-17 | 株式会社リコー | Transmission terminal, transmission method, and program |
JP2016143182A (en) | 2015-01-30 | 2016-08-08 | 株式会社リコー | Information processing apparatus and program |
-
2017
- 2017-07-31 US US15/663,943 patent/US20180095711A1/en not_active Abandoned
-
2022
- 2022-05-02 JP JP2022075769A patent/JP7255728B2/en active Active
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120271915A1 (en) * | 2005-01-05 | 2012-10-25 | Microsoft Corporation | Processing files from a mobile device |
US20070121146A1 (en) * | 2005-11-28 | 2007-05-31 | Steve Nesbit | Image processing system |
US20160098172A1 (en) * | 2014-10-03 | 2016-04-07 | Radim BACINSCHI | User-driven evolving user interfaces |
US20170078877A1 (en) * | 2015-09-11 | 2017-03-16 | Lenovo Enterprise Solutions (Singapore) Pte. Ltd. | Peer-to-peer wireless connection between server computing device and mobile client computing device for authentication and server management |
US20170278549A1 (en) * | 2016-03-24 | 2017-09-28 | Fujitsu Limited | Drawing processing device and method |
Cited By (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20170300285A1 (en) * | 2016-04-13 | 2017-10-19 | Seiko Epson Corporation | Display system, display device, and method of controlling display system |
US10496356B2 (en) * | 2016-04-13 | 2019-12-03 | Seiko Epson Corporation | Display system, display device, and method of controlling display system |
US10867585B2 (en) * | 2017-05-12 | 2020-12-15 | Samsung Electronics Co., Ltd. | Electronic apparatus and method for displaying a content screen on the electronic apparatus thereof |
US10528317B2 (en) | 2017-07-28 | 2020-01-07 | Ricoh Company, Ltd. | Communication system, communication method, and electronic apparatus |
US11061639B2 (en) * | 2018-03-14 | 2021-07-13 | Ricoh Company, Ltd. | Electronic whiteboard system, electronic whiteboard, and method of displaying content data |
CN110070585A (en) * | 2019-01-31 | 2019-07-30 | 北京字节跳动网络技术有限公司 | Image generating method, device and computer readable storage medium |
US11061641B2 (en) | 2019-02-28 | 2021-07-13 | Ricoh Company, Ltd. | Screen sharing system, and information processing apparatus |
US11418628B2 (en) * | 2020-03-02 | 2022-08-16 | Ricoh Company, Ltd. | Communication terminal, communication system, and communication method |
CN114025217A (en) * | 2020-07-15 | 2022-02-08 | 华为技术有限公司 | Image display method, equipment and storage medium |
US20220083306A1 (en) * | 2020-09-16 | 2022-03-17 | Ricoh Company, Ltd. | Information processing device, non-transitory recording medium, and information processing system |
US11656834B2 (en) * | 2020-09-16 | 2023-05-23 | Ricoh Company, Ltd. | Information processing device, non-transitory recording medium, and information processing system |
US11799925B2 (en) | 2020-09-16 | 2023-10-24 | Ricoh Company, Ltd. | Communication system, communication terminal, and screen sharing method |
US11778137B2 (en) | 2021-07-26 | 2023-10-03 | Ricoh Company, Ltd. | Information processing device, information processing method, and non-transitory recording medium |
Also Published As
Publication number | Publication date |
---|---|
JP7255728B2 (en) | 2023-04-11 |
JP2022107608A (en) | 2022-07-22 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20180095711A1 (en) | Communication terminal, communication system, transmission method, and recording medium storing program | |
US10742932B2 (en) | Communication terminal, communication system, moving-image outputting method, and recording medium storing program | |
US9959084B2 (en) | Communication terminal, communication system, communication control method, and recording medium | |
US11245814B2 (en) | Shared terminal transmits print data with name of the shared terminal as a print requester to printer when the terminal device identification is not received | |
US10831435B2 (en) | Shared terminal, communication system, image transmission method, and recording medium | |
US10297058B2 (en) | Apparatus, system, and method of controlling display of image, and recording medium for changing an order or image layers based on detected user activity | |
US10628117B2 (en) | Communication terminal, communication system, display control method, and recording medium | |
US10637895B2 (en) | Communication terminal, communication system, communication control method and program | |
US10915289B2 (en) | Shared terminal and image transmission method | |
US20150249696A1 (en) | Transmission terminal, transmission system, transmission method, and recording medium storing transmission control program | |
US20170249764A1 (en) | Communication terminal, communication system, and communication control method | |
US10848483B2 (en) | Shared terminal, communication system, and display control method, and recording medium | |
US20180309742A1 (en) | Shared terminal, communication method, and non-transitory computer-readable medium | |
US20150249695A1 (en) | Transmission terminal, transmission system, transmission method, and recording medium storing transmission control program | |
US10637852B2 (en) | Shared terminal and display control method | |
WO2017138233A1 (en) | Image processing device, image processing system and image processing method | |
US11163866B2 (en) | Shared terminal, display control method, and non-transitory computer-readable medium | |
US20170034698A1 (en) | Terminal, communication method, and recording medium storing program | |
US10356361B2 (en) | Communication terminal, communication system, and display method | |
JP2018060513A (en) | Communication terminal, communication system, transmission method, and program | |
US10298878B2 (en) | Terminal, communication method, and recording medium storing program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: RICOH COMPANY, LTD., JAPAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:KANDA, TOMOKI;NAKAMURA, SHIGERU;KATO, YOSHINAGA;AND OTHERS;SIGNING DATES FROM 20170724 TO 20170725;REEL/FRAME:043379/0180 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |