JP2008187716A - Conference task assisting method, conference task assisting system, user interface device, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a system etc., which assists a specified task of, for example, opening a conference by adjusting operations of a hardware device and a software application. <P>SOLUTION: The system includes one or more computers connected via a network, at least one setting storage unit, at least one room control unit, and one or more devices and applications provided by room control units. A presenter for a conference can make environmental settings of the conference or may use the default settings. Settings of devices and applications of one or more presenters may be included so that the conference can be applied to the plurality of presenters. Settings of the conference are stored in the setting storage unit. Settings of the individual presenters include subsets of those one or more devices and applications. Operations of the devices and applications for the conference are adjusted by the room control unit, based on the settings of the presenters for the conference. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to coordination of a plurality of devices based on user configurations of conference rooms. That is, the present invention relates to a conference task support method, a conference task support system, a user interface device, and a program that support specific tasks such as holding a conference by adjusting the operation of hardware devices and software applications of a computer system. .

  Environments where technology is abundant, such as conference rooms, are often difficult to use. This is because various components (parts) in such an environment do not interoperate well. Alternatively, the various components in the environment often do not recognize each other and need to be controlled separately. For ordinary users, it is difficult to perform special tasks, such as coordinating the use of such devices and hosting teleconferences.

  Ubiquitous computing (abbreviated as “ubicomp”) is one method for providing an environment in which technologies such as conference rooms are used abundantly. Ubicomp does not have a computer as a clear object, but incorporates computation into the environment. Other terms for Ubicomp include popular computing, Calm technology, “things that think”, and everywhere. Ubicomp embeds computing processing in the environment and everyday things, making people more natural and casual compared to today, and regardless of where or where they are placed. The purpose is to be able to interact with.

  However, in a sense, ubicomp is an oxymoron. In particular, in a “smart” meeting room (smart room: computer controlled room), the application of ubicomp technology has generally been far from user friendly. Recent work on high-end room systems has often been interactive, with a variety of thin and bright display screens, both large and small, and the rich media handling capabilities used with them. This is related to whiteboards, robot cameras, and remote conference systems.

  Rich media is information that consists of some combination of graphics, audio, video, animation, etc. that uses more storage capacity and wider bandwidth than normal text. However, using all these technologies in one room is a daunting task. With more than two display screens in front, many presenters simply choose to play the same image on all these screens.

  To make matters worse, which room functions the device performs are essential to the tasks that different users want to perform, which room functions are essential to a particular room, and which room functions fit a particular culture. It is a design problem of how to select whether or not the user is present. Taking a specific room as an example, the video conferencing room function is often essential to the design of a small conference room rather than a large conference room design.

  For example, in terms of culture, the design differs between American conference rooms and Japanese conference rooms. In Japan, business meetings are generally scripted and proceed closely on the agenda. These meetings are followed by a brainstorming session. However, in the United States, a business meeting is usually a brainstorming session. In the design of conference rooms in Japan, “Power Point” slides are emphasized, while in the design of conference rooms in the United States, an interactive whiteboard is emphasized.

  There is another problem of maintenance. Almost all smart rooms have resident staff to maintain the functionality of the room system. In addition, staff often have to refrain from daily use so that the user can use the room. Such smart room systems are designed with or without the existence of human "wizards". Such systems are rarely designed with the user's actions in mind. In addition, the user does not know what to expect from the smart room. This is because there is no technical standard for next-generation conference rooms.

  In general, it is beneficial to improve the design of the conference room system. It is particularly beneficial to implement such improvements in a smart room environment.

5 people from Tom Hope, “Community Formation: Co-construction of Meaning and Use with Interactive Information Kiosks” [online], [2007 December 20 search], Internet <URL: http://ymatsuo.com/papers/ubicomp2006hope.pdf>

  The present invention provides a conference task support method, a conference task support system, a user interface device, and a program for supporting a specific task such as holding a conference by adjusting the operation of a hardware device and a software application. For the purpose.

  The system of the present invention includes one or more computers connected by a network, at least one configuration storage component, at least one room control component, and each room control unit. Includes one or more devices and applications to be equipped. The conference presenter can set the conference environment, or may use default settings. The conference includes one or more presenter devices and application settings to simultaneously accommodate multiple presenters. The conference settings are stored in the setting storage unit. Individual presenter settings include a subset of these one or more devices and applications. The device and application operations in the conference are adjusted by the room controller based on the presenter settings for the conference.

  In order to achieve the above object, the invention according to claim 1 is a conference using a computer system that coordinates the operation of hardware devices and software applications to support a specific task such as holding a conference. A task support method, wherein the method provides one or more computers connected to a network, and provides at least one setting storage unit provided in any of the one or more computers. Providing at least one room controller provided in any one of the one or more computers, providing one or more devices and applications corresponding to each room controller, and providing one or more persons; Allows multiple presenters to hold meetings and each meeting's representation is to support multiple presenters Including configuration of devices and applications by more than one presenter, the conference settings are stored by the configuration storage, and each presenter configuration includes a subset of the one or more devices and applications And adjusting the operation of the device and application in the conference by the room control unit based on settings of the one or more presenters for the conference.

  Further, in the first aspect of the present invention, it may further include adding an additional device and application to the system while the system is running.

  According to a second aspect of the present invention, in the first aspect of the present invention, in order to adjust the operation of the device and the application, the adjustment by the room controller is performed autonomously, and the presenter One of performing the adjustment based on settings made via the interface.

  In the invention described in claim 1, it may further include providing at least one display device for each room control unit.

  According to a third aspect of the present invention, in the first aspect of the present invention, in order to adjust the operation of the device and the application, the presenter adjusts when presenting at the same time, when presenting continuously. And adjustment in the case of presenting a combination of simultaneous and continuous presentations.

  The invention of claim 4 is the invention of claim 1, wherein both allowing one or more presenters to set up a conference and coordinating device and application operations are Including first authenticating one of the one or more presenters with an authentication device.

  According to a fifth aspect of the present invention, in the first aspect of the present invention, the conference information including conference, room, presenter, setting, device, device status, and attribute information is stored in one or more databases. It is further characterized by including.

  Further, in the invention described in claim 5, it may further include storing device-specific information in the one or more databases.

  The invention according to claim 6 is the invention according to claim 5, wherein the one or more databases include one of existing commercially available databases, custom databases, and open source software, The application includes existing commercial software, and the device includes one or more of a projector, an electronic whiteboard, a video switch, an audio switch, a video conferencing device, lighting, an audio device, and an air conditioner. It is characterized by.

  The invention described in claim 7 is characterized in that in the invention described in claim 5, the system further includes making the system independent of the selection of the database by using the database extraction layer.

  Also, in the first aspect of the present invention, the conference setting may include one or more of location, conference type, user preference, and presence of remote participants.

  In order to achieve the above object, an invention according to claim 8 is a conference task support system for supporting a specific task such as holding a conference by adjusting the operation of a hardware device and a software application. The system includes one or more computers connected by a network, at least one setting storage unit provided in any one of the one or more computers, and the one or more computers. A conference held by one or more presenters, comprising at least one room control unit, and one or more devices and applications corresponding to each room control unit. And the device and application by one or more presenters for the conference setting to support a plurality of presenters. The conference settings are stored by the settings storage, each presenter setting includes a subset of the one or more devices and applications, and adjustments to the operation of the devices and applications include: The room control unit performs the setting based on the setting of the one or more presenters for the conference.

  The invention according to claim 9 is the invention according to claim 8, further comprising a software wrapper for separating a difference specific to the device and the application, so that the system can be run by using the software wrapper. Inside, additional devices and applications are further added to the system.

  The invention according to claim 10 is the invention according to claim 8, wherein the adjustment of the operation of the device and the application is performed autonomously by the room control unit and the interface by the presenter. One of the adjustments based on the setting is included.

  In the invention according to claim 8, each room control unit may further include at least one display device.

  The invention according to claim 11 is the invention according to claim 8, wherein the adjustment of the operation of the device and the application is adjustment in the case of simultaneous presentation and adjustment in the case of continuous presentation by the plurality of presenters. And one of the adjustments in the case of a presentation that combines simultaneous and continuous.

  The invention according to claim 12 is the invention according to claim 8, further comprising an authentication device for authenticating the presenter, both a conference set by one or more presenters and adjustment of operation of the device and the application The authentication device requires authentication by at least one of the one or more presenters.

  The invention according to claim 13 is the invention according to claim 8, further comprising a database for storing conference information including conference, room, presenter, setting, device, device status, and attribute information. It is said.

  Further, in the invention described in claim 8, information for each device may be stored in the one or more databases.

  The invention according to claim 14 is the invention according to claim 13, wherein the database includes one of an existing commercially available database, a custom database, and open source software, and the application is an existing one. Commercially available software, wherein the device comprises one or more of a projector, an electronic whiteboard, a video switch, an audio switch, and a video conferencing device.

  The invention according to claim 14 is the invention according to claim 13, further comprising a database extraction layer that recognizes a plurality of different databases and provides a common interface so that the system does not depend on the selection of the database. It is characterized by that.

  In the invention according to claim 8, the conference setting may include one or more of a location, a conference type, a user preference, and the presence of a remote participant.

  The invention according to claim 16 is a program for causing a computer to execute a function of supporting a specific task such as holding a conference by adjusting operations of a hardware device and a software application. Providing one or more computers connected to a network, providing at least one setting storage provided in any of the one or more computers, and the one Providing at least one room controller provided in any of the above computers, providing one or more devices and applications corresponding to each room controller, and one or more persons To create a meeting for each presenter, and each meeting setting has multiple presenters. Device and application configurations by one or more presenters to support the conference, the conference settings are stored in the configuration storage unit, and each presenter configuration is a part of the one or more devices and applications. Including a set, and adjusting the operation of the device and application in the conference by the room controller based on the configuration of the one or more presenters for the conference. It is characterized by including.

The invention according to claim 17 is a user interface device connected to a computer for operating one or more physical devices used for conference room setting,
a) a touch screen that presents the user with various options;
b) a plurality of touch screen controllers that are displayable to the user to allow the user to select any of the options;
Each of the touch screen controllers has target-oriented information that allows the user to select a target, and controls the physical equipment that is required to execute the target based on the selection at the controller It is characterized by executing this process.

The invention according to claim 18 is the invention according to claim 17, wherein the target is control of at least one of a video conference, a presentation, a whiteboard, and an external laptop.
The invention according to claim 19 is the invention according to claim 17, wherein the goals are selection of a video conference attendee from a list of attendees, selection of a presentation from a list of presentations, and a list of whiteboard memo sets. A selection of a memo set from the list of laptops and a selection of a laptop from a list of laptops.

  The invention described in claim 20 is characterized in that, in the invention described in claim 19, the target is any one of a function of turning a page of a list, a presentation, and a whiteboard memo.

  The invention according to claim 21 is a program for causing a computer to realize on a touch screen a user interface for operating one or a plurality of physical devices used for conference room setting. a) the ability to present the user with various options having goal-oriented information that allows the user to select a target on the touch screen; b) execute the target based on the user's selection of any of the options And a computer program for realizing a function of executing a process for controlling the physical device, which is necessary for the purpose.

According to the present invention, there are provided a conference task support method, a conference task support system, and a program for supporting a specific task such as holding a conference by adjusting operations of a hardware device and a software application. There is an effect.
Also, according to the present invention, the process for controlling the physical device has target-oriented information that enables a user to select a target, and is necessary to execute the target from one selection of a controller. , Thereby providing a user interface device and computer program that isolates the user from the underlying complex processes required to perform the goal.

  Preferred embodiments of the present invention will be described in detail with reference to the following drawings.

  The present invention includes a flexible and extensible architecture that organizes device behavior for users based on settings created for a particular situation. Such situations include smart environments, especially with respect to conference rooms and classrooms. In a conference room setting, the system can support video conferencing by coordinating the use of projectors, computer displays, and video conferencing systems and sharing documents. Multiple devices are adjusted based on a given setting so that each device is in the proper state for a particular user task. Users can control the system by selecting task-oriented actions while performing the device organization required for the system to perform tasks It is.

  The system includes a number of components as shown below. A configuration store, one or more organizers for each of the physical spaces that need to be controlled, an optional user console (terminal) for each space, and one or more controlled for each physical space It is a device. The system architecture includes a database that stores information about devices, rooms, people, and meetings. The system software associates contextual information (eg, video conferencing system IP address, application settings, and phone number) with specific users, rooms, and specific devices corresponding to the conference. Inheritance (inheritance) mechanism using prototype instance notation is realized.

  This system architecture is extensible. The reason for extensibility is that this architecture supports a dynamic plug and play device function, unlike a canned room control environment that is designed to work with a specific set of devices. Because. Devices can be swapped without rebooting the system or affecting existing settings. New devices can be introduced without requiring any changes to the software code in the existing organization framework. Thus, the system software does not need to be rebuilt to accommodate new devices added to the conference room settings.

  Organizing devices eliminates the need to manage multiple devices independently. Therefore, the usefulness and usability of the conference room are improved. The present invention can incorporate existing complex devices such as video switches, video conferencing systems and applications that are not “basic” such as joysticks and buttons. No dedicated remote control device is required to operate the entire system.

  The present invention is predictable and controllable for the user. The present invention can relate to supporting short-term meetings, rather than long-term tasks that can move from one location to another. For example, if a projector is turned off when a meeting starts, the system of the present invention turns it on. It does not perform the less desirable action of rerouting the display to another projector. Reduces the need for online support staff to manipulate the environment space, reduces system setup and maintenance time, and reduces the number of expert users required to set up and operate equipment .

<I. System software architecture>
FIG. 1 shows an example of the software architecture of a system according to an embodiment of the present invention, illustrated as a stack structure of components. The stack structure of each component of the system software 100 can be classified into the illustrated seven layers. These layers are shown in box 102. Layer I includes third-party products, open source devices, and applications. Layers II, III, and IV include wrappers, persistence interfaces, and data objects, respectively. Layers V, VI, and VII include a security layer, an application layer, and a user interface (UI) layer, respectively.

  In FIG. 1, the layer to which each component belongs is shown in parentheses after the part name. Starting with the lowest layer components, device and application layer I includes an authentication device 145, a database 105, and a communication / presentation device 130. The wrapper layer II includes an authentication device wrapper 150, a database extraction layer 110, and a communication / presentation device wrapper 135. The persistence interface layer III includes an authentication and access persistence interface, that is, an AA persistence interface 155 and a persistence interface 115.

  The data object layer IV includes a generic object 120 and a business object 125. The security layer V includes an authentication device 147 for generating an authentication request, illustrated as security 147, along with authentication and access control 140. The application layer VI includes application logic 160. Further, the UI layer VII includes a UI 149 of the authentication device 145 and a UI 132 of the communication / presentation device 130 in addition to the user interface (UI) 165.

  The system includes one or more databases 105 that store setting information and authentication information. Examples of such databases include, but are not limited to, “MySQL” and “Oracle” relational databases. These databases 105 are wrapped (accessible) by the database extraction layer 110. The database extraction layer 110 manages access to the database 105 and configures the rest of the system software 100, such as application logic 160, regardless of the particular choice or brand of the database 105. In other words, the database extraction layer 110 is aware of the various types of databases and provides the same interface for that client regardless of which database is selected and used.

  In addition, persistence interface 115 allows database records to be manipulated as first class objects. It also allows the persistent storage details to be isolated from the system software 100 for persistent data that exists from session to session. Persistence interface 115 transparently associates objects in memory with records in database 105. This interface reads the database 105 to create an object or stores the object in the database 105.

  When the system software 100 uses these databases, the database extraction layer 110 and the persistence interface 115 collaborate to store appropriate data in memory. The database 105, the database extraction layer 110, and the persistence interface 115 may be software of other companies, commercially available software, or open source software.

  The generic object 120 represents the schema of an object stored in the database 105. These objects include conferences, rooms, settings, conference creators, devices, and device status, as well as related attributes. These objects are described in detail below with reference to FIG. Database extraction layer 110, persistence interface 115, and generic object 120 represent data in database 105, but do not perform conference-specific functions.

  On the other hand, business objects 125 include objects that directly control the various communication / presentation devices and applications 130 that form an instance of system software 100. Examples of communication / presentation devices and applications include, but are not limited to, the Tandberg system and Sony's “PCS1 video conferencing system”. The business object 125 is an IDevice type object created by the room control unit on the room control device PC315, as will be described in detail with reference to FIG.

  The device state collection information obtained from the generic object 120 is passed to the business object 125. The business object 125 is converted into new collection information of an IDevice type object that knows what operation should be performed to manage the conference. The communication / presentation device wrapper 135 manages device and application access and allows the remainder of the system software 100, such as application logic 160, to be selected with specific selections, including device and application 130 types and brands. Has an unrelated configuration.

  For example, if the devices are different for two video conferencing devices, the actual commands sent to turn each of these devices on / off and dial will also be different. However, the fact that the two video conferencing devices have the ability to dial and connect / disconnect is important. Communication / presentation device wrapper 135 provides extensibility. This is because the device and application 130 can be dynamically loaded without restarting the system or affecting existing settings.

  Authentication and access control is generally performed by a security component indicated at 140, ie, an authentication and access control device. This software component includes authentication of the user following sign-in (connection), generation of communication tokens for the signed-in user, distribution of these communication tokens to other components of the system software 100, and upon request The operation includes confirmation of the token for communication. This part maintains a user account, but if the login credentials correspond to the login of a Windows (Windows) domain, the Windows domain (Windows) Domain) authentication can be used to authenticate the user. Therefore, this component enables both domain users and remote users to use the system software 100.

  In addition, an authentication request can be generated using the authentication device 145 with high reliability. These authentication requests are illustrated as security 147. Examples of authentication devices include, but are not limited to, readers such as “FeliCa” that performs radio frequency identification (RFID) of user badges, and user biometric readers (user biometric readers) ) And other hardware.

  The authentication device wrapper 150 manages the access of the authentication device and configures the rest of the system software 100, such as application logic 160, regardless of the particular choice, including the type and brand of the authentication device. Yes. Authentication device wrapper 150 provides similar access to various authentication devices 145.

  In addition, the authentication and access persistence interface, ie, the AA persistence interface 155, manipulates the database records associated with authenticating the user and authorizing the user to access as a first class object. Make it possible to do. The AA persistence interface 155 also allows persistent storage details to be isolated from the rest of the system software 100, such as application logic 160.

  Application logic 160 corresponds to a network of distributed parts that communicates the appropriate configuration information to the correct room at the correct time and organizes the conference. Application logic 160 is in communication with a console that allows the user to use the system.

  Ultimately, multiple user interfaces (UI) 165, such as a conference console and configuration editor, allow the user to create, modify, and use information stored in the system software 100. Is possible. Multiple users can connect to the system in parallel (simultaneously) through these UIs 165. Further, UI 132 allows a user to interface with communication / presentation devices and applications 130, and UI 149 allows a user to interface with authentication device 145. A more detailed description of UI165 can be found in VIII. It is described in the user interface device.

<II. Database schema>
FIG. 2 shows an example of the generic object 120 of FIG. 1 according to the embodiment of the present invention. FIG. 2 shows a schema (structure) of an object stored in the database 105. This database structure organizes database objects as follows. Each object has a unique identification number (ID), each object contains zero or more properties that are stored as fields in the table, and each object is associated with a number of attributes It is possible.

  Object types include conference 210, room 220, settings 230, creator 240, device state 250, device 260, and attributes 270. Each of the objects includes a primary key, indicated as PK in each object, which is the object's ID. Each of the objects may also include one or more foreign keys, as illustrated as FK1, FK2, FK3 in the various objects of FIG. These foreign keys are pointers to other objects.

  Hereinafter, when objects are represented, they may be referred to as a conference object 210, a room object 220, a setting object 230, a creator object 240, a device state object 250, a device object 260, and an attribute object 270. In the case of representing a table, there may be a conference table 210, a room table 220, a setting table 230, a creator table 240, a device state table 250, a device table 260, and an attribute table 270.

  Each meeting is associated with one room. Each configured conference has a record in conference table 210. Each physical space such as a conference room is represented by a record in the room table 220. In addition, each conference can be associated with one or more configurations to accommodate multiple presenters simultaneously. Each conference presenter using the system has a corresponding record in the creator table 240. In the case of the conference table 210, FK1 is a pointer to the room table 220, and FK2 is a pointer to the setting table 230.

  Each of the settings in the conference has a person who owns the conference, i.e. a creator. Each setting is a collection of one or more device states and has a corresponding record in the setting table 230. In the case of the setting table 230, FK1 is a pointer to the creator table 240, and FK2 is a pointer to another record in the setting table 230. This makes it possible to inherit one setting from another setting. FK3 is a pointer to the device status table 250.

  Each device is associated with one or more device states. Each physical device type and application type has a corresponding record in the device table 260, and each device state has a record in the device state table 250. In the case of the device status table 250, FK1 is a pointer to the device table 260. In addition, FK3 is a pointer to the root table and provides more context for the object.

  Thus, each device state object stored in table 250 can serve as a prototype for other device states. For example, a first device state object refers to a second device state object. In that case, using the prototype-instance notation inheritance, this second object can point to a third object, and in turn can point to one after another. Each additional device state object stored in table 250 may have additional attributes, examples of which are described in connection with FIG.

  Each of the other types of objects, with the exception of the attribute object 270, can have zero or more arbitrary attributes. Only the room object 220, the device state object 250, and the device object 260 show these attributes in FIG. Attributes for these objects are stored in attribute object 270. A value is stored in the attribute object 270. It is also possible to assign values dynamically. Therefore, information that is rarely used can be stored as an attribute, and no space is taken as a field in the table.

  Device related information is represented as attributes. For example, in the case of a projector controlled by a different serial port, the value of the serial port can be expressed as an attribute. As another example, a URL (universal resource locator) associated with a service representing a room can be represented as an attribute. For the room object 220, device state object 250, and device object 260, FK1, FK2, and FK1 are each an associated pointer to the attribute object 270.

<III. Configuration of each part>
FIG. 3 shows a diagram relating to an example of system logic components corresponding to the software components of FIG. 1 and their connections, according to an embodiment of the present invention. These logic components are exemplary subsets of the available components that the room comprises and can include other types of logic components not shown, as will be appreciated by those skilled in the art. These logic components include services that drive applications to control devices in the room, perform authentication, and perform configuration.

  Although all hardware and software components of the system of the present invention can be executed on the same computer, these components are generally distributed over several machines, and these machines are placed in the same place. Some machines are located, while others are located in remote locations. Therefore, the realization of logical components and services is arbitrary.

  Communication between components is mediated by web service calls, but other communication mechanisms such as remote procedure calls (RPC) and other custom protocols can be used. The hardware components can be connected to each other using various methods such as video graphics array (VGA), universal serial bus (USB), Ethernet. In the example of FIG. 3, the VGA (video) connection is indicated by a solid line. USB (serial) connections are indicated by small broken lines. Ethernet connections are shown with large dashed lines. The system connection types and combinations shown in FIG. 3 are for illustrative purposes and are not limited to those shown in this figure. For example, NTT's video conferencing system uses "Telenet", while Tambo's video conferencing system uses serial connection. As another example, other types of connections not shown in FIG. 3 may be used.

  FIG. 3 illustrates one hardware embodiment 300 of the system. The system hardware 300 includes a setting repository (storage unit) 310 and a room control personal computer (PC) 315. The system hardware 300 includes a first display PC (display PC 1) 320 and a second display PC (display PC 2) 325 as two controllers. The setting repository 310 is a server that stores conference settings. When making settings, someone specifies what settings should be used for a conference presenter, device, application, and document for a specific conference in a specific room.

  One setting repository can be shared by a plurality of rooms on one network. In the case of equipment that includes only one room, the room control PC 315 can store the conference settings. In the case of facilities that include multiple meeting rooms, it is possible to use one or more configuration repositories. If a system is installed in multiple geographic areas and a conference is held in a remote area, the configuration repository 310 is replicated in these various geographic areas, which is referred to as latency. It is possible to avoid problems.

  The display PC 320 and the display PC 325 project the display on the electronic whiteboard of the corresponding SMART board by the corresponding overhead projector. The first display PC (display PC 1) 320 projects the display on the first smart board (smart board 1) 330 by the first projector (projector 1) 340. The second display PC (display PC 2) 325 projects a display on the second smart board (smart board 2) 335 by the second projector (projector 2) 345.

  The room control PC 315 can control the projector 340 and the projector 345 via a service executed on the machine. Alternatively, the display PC 320 and the display PC 325 can control the projector 340 and the projector 345, respectively. The room control PC 315 controls the video switch 350, which in turn controls the routing of the display. Routing includes a guest laptop 355 that can be plugged into a VGA cable in the room. Using the video switch 350, a plurality of different video sources can be selected. The room control PC 315 can also control other functions such as room lighting, audio, and air conditioning.

  The system hardware 300 can be accessed by the user via any number of devices, such as a remote computer 360, a console tablet PC (console tablet) 365, and a personal digital assistant (PDA) 370, for example. It is possible to control. For example, a “Felica” badge reader (Felica reader) 375 using a RFID, a fingerprint reader 380, a biometric reader 385, an optical device (not shown), a magnetic card reader ( Performed by any number of devices, such as a cellular telephone (not shown).

  In general, any one of the user consoles, that is, the computer 360, the console tablet 365, and the PDA 370 is connected to and controlled by any of the Felica reader 375, the fingerprint reader 380, and the biometric reader 385 that are authentication devices. However, other machines can be connected to these authentication devices to control them. For example, in the case of equipment having one room, it is possible to connect the room control PC 315 to the authentication device and control them. In general, it is only necessary to provide an authentication service for handling user authentication on one server each time the system is set up. The authentication service is generally located with a configuration repository. In the case of equipment having only one room, the authentication service and the setting repository can be provided in the room control PC 315.

  Video conferencing devices 395 such as Tambo, Sony, QUALCOMM, and NTT can be used for video conferencing. The video conference device 395 is controlled by the room control PC 315 via a service executed on this machine. The P box 390, that is, the projector box, is a system for recording (capturing) and reproducing (playback) a conference that can be freely operated. The video switch 350 can output the presentation that is input to the video switch 350 as it is. This output is sent to the P box 390 for later playback. Details of P-Box 390 are published US patent application entitled “System and Method for Presentation Acquisition and Storage” filed October 26, 2004 by Laurent Denoue et al. 10 / 973,698.

  Two or more serial devices and two or more TCP / IP devices can be controlled simultaneously. Similarly, one or more applications can be executed on each display PC of the display PC 320 and the display PC 325 corresponding to the smart board 330 and the smart board 335. In order to operate the entire system including the system hardware 300 and the system software 100, no dedicated remote control device is required. Any of the devices of computer 360, console tablet 365, and PDA 370 can control the system regardless of which device is used. Furthermore, the present system can be autonomously executed by a conference setting including a setting for autonomously performing a conference at a certain day without being controlled by a user in front of the user console. .

  In the following description regarding the interaction between the system components, the system software component of FIG. 1 and the system logic component of FIG. When referred to as “system”, this “system” includes the components shown in FIGS.

<IV. Device and device status>
The setting repository 310 manages the database 105 of each object of the conference 210, the room 220, the setting 230, the creator 240, the device state 250, the device 260, and the attribute 270 corresponding to the schema of the database shown in FIG. . Each physical room can include one or more devices. Each of the conferences uses one or more settings to organize the use of some subset of devices provided in the physical room. The configuration object 230 represents a device by collecting a set of device state objects 250. Each device state object 250 represents some device in a particular context.

  In the example of FIG. 3, the room may include two identical projectors 340 and projectors 345. The database 105 includes one device object 260 that characterizes both the projector 340 and the projector 345 projector types. There is one conference setting for each of the projector 340 and the projector 345, and two device state objects 250 are included in total. Device state 250 specifies how each of its attributes should connect a particular projector 340 and projector 345 to the system. Among them are specifying how the projector itself is connected to the video switch device 350 and which serial port should be used to control the projector itself.

<V. Meeting>
Each meeting is associated with at least one presenter. Each presenter is also associated with a setting. Any associated presenter can initiate a meeting. Prior to initiating a conference, each of the conference presenters sets up their part of the conference. Settings are made by selecting the room, device, and application to be used, and by specifying the document and destination for the video conference if used during the conference. Each room can have a default configuration, but each meeting can be customized with respect to the room settings. Through the UI group 165, the presenter can access the application logic 160 using a device such as a computer 360 somewhere to schedule and set up a meeting. After the presenter sets his / her part of the conference, the settings are saved in the settings repository 310.

  At the scheduled meeting time, any user associated with this meeting can start the meeting using the UI 165, even if it was set by one of the presenters of the meeting. is there. Users associated with the conference must be authenticated by any of the authentication devices Felica reader 375, fingerprint reader 380, and biometric reader 385, according to their individual settings. Alternatively, the system can start the conference autonomously at the scheduled conference time. The system also includes a preview mode that allows any presenter to preview the meeting prior to the scheduled meeting time.

  When the system is started, the organizing parts included in the application logic 160 request the settings repository 310 to make the appropriate settings for the room or space to be controlled. Requests time setting, though optional. Also, optionally, one of the presenters requests that a meeting be started. The information returned by the setting repository 310 is converted into a conference setting and transmitted to the room control service being executed on the room control PC 315. If this conference is associated with more than one setting, only one setting is active at a time.

  During the meeting, the presenter can select (dynamically) who will give a presentation at a given time as appropriate. This configuration includes a collection of device state objects 250 that describe how each device and application involved in the conference should be configured. This allows the system to place devices involved in the conference in a known state regardless of the previous state. Therefore, even if the device is manually operated, for example, switching between video sources or changing projector settings between meetings, this does not affect the behavior of the system during the scheduled meeting. For example, even if the presenter switches to another video source, the system can be used when the scheduled meeting is resumed by using the “well known state” information in the device state object 250 of the original video source. Can automatically return to the original video source.

  FIG. 4 shows an example of a logical structure for setting a conference at the start of the conference according to the embodiment of the present invention. As described above, the setting repository 310 transmits this setting to the room control service on the room control PC 315. The illustrated conference, or conference 405, is scheduled to be held in room 410 at some time of the day. Two creators 415 and 420, that is, presenters, are scheduled to make a presentation at a conference, and settings 425 and 430 are created. Each of these settings is associated with three device states and two devices.

  The setting 425 has two devices 435 and 440 associated with the setting 425 itself. An example of the device 435 is a projector. Here, there are two device states 450 and device states 455 corresponding to the projector 340 and the projector 345, respectively. For setting 425, device 440 has one device state 460 associated with device 440 itself. Device state 465 is the context for device state 460 and device state 470 is the context for device state 465.

  The setting 430 is associated with the device 440 and the device 445. For configuration 430, device 440 has two device states 475 and 480 associated with device 440 itself. Device state 475 and device state 480 are two different states for device 440 that are the same device. Finally, in the conference setup illustrated here, device 445 has only one device state 485 associated with device 445 itself. The illustrated device 445 is a video switch having a unique device state 485 corresponding to one instance of the video switch 350, as in FIG.

<VI. Devices at the meeting>
For each device state object 250 being set associated with the conference to be started, the room control unit on the room control PC 315 creates an IDevice type object for managing the device or application. . “IDevice” is an interface through which the room controller interacts with all devices and applications regardless of their purpose. These IDevice type objects are business objects 125, and the room control unit is included in the application logic 160.

  Each IDevice has a unique identification number (ID), and it is possible to implement an initialization code described in detail below and execute a command. This interface makes the system extensible. This is because any component that implements this interface can be added to the running system without requiring any changes to the knitting component. A factory that creates an object can receive as input a string that specifies a given logical object type, that is, an object class and an external library in which the class is defined.

  These specifications are passed as attributes relating to the device state object 250. Such a library can be introduced at any time. Thus, objects can be added dynamically to the system without having to recompile or restart any computer in the system. Thus, for example, a physical device can be replaced, such as replacing one model of projector 340 or projector 345 with another model. It is also possible to update a setting and control a new device by modifying one entry (registration information) in the setting repository 310 or adding one file to a computer that executes a knitting component. Is possible.

  Some interaction is required between devices. For example, an IDevice representing an application needs to know an IDevice representing a display machine on which the application runs. Therefore, IDevice creation is a process consisting of three phases.

  In phase 1, a new instance of IDevice is created for each device state object 250 being configured. In general, every time an IDevice is realized, an attribute of the device state object 250, which is a basis for creating the internal state, is used. Along with this, ID reference information for other IDevices used in the operation of the instantiated IDevice is often stored. Other parameters such as, for example, serial port information can also be stored in this phase.

  In Phase 2, reference information for various IDevices is analyzed. Using the ID cached (temporarily stored) in phase 1, IDevice instances are found, and reference information for these IDevice instances is stored.

  In phase 3, the device code is executed by the business object 125 to bring the device into a known state, for example, to launch an application. This phase can be executed by assigning priority to the device state object 250 being set. Thus, for example, applications start before the user console attempts to send commands to these applications.

<VII. Application>
The devices controlled by the present system include, for example, serially controlled devices such as the exemplary projector 340, projector 345, video switch 350, and video conference device 395, a setting repository 310 that is another computer, and room control. Network-controlled devices such as a display PC 315, a display PC 320, a display PC 325, and a P box 390. Applications such as “Power Point” running on a computer in a controlled space also constitute the device to be organized.

  The application is controlled by controller components on the first display PC 320 and the second display PC 325 included in the application logic 160. The controller component receives input from the knitting component and selects a corresponding application for display on, for example, the first smart board 330 or the second smart board 335. A plurality of applications can be run simultaneously on each of the first display PC 320 and the second display PC 325 which are display machines. The system places no restrictions on what kind of applications can be run in this way. Further, in the present system, there is no limitation when two display PCs are provided, and any number of display PCs can be set in the system.

  Each of the applications can receive commands via the controller component. There are three classes of applications in this system. These are system-specific applications, opaque applications, and partially automatically controlled applications.

  An electronic whiteboard application is an example of a system specific application. This can be controlled at a fine-grained level by inputting a custom message into the message queue (holding unit). An opaque application such as a Web browser can be opened, closed, hidden, and displayed. However, it cannot be controlled in any custom way by the system. Finally, the third type of application is an application having a public application program interface (API) in order to control its execution. For example, “Power Point” is a “COM (component object model)” API that advances a slide and performs other remote control operations.

  It is possible to create a special IDevice that controls “PowerPoint” programmatically. Thus, the presenter of the present system, for example, a console such as the console tablet 365 without physically interacting with a computer controlling the display, such as the first display PC 320 controlling the first smart board 330. Can be used to advance the presentation on a general display. Therefore, a remote presenter for a certain space can perform a presentation for a local audience in the same manner as a local presenter in a certain space.

<VIII. User interface device>
In this embodiment, a UI 165 necessary for designing a user interface of a conference room designed for ease of use in the conference room having the next-generation functionality is provided. In an embodiment, the Usable Smart Environment (“USE”) system is flexible and expandable designed specifically to improve usability in smart environments (especially conference rooms and classrooms) A simple architecture. The USE system features an easy-to-use, customized central control console. The design of the console and the architecture of the foundation system are based on a comparative cultural ethnological study of how people use conference rooms. The system allows customization and personalization of smart environments tailored to specific people and groups, types of work, and unique physical spaces.

  In embodiments, the focus of the USE system is to allow users to enter a conference room to create and maintain interrelationships, not necessarily using technology. This focus will lead to the integration of individual technology equipment to support the natural behavior of people meeting in the conference room without burdening the technology.

  In an embodiment, the USE system is based on a “no wizard required” conference room designed for ease of use while ensuring next generation functionality. The USE includes features such as a unique user interface (UI) that mediates the multi-display system, immersive (surrounded by the display) conference with document support, digital whiteboard / annotation, and secure authentication. When scheduling a meeting, the user can select pre-defined configurations or modify those configurations to meet the needs of the meeting by assigning an application to the display. In the following, user interface design is described in detail.

  In an embodiment, the USE system adjusts the operation of the device on behalf of the user based on a configuration created for a specific situation. For example, in a conference room setting, the system coordinates the use of projectors, computer displays, and teleconferencing systems to support videoconferencing with shared documents. Unlike existing infrastructure, dedicated remote control equipment is not required to operate the entire system. Unlike other conference room control systems, this approach does not require system software that needs to be rebuilt when new equipment is imported.

  In an embodiment, the USE system effectively balances ease of use with a new type of functionality. Examples of new types of functionality include multiple displays, new interfaces, rich media systems, new upload / access / security systems, and robust mobile integration (incorporating mobile devices such as smartphones and laptop computers into meetings) There is. As developed in areas such as context-aware computing, interactive furniture, embedded systems, and mobile devices, users adapt their personal devices across all technologies they encounter (especially in smart environments) Expect to have the ease of use possible.

(User console user interface)
(Explanation)
The UI assumes that a plurality of physical devices are operated from one touch screen or another mobile computing device, for example, using the console tablet 365 in FIG. 3 as described above. The UI is a graphical interface capable of communicating with the conference room controller via a web service that operates as a bridge between the UI and the controller. All communications going back and forth through the web service bridge can be handled via an XML socket connection.

  FIG. 5 illustrates that the user interface can be adapted from a goal-oriented interface to a process-oriented interface, according to an embodiment. The UI is adaptable to each individual user. FIG. 5 shows the correspondence between items related to goal orientation and process orientation, and all UI variations for different users will fall within the scope of these items. The user's location in these items is assigned as a rating when the user registers with the system. This rating can be updated as the user continues to use the system.

  In its simplest form, the UI presents only the end result possible (eg, “Call Tom” or “Present your PowerPoint file”) to the user. This goal-oriented UI isolates the user from the complex processes needed to fulfill the user's request, as shown in “Goals” in FIG. Individual users of this type trust the system by presenting all the options in a manner that can be reassured by goal-oriented users. This allows the user to concentrate on their work rather than using the UI. For example, this goal-oriented UI is likely to be used by executives such as the CEO. Thus, as shown under “Goals” in FIG. 5, the goal-oriented UI is “visually simple UI” and “clear single-step operation” (which allows the user to For users who want a more “opaque process”, which allows the user to achieve their goals without having to worry about the complex processes to fulfill the user's request. Processes necessary for executing each goal are registered in advance in association with each goal so as to be instructed and executed through the application block 160 of the conference task support system.

Many users do not trust systems that hide processes from users too much. Such a user can then be presented with a more detailed, process-oriented UI as shown under “Process” in FIG. Thus, this UI exposes more about the underlying system to the user and allows finer user control. This level of detail can confuse many users. Thus, at some point it is impractical for the user to add complexity to the UI. As the UI presents the functionality of any device controller on a one-to-one basis, the UI becomes increasingly similar to using the original control. Therefore, as shown under “Process” in FIG. 5, the process-oriented UI allows the user to perform finer user control, “visually complex UI”, “multi-step operation”. , And for users seeking a “clear process”.
(Design considerations)

  Some conference room equipment in a conference room may take several seconds to respond to user input. In an embodiment, the UI may slow down the user slightly by slowing down any animation or transition in the UI. In an embodiment, even an object such as a button shown in FIG. 18 can have a slower animation cycle than normal.

  In an embodiment, the UI is capable of operating on a touch screen tablet PC. Alternatively, the UI can run on any type of computer or laptop. Instead of a touch screen, any type of pointing device can be used with a standard computer screen. However, as already indicated, one of the goals of the UI is to provide the user with a non-computer-like interface. For that user, an individual console with a UI is simply a tool that controls the user's meeting, not a computer. If the user views the console as a computer and exits the UI to check mail or surf the web, the user loses control of the conference room.

  In an embodiment, the UI element that the UI characterizes is a clearly labeled virtual button. The buttons are circular or triangular in shape for ease of use. In addition, the buttons are shown in green, red, and yellow primaries, again for ease of use. In an embodiment, any button may be any shape, size, and color. In embodiments, one or more of the buttons may be any type of controller (such as a slider).

This UI has a completely easy-to-use design with little need to interpret the meaning of the information displayed on the screen. The examples of FIGS. 8-18 are screen shots that capture a simpler goal-oriented UI embodiment of the control console. In an embodiment, the UI features any combination of UI elements, such as a toolbar, palette, and clearly labeled buttons.
(Design details)

  FIG. 6 is a photograph showing an exemplary conference room with two display screens 610 and 620, according to an embodiment. The two display screens 610 and 620 are shown in the back wall of FIG. 6, but may be anywhere in the room. In an embodiment, any number of display screens in any configuration can be used in the USE UI. FIG. 6 further shows two projectors 630 and 640 suspended from the ceiling, which project displays from different applications onto the display screen. In FIG. 6, two speakers 650 and 660 are also shown on the back wall, which are used for teleconferencing or other applications.

  FIG. 7 is a photograph showing an example meeting room with an example USE UI on an example touch screen tablet PC 705 used with two example display screens 710 and 720, according to an embodiment. This conference room is set up in the same manner as the conference room shown in FIG. The user at the seat is not shown, but a video conference is in progress. In general, there are users and any number of other people in a conference room. The user who convened the meeting uses the touch screen tablet PC 705 on the table to open and control the meeting as will be described in detail later. In this example, the PowerPoint presentation is also part of the video conference. Shown on the left display screen 710 is a slide of a PowerPoint presentation. Shown on the right display screen 720 is one of the video conference attendees at a remote location (which may be another conference room in the same building or some other location). The description associated with FIGS. 8-18 includes references to the elements shown in FIG.

  FIG. 8 is a UI screen shot showing an exemplary start screen, according to an embodiment. When the user enters the meeting room to open a meeting, the user sees this screen displayed on a UI touch screen (eg, the touch screen of tablet PC 705). In an embodiment, a stand-alone Radio Frequency Identification (RFID) smart card device (not shown) is connected to the USE via USB and placed on the table next to the touch screen tablet PC 705. . In an embodiment, the user causes his smart card to be read through an RFID smart card device for identification (which will be described in detail later). Shown in the center of the UI touch screen is a “Start” button 810 that allows the user to start USE by simply touching this button. When the user decides to end the conference using the UI touch screen, the USE system returns to this screen.

  FIG. 9 is a UI screen shot showing an exemplary main menu of an application operable for the exemplary two screens of the conference room shown in FIG. 7, according to an embodiment. This screen is divided into a left half and a right half because the conference room shown in FIG. 7 has two display screens. In FIG. 9, the left half of the screen is labeled “Left Screen Options” 905 and controls the applications that can be run on the left display screen 710 of FIG. The right half of the screen of FIG. 9 is labeled “Right Screen Options” 910 and controls the applications that can be run on the right display screen 720 of FIG. The types of applications that can be run on different display screens may be different because the functions of the display screen may be different. Examples of applications that can be run and displayed on a display screen include PowerPoint (PowerPoint Presentation), whiteboards (bidirectional whiteboards with electronic markers that allow writing on electronic whiteboards), and external laptops ( There is a laptop that is added to the UI console regardless of whether the UI console is a laptop). Employees or guests may bring these laptops into the meeting room.

  Applications that can be used on individual screens are programmed into the USE before the user initiates the conference using the USE UI. In addition, information about each user is programmed into the USE, and each user has an associated identification number (“ID”). For example, suppose a user intends to incorporate one or more PowerPoint presentations into a meeting. The user's file containing the presentation is preloaded in the USE and is identified by the user's ID. In another example, information about the expected meeting attendees is preloaded into the USE and associated with the user's ID. Attendee information includes the attendee's name and contact information (such as a phone number). When a user causes his RFID smart card to be read through the device on the touch screen, the information associated with the user's ID already programmed in the USE is displayed during the user's current USE session. It becomes usable for the user.

  The example of FIG. 9 shows only two screens, but in an embodiment, the USE can handle any number of screens. When dealing with more than two screens, the user interface of FIG. 9 can be tiled. The USE can configure itself depending on how many screens need to be handled simultaneously. Thus, for more than two screens, the main menu of FIG. 9 can be displayed as tiles that are each squared in a row and column arrangement. For multiple screens, additional screen labels are required instead of the two screen labels 905 and 910. For example, the screen labels for multiple screens would be screen 1, screen 2, screen 3, ..., screen N, where N is the number of screens.

  In an embodiment, additional applications can be embedded in the USE controller to support the main menu application of FIG. These additional applications can allow the use of external hardware devices. For example, a multi-screen processing application (eg, an application that uses Explicit Parallel Instruction Computing (“EPIC”)) is used in USE to perform multi-screen processing as a background function. It is possible. Another example is an application that can support a High Definition (“HiDef”) conference. Another example is an application that allows a user to display a three-dimensional (“3D”) PowerPoint presentation. Another example is an application that can handle the use of mobile phones in meetings.

  In FIG. 9, applications that can be displayed on the left display screen 710 of FIG. 7 are listed as buttons that the user can select. These exemplary buttons are a “presentation” button 915, a “whiteboard” button 920, and an “external laptop” button 925. If the user wants to turn off the left display screen 710 of FIG. 7 and use only the right display screen 720, the user can select an “off” button 930. In FIG. 9, applications that can be displayed on the right display screen 720 of FIG. 7 are listed as buttons that the user can select. These exemplary buttons are a “presentation” button 935, a “whiteboard” button 940, a “video conference” button 945, and an “external laptop” button 950. If the user wants to turn off the right display screen 720 of FIG. 7 and use only the left display screen 710, the user can select an “off” button 955.

  In any screen shown in FIGS. 9 to 17, when the user selects the “option” button 980, the USE displays a main menu of available applications displayed in the left half of the screen in FIG. 9. These are applications that can be used for the left display screen 710 of FIG. When the user selects the “option” button 985 on any screen shown in FIGS. 9 to 17, the USE displays a main menu of available applications displayed on the right half of the screen in FIG. 9. These are applications that can be used for the right display screen 720 of FIG. Further, on any screen shown in FIGS. 9 to 17, the user can end the conference at any time by selecting the “end conference” button 990.

  FIG. 10 is a UI screen shot showing an exemplary list of video conference attendees for the user for the exemplary video conference selected by the user in FIG. 9, according to an embodiment. In this example, the user selects the “TV conference” button 945 on the “right screen” 910 in FIG. 9, and the UI screen shot in FIG. 10 is displayed. The right half of the screen in FIG. 10 is now labeled “Video Conference” 1010. The brand name of the video conference system (eg, “Tandberg” (Tandberg Device Library) can be displayed on the screen, for example, just below “Video Conference” 1010.

  Below the “video conference” 1010 is a list of conference attendees convened by the user. This list of attendees is linked to the user identification number in the USE. A phone number is stored with each attendee's name. In embodiments, other types of contacts can be used instead of or in conjunction with a telephone number. The attendee's name may be the attendee's whereabouts (such as a conference room or other room). The user can enter the attendee information into the USE before the time to start the meeting in the USE UI. Alternatively, a user can be given system permission to enter prospective attendee information on behalf of another user. For example, the assistant can enter the contact information of the attendees that the executive has called before the time when the executive uses the USE UI to start the meeting.

  In FIG. 10, an exemplary list of attendees has three room names and one attendee name (“Nakai SCR Denmark”, “Takiguchi Koichi”, “Administrator Meeting”, and “Spider”. conference room"). The user selects an attendee by selecting the corresponding button. For example, the user can select one of three rooms ("Nakai SCR Denmark" button 1035, "Administrator Meeting") button 1045, and "Spider Meeting Room" button 1050). In these rooms, one or more attendees can gather to participate in the conference. The user can also select the “Takiguchi Kouichi” button 1040 of the attendee. For example, Mr. Takiguchi, who is scheduled to attend, is considered to be at his own phone number (eg, office). Other attendees can also join the meeting with Mr. Takiguchi at Mr. Takiguchi's location. The user can use the left arrow button 1060 and the right arrow button 1065 to page through the list of attendees. Shown between the two arrow buttons 1060 and 1065 is the current page of the total number of pages of attendees. The page indicator 1063 indicates that the user is displaying the second page (that is, “page 2/3”) of the attendees for three pages.

  If the user wants to see other contacts to be called for the meeting, he may select the “Browse” button 1075, which will display another screen (not shown). The On this screen, there is a list of user contacts for making a selection to convene at the conference, and the user can also enter a phone number.

  Microphone icon 1068 indicates whether audio for all other conference attendees is on or off. If the microphone is on, a microphone icon 1068 is displayed as shown in FIG. If the microphone is off, the microphone icon 1068 is displayed as if the microphone is surrounded by a circle and a slash is applied to the circle. The mute button 1070 allows the user to mute and unmute the microphone at any time.

  The “left screen” 905, application buttons 915, 920, 925, and “off” button 930 in FIG. 10 are the same as those in FIG. When the user selects the “option” button 980, the USE displays a main menu of available applications displayed on the left half of the screen of FIG. These are applications that can be used for the left display screen 710 of FIG. When the user selects the “option” button 985, the USE displays a main menu of available applications displayed on the right half of the screen of FIG. These are applications that can be used for the right display screen 720 of FIG. Further, the user can end the conference at any time by selecting the “end conference” button 990.

  FIG. 11 is a UI screen shot showing an exemplary list of presentations of the user for the exemplary presentation selected by the user in FIG. 10 according to an embodiment. In this example, the user selects the “presentation” button 915 from the “left screen” 905 in FIG. 10, and the UI screen shot in FIG. 11 is displayed. The left half of the screen in FIG. 11 is now labeled “Presentation” 1105.

  Under “Presentations” 1105 is a list of the user's presentations that the user can reference during the video conference. This list of presentations is linked to the user's identification number in the USE. These presentations may be PowerPoint presentations or other types of presentations. The user can enter the file names of these presentations into the USE before the time to start the meeting with the USE UI. Alternatively, a user can be given system permission to enter a presentation file name on behalf of another user. For example, the executive assistant can enter the presentation file name of the executive before the time when the executive starts the meeting using the USE UI.

In FIG. 11, an exemplary list of presentations is “Agenda Schedule”, “Enterprise Content Services”, “ECS / OOM Resolution”, “ECM, BCS, and SES: TLA's for CM”, and “USE Design Presentation”. It is. The user selects a presentation by selecting the corresponding button. For example, the user may select “Agenda Schedule” button 1115, “Enterprise Content Services” button 1120, “ECS / OOM Resolution” button 1125, “ECM, BCS, and SES: TLA's for CM” button 1130, and “USE Design”. One of the “Presentation” buttons 1135 can be selected. Selecting the "Email" button 1140 displays another screen (not shown) where the user can call and read the user's email as part of the meeting. The user can use the left arrow button 1150 and the right arrow button 1155 to turn the pages of the presentation list. Shown between the two arrow buttons 1150 and 1155 is the current page of the total number of pages in the presentation. This example shows that the user is displaying the first page (that is, “page 1/2”) of the two-page presentation. The left arrow button 1150 being displayed in a light color indicates that the user cannot select a page before the first page. Similarly, if the user is displaying the last page of the presentation, the right arrow button 1155 is dimmed, indicating that the user cannot select a page after the last page.

  If the user wants to refer to other presentations to be included in the meeting, in some cases, the user can select a “Browse” button 1145, thereby displaying another screen (not shown). This screen has an additional list of user presentations for selection to include in the conference. This screen also allows the user to browse the file names of presentations that are not yet loaded into the USE but are accessible from the USE system (eg, presentations in a network file system).

  The “video conference” 1010 (attendee buttons 1035, 1040, 1045, and 1050, page arrow buttons 1060 and 1065, page indicator 1063, microphone icon 1068, microphone button 1070, and “reference” button 1075 in the half of the screen of FIG. Is the same as FIG. When the user selects the “option” button 980, the USE displays a main menu of available applications displayed on the left half of the screen of FIG. These are applications that can be used for the left display screen 710 of FIG. When the user selects the “option” button 985, the USE displays a main menu of available applications displayed on the right half of the screen of FIG. These are applications that can be used for the right display screen 720 of FIG. Further, the user can end the conference at any time by selecting the “end conference” button 990.

  FIG. 12 is a UI screen shot showing an example presentation control for the example presentation selected by the user in FIG. 11, according to an embodiment. In this example, the user selects a “USE Design Presentation” button 1135 from the left side of the screen below “Presentation” 1105 in FIG. 11, and the UI screen shot in FIG. 12 is displayed. The left half of the screen of FIG. 12 is now labeled “Presentation” 1205. The brand name of the presentation (eg, “PowerPoint”) can be displayed on the screen, eg, just below “Presentation” 1205. In this example, “USE Design Presentation” is a PowerPoint presentation, and “PowerPoint” is displayed immediately below “Presentation” 1205 on the screen.

  Below “Presentation” 1205 is an area labeled “USE Design Presentation” 1250. The user currently operating the conference, as well as the date of the conference, can be displayed on the screen (eg, “John Doe” and “01/29/2007” 1270). The user's “USE Design Presentation” already loaded into the USE system as described above is displayed on the left display screen 710 shown in FIG. John Dow can use the left arrow button 1255 and the right arrow button 1260 to turn the slide page of the presentation while referring to the presentation display on the left display screen 710 of FIG. Shown between the two arrow buttons 1255 and 1260 is the current slide of the total number of slides in the presentation. By default, the presentation begins with the display of the first presentation slide. This exemplary slide indicator 1258 indicates that John is displaying the first slide (ie, “slide 1/9”) of the nine slides in the presentation. The fact that the left arrow button 1255 is displayed in light color indicates that there is no slide selected by John before the first slide. Similarly, if John is displaying the last slide of the presentation, the right arrow button 1260 is dimmed, indicating that there is no slide that John selects after the last slide.

  If John Dow wants to switch to another presentation, John can select a “List” button 1265 which, when selected, displays another screen (not shown) similar to FIG. Displays a list of John's presentations other than “USE Design Presentation”, ie, presentations 1115, 1120, 1125, 1130, and “email” 1140. John can select one or more presentations for use in the conference. John can also retrieve the file names of presentations that are not yet loaded into the USE but are accessible from the USE system (eg, presentations in a network file system). In an embodiment, the system remembers where the user was viewing each presentation. John can toggle between multiple open presentations by selecting the “List” button 1265 to display a list of presentations. In an embodiment, a toggle button (not shown) may be displayed on the screen to allow John to toggle between open presentations.

  “Video conference” 1010 (attendee buttons 1035, 1040, 1045, and 1050, page arrow buttons 1060 and 1065, page indicator 1063, microphone icon 1068, microphone button 1070, and “reference” button 1075 in the half of the screen of FIG. Is the same as FIG. When the user selects the “option” button 980, the USE displays a main menu of available applications displayed on the left half of the screen of FIG. These are applications that can be used for the left display screen 710 of FIG. When the user selects the “option” button 985, the USE displays a main menu of available applications displayed on the right half of the screen of FIG. These are applications that can be used for the right display screen 720 of FIG. Further, the user can end the conference at any time by selecting the “end conference” button 990.

  FIG. 13 is a UI screen shot illustrating an exemplary video conference control for the exemplary conference attendee selected by the user in FIG. 12, according to an embodiment. In this example, User John Dow selects the “Takiguchi Koichi” button 1040 from the right side of the screen below “Video Conference” 1010 in FIG. A sequence of events is initiated and the teleconferencing system is requested to call Mr. Takiguchi via the teleconferencing system. When John establishes contact with Mr. Takiguchi, Mr. Takiguchi agrees to attend the video conference, and the UI screen shot of FIG. 13 is displayed. It is also possible to indicate that Mr. Takiguchi is one of the current attendees participating in the video conference by displaying a photograph 1371 of Mr. Takiguchi next to the “Takiguchi Kouichi” button 1040 on the UI screen. . John Doe can select as many attendees as he wants to attend the meeting. John uses the arrow buttons 1060 and 1065 to scroll through the attendee's pages, the “Browse” button 1075 to browse other attendees, and the attendee / room buttons (eg, 1035, 1045, 1050). You can easily contact any attendee on your list by making a selection. In this example, the microphone remains unmuted, as indicated by the microphone icon 1068. John can mute the microphone at any time by selecting the microphone button 1070.

  If only Mr. Takiguchi is participating in a video conference from a remote location, Mr. Takiguchi's real-time video is displayed on the right display screen of FIG. If two or more people are participating from a remote location, the right display screen 720 of FIG. 7 is divided into tiles to display real-time video of everyone participating from the remote location. If the “Attendees” from a remote location are people in a room, such as the “Spider Meeting Room” selection 1050 in FIG. 720 is displayed. It is also possible to display the user's picture on the screen as a participant in the video conference. Displaying these videos on the right display screen 720 of FIG. 7 is handled by video conferencing software (such as Tandberg) preloaded in the USE.

  If John Doe decides that Mr. Takiguchi will leave the meeting before the meeting ends, John can terminate the telephone connection with Mr. Takiguchi by selecting the “End Call” button 1372. . This “end call” button 1372 is more likely to be used when two or more attendees are participating in the conference. If the attendee decides to leave the video conference by hanging up, the attendee's photo 1371 and the corresponding “End Call” button 1372 disappear from the screen.

  The “presentation” 1205 (presentation area labeled “USE Design Presentation” 1250, slide arrow buttons 1255 and 1260, slide indicator 1258, “list” button 1265, and user and date (“ "John Doe" and "01/29/2007") 1270) is the same as FIG. The only difference in “Presentation” 1205 in the half of the screen of FIG. 13 is that John selected the right arrow button to advance the presentation to Slide 2, and the display of slide indicator 1258 is “Slide 2/9”. " The left slide arrow button 1255 in FIG. 13 is no longer displayed in light color.

  When the user selects the “option” button 980, the USE displays a main menu of available applications displayed on the left half of the screen of FIG. These are applications that can be used for the left display screen 710 of FIG. When the user selects the “option” button 985, the USE displays a main menu of available applications displayed on the right half of the screen of FIG. These are applications that can be used for the right display screen 720 of FIG. Further, the user can end the conference at any time by selecting the “end conference” button 990.

  14 is a UI screen shot showing an exemplary video conference control and display for the video conference setup of FIG. 13, according to an embodiment. User John Doe has established contact with all attendees at John's video conference. John's meeting includes only John, everyone in the meeting room with John, and Mr. Koichi Takiguchi. In this example, John selects the microphone button 1070 to mute the audio to a conference attendee at a remote location. This allows John to have an offline discussion with, for example, an attendee in the conference room with John. At this time, the microphone icon 1068 is displayed as if the periphery of the microphone is circled and a slash is applied to the circle. The button 1070 is displayed as “unmute” at this point. The region 1405 can be displayed in a different color (such as red) to make the user aware that the microphone is muted. In addition, a list of attendee locations can be displayed on the screen. In this example, “Takiguchi Koichi” is displayed as being in “Yuki Conference Room” 1400.

  Other elements of “video conference” 1010 in the half of the screen of FIG. 14 (Attendees buttons 1035, 1040, 1045, and 1050, page arrow buttons 1060 and 1065, page indicator 1063, microphone icon 1068, microphone button 1070, and “ The “reference” button 1075) is the same as FIG. When the user selects the “option” button 980, the USE displays a main menu of available applications displayed on the left half of the screen of FIG. These are applications that can be used for the left display screen 710 of FIG. When the user selects the “option” button 985, the USE displays a main menu of available applications displayed on the right half of the screen of FIG. These are applications that can be used for the right display screen 720 of FIG. Further, the user can end the conference at any time by selecting the “end conference” button 990.

  FIG. 15 is a UI screen shot illustrating an example of a whiteboard selected by a user according to an embodiment. In this example, when the user selects the “option” button 980 in FIG. 14, the UI screen shot shown in FIG. 10 is displayed. The user selects the “whiteboard” button 520 from the “left screen” 905 in FIG. 10, and the UI screen shot in FIG. 15 is displayed. The left half of the screen of FIG. 15 is now labeled “Whiteboard” 1500. The whiteboard brand name (e.g., SmartBoard) can be displayed on the screen, e.g., just below "Whiteboard" 110.

  Below the “whiteboard” 1500 is an area labeled “whiteboard” 1501. In this area, a marker / pen image 152 can be displayed to indicate to the user that the whiteboard is in use. The user currently operating the conference, as well as the date of the conference, can be displayed on the screen (eg, “John Doe” and “01/29/2007” 1507). Whiteboard software preloaded in the USE system displays the electronic whiteboard on the left display screen 310 shown in FIG. Whiteboard software allows John to begin drawing on a blank whiteboard. The whiteboard software also allows John to draw a note from a saved whiteboard session and draw it on it. In the embodiment, the drawing tool of the whiteboard software is displayed on the whiteboard screen instead of the console UI.

  In this example, John is pulling out a previously stored note of 12 pages. Immediately after John selects “Whiteboard” 920 in FIG. 10 and immediately before FIG. 15 is displayed, a screen of a usable set of whiteboard notes that John can select (not shown) ) Is displayed. This screen is very similar to the presentation list of FIG. 11 except that a list of whiteboard memo collections is displayed. Assume that John selects a set of memos for 12 pages from this screen. Then, the screen shot of FIG. 15 is displayed.

  John can turn the note page using the left arrow button 1503 and the right arrow button 114 while referring to the whiteboard display on the left display screen 710 of FIG. Displayed between the two arrow buttons 1503 and 1504 is the current note of the total number of notes. By default, the whiteboard software displays the first page of notes. Exemplary memo indicator 1505 indicates that John is displaying the third page of the 12 pages of memos (ie, “Memo 3/12”).

  If John Dow wants to switch to another set of whiteboard notes, John can select a “reference” board 1506, which allows another list with another saved set of notes to be stored. A screen (not shown) is displayed. John can select a set of one or more notes for use in the conference. John can also retrieve the file name of a memo (eg, a memo in a network file system) that has not yet been loaded into the USE but is accessible from the USE system. In an embodiment, the system remembers where the user was in each set of whiteboard notes. John can toggle between sets of open whiteboard notes by selecting a “reference” button 1506 to display a list of sets of notes. In an embodiment, a toggle button (not shown) may be displayed on the screen to allow John to toggle between the set of open notes.

  In an embodiment, the system remembers where the user was displaying each presentation and each set of whiteboard notes. When the user selects the “option” button 980, the USE displays a main menu of available applications displayed on the left half of the screen of FIG. These are applications that can be used for the left display screen 710 of FIG. In an embodiment, if the user selects “Presentation” 915 and then selects “Use Design Presentation” 1135 shown in FIG. 11, the user in the presentation, as shown in FIG. The displayed location is displayed. Alternatively, a button (not shown) can be provided on the screen of FIG. 15 that can display a menu screen that displays menu items for the presentation and whiteboard memo collection currently in use. The user can select one of these menu items, and the system directly selects the location that the user was viewing in the presentation or note (eg, the presentation shown in FIG. 14). It is possible to display.

  “Video conference” 1010 (attendee buttons 1035, 1040, 1045, and 1050, page arrow buttons 1060 and 1065, page indicator 1063, microphone icon 1068, microphone button 1070, and “reference” button 1075 in the half of the screen of FIG. Is the same as FIG. When the user selects the “option” button 985, the USE displays a main menu of available applications displayed on the right half of the screen of FIG. These are applications that can be used for the right display screen 720 of FIG. Further, the user can end the conference at any time by selecting the “end conference” button 990.

  FIG. 16 is a UI screen shot illustrating an example of a laptop selected by a user according to an embodiment. In this example, when the user selects the “option” button 980 in FIG. 11, the UI screen shot shown in FIG. 10 is displayed. The user selects the “external laptop” button 525 from the “left screen” 905 in FIG. 10, and the UI screen shot in FIG. 16 is displayed. The left half of the screen of FIG. 16 is now labeled “Laptop” 1600.

  Below the “laptop” 1600 is an area labeled “laptop” 1601. Within this area is a laptop image 1602 that can be displayed to indicate to the user that the display of the laptop is displayed on the left display screen 710 of FIG. The user can choose from several laptops. In this example, the user can select any of the four laptops by selecting any of buttons 1603, 1604, 1605, and 1606. In this example, the laptop corresponding to the button 1604 is selected. The display of the laptop, as displayed on the laptop, is displayed on the left display screen 710 of FIG. It is assumed that the user is using this laptop and wants to show items on the laptop to other people participating in the video conference. The user currently operating the conference and the date of the conference can also be displayed on the screen of FIG.

  If John Dow wants to switch to a different laptop than the laptop corresponding to button 1604, John can select any of the other buttons 1603, 1605, and 1606. In an embodiment, the system remembers where on the laptop each user was viewing. John can toggle between the laptops he is using by selecting any of the buttons 1603, 1604, 1605, and 1606. In an embodiment, a toggle button (not shown) may be displayed on the screen to allow John to toggle between laptops in use.

  In an embodiment, the system remembers where the user was viewing each presentation, each set of whiteboard notes, and each laptop session. When the user selects the “option” button 980, the USE displays a main menu of available applications displayed on the left half of the screen of FIG. These are applications that can be used for the left display screen 310 of FIG. In an embodiment, if the user selects “Presentation” 915 and then selects “Use Design Presentation” 1135 shown in FIG. 11, the user in the presentation, as shown in FIG. The displayed location is displayed. Alternatively, the screen of FIG. 15 can be provided with a button (not shown) that can display a menu screen that displays menu items for the presentation, whiteboard note set, and laptop session currently in use. is there. The user can select one of these menu items and the system displays the user in a presentation, note, or laptop session (eg, the presentation shown in FIG. 14). It is possible to directly display the place where it was.

  The “video conference” 1010 (attendee buttons 1035, 1040, 1045, and 1050, page arrow buttons 1060 and 1065, page indicator 1063, microphone icon 1068, microphone button 1070, and “reference” button 1075 in the half of the screen of FIG. Is the same as FIG. When the user selects the “option” button 985, the USE displays a main menu of available applications displayed on the right half of the screen of FIG. These are applications that can be used for the right display screen 720 of FIG. Further, the user can end the conference at any time by selecting the “end conference” button 990.

  FIG. 17 illustrates an exemplary UI screen shot and a corresponding exemplary photo of a conference room display screen, according to an embodiment. These UI screen shots are similar to those shown in FIGS. 9, 11 and 13, but are not exactly the same. These corresponding exemplary photographs are similar to the photographs of FIGS. 6 and 7, but not exactly the same.

  Screen shot 10 is the same as the screen shot of FIG. The “left screen” 14 controls the right display screen 24 of the conference room 20. “Right screen” 16 controls the right display screen 26 of the conference room shown in photo 20. Below the “left screen” 14 is a list of available applications that the user 22 can select. Similarly, below the “right screen” 16 is a list of available applications that the user 22 can select. At this point, since the user 22 has not yet selected an application, the screen 24 is shown to indicate that “USE is active, but the application displayed on these screens has not yet been selected”. And 26 can display the USE logo.

  The screen shot 30 is the same as the screen shot of FIG. In the screen shot 10, the user 22 selects “presentation” 18 from the “left screen” 14 and selects “video conference” 19 from the “right screen” 16. A list of user presentations is displayed on the “presentation” 34 side of the screen shot 30. A list of attendees of the user is displayed on the “video conference” 36 side of the screen shot. If the user 22 selects the presentation “View of the Future—Beyond Web 2.0” button 38, slides from this presentation are displayed on the left display screen 24 of the meeting room for the photos 40. At this point, since the user 22 has not yet selected the attendee of the video conference, the USE logo is still displayed on the right screen 26 of the meeting room of the photo 40.

  The screen shot 50 is the same as the screen shot of FIG. In the screen shot 30, the user 22 selects the presentation “View of the Future—Beyond Web 2.0” button 38 from the “presentation” 34 side of the screen shot, and the attendee “ “Sasuga Kazuya” 39 was selected. On the “presentation” 54 side of the screen shot 50, an area 58 for controlling the slide of the presentation “View of the Future—Beyond Web 2.0” is displayed. The slide 2 of this presentation is displayed on the left display screen 24 of the meeting room of the photograph 60. On the “video conference” 56 side of the screen shot 50, “Sasuga Kazuya” 59 is highlighted, indicating that Mr. Sasuga is participating in the video conference. A real-time image of Mr. Sasuga is displayed on the right display screen 26 of the meeting room of the photo 60.

  18 illustrates various button states of the buttons illustrated in FIGS. 8-17, according to an embodiment. An “active state” button 1801 indicates that the user can select this button. If the USE is being used with a mouse on a laptop or other device, the “Stand Up” button 1802 allows the user to move the mouse over this button, where the user still clicks the button with the mouse. Indicates that it is not. Similarly, when the user is moving the mouse, a “click state” button 1803 indicates that the user clicked this button with the mouse to select the button. The “active state” button 1804 allows the USE to be processing in the background, allows the user to wait until processing is complete, and allows the user to select any other button until processing is complete Indicates that it will not be. When the USE UI is used with a touch screen, a “Selected” button 1805 indicates that the user has selected this button by touching it on the touch screen. The “selected state” button 1805 is the same as the “clicked state” button 1803 except that the user touches the button instead of clicking the button with the mouse. An “unusable state” button 1806 indicates that the user cannot select this button. For example, in FIG. 16, a “return” triangle mark under “presentation” is shown as “unusable”. This is because page 1 of the presentation for two pages is displayed and there is no previous page. A “temporarily unavailable state” button 1807 indicates that the user cannot temporarily select this button. For example, if the application of FIG. 9 is upgraded to a new version, it can be temporarily disabled.

  The buttons shown are circular or triangular in shape for ease of use. In addition, the buttons are shown in green, red, and yellow primaries, again for ease of use. In an embodiment, any button may be any shape, size, and color. In an embodiment, one or more of the buttons may be any type of controller (such as a slider) instead of a button.

<IX. Interaction>
By using the settings in the system, much of the interaction with the system for control purposes can be done with only a small number of simple selection operations or completely unnecessary.

  One class of interaction logs in to the system under authentication and access control 140 control. The presenter can log into the system using the badge by holding or passing the badge over a FeliCa (RFID) reader 375. In addition, the presenter can log in to the system by using the fingerprint reader 380 or other biometric reader 385, or by typing the user ID and password into the trusted application. is there.

  A single pass of the thumb or RFID opens a document or electronic whiteboard application on a screen such as the first smart board 330 or a video conference call on another screen such as a second smart board 335 (Call) can be started. In each example, the presenter concentrates on tasks such as sketching, presenting, or communicating rather than understanding device control procedures and which remote control to use. .

  However, the login operation is not strictly necessary. For example, picking up a pen and drawing it on an electronic whiteboard automatically bootstraps the correct application and creates a context. In this context, it is possible to permanently save the results of drawing on the whiteboard. Subsequently, if the presenter wants to save the data created in this ad hoc context, the presenter can log in and have this data transferred to the presenter's own personal space.

  Once the presenter logs in, the person need not use the buttons on the remote control or various devices in the system to control the remote control or room. Most device states are set automatically. For example, in order to set up a video conference, there are some cases where some input is required, such as which telephone number or IP address to select and dial. Such an input can be performed once selected on a touch screen type console used for controlling the room. The system automatically switches the projector 340 or projector 345 to video mode, determines the appropriate way to initiate the connection, and connects to the remote site. This avoids one of the pitfalls that there is an unorganized space in which some devices are left in an unknown state. In an unorganized space, it becomes difficult to understand what needs to be done to accomplish a desired task.

<X. System hardware, software, and components>
Embodiments of the present invention include conventional general purpose computers, special purpose digital computers, computerized methods and systems, programmed according to the teachings of the present disclosure, that can be implemented using a microprocessor. It is possible. Based on the teachings of the present disclosure, a programmer can easily create appropriate software coding. Embodiments of the invention can include a program of instructions executable by a computer to perform any of the features presented herein.

  Embodiments of the invention can include computer readable media, such as computer readable storage media. The computer readable storage medium can store instructions that can be used to program the computer to perform any of the features presented herein. This storage medium includes, but is not limited to, any type of disk such as a floppy disk, optical disk, DVD, CD-ROM, microdrive, and magneto-optical disk, ROM, RAM, EPROM, EEPROM, DRAM, It can include flash memory and any medium or device suitable for storing instructions and / or data.

  The present invention controls both general purpose / special purpose computers or computer hardware such as microprocessors and uses the results of the present invention to allow them to interact with human users and other mechanisms. It is possible to include software that enables it. Such software can include, but is not limited to, device drivers, operating systems, execution environments / containers, user interfaces, and user applications.

  Embodiments of the present invention can include providing code that implements the process of the present invention. The providing method can include providing the code to the user in some manner. For example, the providing method may include sending a digital signal containing the code to the user, providing the code on a physical medium to the user, or any other method that makes the code available. It is possible to include.

  Embodiments of the present invention can include a computer-implemented method for transmitting computer-executable code to perform any of the processes of the embodiments of the present invention. This method of transmission involves transfer over a part of a network such as the Internet, transfer over a wire, transfer over the atmosphere or space, or transfer via any other type of transmission. It is possible to include. This transmission method includes a method for initiating transmission of a code or a method for causing a code to be sent from one region or country to another region or country. The method of transmitting to the user may include a method in which the user in the region or country receives any transmission regardless of where the transmission is sent from.

  Embodiments of the present invention can include a signal that includes a code that can be executed by a computer to perform any of the processes of the embodiments of the present invention. This signal can be transmitted over a network such as the Internet, transmitted over a wire, transmitted over the atmosphere or space, or any other type of transmission. The entire signal need not be transmitted simultaneously. This signal can be extended in time during the transfer. This signal should not be considered a snapshot that is currently being transmitted.

  The foregoing description of the preferred embodiment of the present invention is for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations will be apparent to practitioners skilled in this related art. For example, the steps performed in the embodiments of the present invention disclosed herein can be performed in a different order, a certain step can be omitted, or additional steps can be added. It is to be understood that other embodiments of the invention can be developed and included within the spirit and scope of the invention and the claims. These embodiments have been chosen and described in order to best explain the principles of the invention and its practical application, so that other persons skilled in the relevant art will envision various embodiments of the invention. It will be possible to understand various modifications suitable for a particular application. It is intended that the scope of the invention be defined by the appended claims and their equivalents.

  The entire disclosure of US Provisional Patent Application No. 60 / 887,115, filed January 29, 2007, including the specification, claims, drawings and abstract, is hereby incorporated by reference in its entirety. It is.

It is a figure which shows an example of the software architecture of the system based on embodiment of this invention illustrated as a stack structure of each component. FIG. 2 is a diagram illustrating a schema of an object stored in a database according to an embodiment of the present invention, and exemplarily showing the generic object of FIG. 1. FIG. 2 is a diagram exemplarily showing system logic components corresponding to the software components in FIG. 1 and their connections according to an embodiment of the present invention. It is a figure which shows an example of the logical structure of the setting of a meeting in the time of the start of a meeting in connection with embodiment of this invention. FIG. 7 illustrates that a user interface can be adapted from a goal-oriented interface to a process-oriented interface, according to an embodiment. 2 is a photograph showing an exemplary conference room with two display screens, according to an embodiment. Exemplary Meeting Room with Usable Smart Environment (“USE”) User Interface (“UI”) Available on Exemplary Touch Screen Tablet PC Used with Two Exemplary Display Screens According to the Present Invention It is a photograph which shows. 5 is a UI screen shot showing an exemplary start screen, according to an embodiment. FIG. 8 is a UI screen shot showing an exemplary main menu of an application operable for the exemplary two screens of the conference room shown in FIG. 7, according to an embodiment. FIG. 10 is a UI screen shot showing an exemplary list of video conference attendees for the user for the exemplary video conference selected by the user in FIG. 9 according to an embodiment. FIG. 11 is a UI screen shot showing an exemplary list of presentations of the user for the exemplary presentation selected by the user in FIG. 10 according to an embodiment. FIG. 12 is a UI screen shot illustrating an example presentation control for an example presentation selected by a user in FIG. 11, according to an embodiment. FIG. 13 is a UI screen shot showing an exemplary video conference control for an exemplary conference attendee selected by a user in FIG. 12, according to an embodiment. FIG. 14 is a UI screen shot showing an exemplary video conference control and display for the video conference setup of FIG. 13, according to an embodiment. 6 is a UI screen shot illustrating an example of a whiteboard selected by a user according to an embodiment. 4 is a UI screen shot illustrating an example of a laptop selected by a user, according to an embodiment. FIG. 4 illustrates an exemplary UI screen shot and a corresponding exemplary photograph of a conference room display screen, according to an embodiment. FIG. 18 is a diagram illustrating various button states of the buttons illustrated in FIGS. 8-17 according to an embodiment.

Explanation of symbols

100 system software 102 box 105 database 110 database extraction layer 115 persistence interface 120 generic object 125 business object 130 communication / presentation device and application 135 communication / presentation device wrapper 140 authentication and access control 145 authentication device 147 security 150 Authentication Device Wrapper 155 AA Persistence Interface 160 Application Logic 132, 149, 165 UI
210 Meeting (object or table)
220 rooms (objects or tables)
230 Settings (object or table)
240 Creator (object or table)
250 Device status (object or table)
260 devices (objects or tables)
270 attribute (object or table)
300 System Hardware 310 Setting Repository 315 Room Control PC
320 Display PC1
325 PC2 for display
330 Smart Board 1
335 Smart board 2
340 Projector 1
345 Projector 2
350 Video switch (device)
355 Guest Laptop 360 Computer 365 Console Tablet 370 PDA
375 Felica (RFID) Reader 380 Fingerprint Reader 385 Biometric Reader 390 P Box 395 Video Conference Device 405 Conference 410 Room 415, 420 Creator 425, 430 Settings 435, 440, 445 Device 450, 455, 460, 465, 470, 475, 480, 485 Device state 610, 620, 710, 720 Display screen 630, 640 Projector 705 Touch screen Tablet PC

Claims (21)

A conference task support method using a computer system that adjusts the operation of a hardware device and a software application to support a specific task such as holding a conference,
The method comprises
Providing one or more computers connected to a network;
Providing at least one setting storage unit provided in any one of the one or more computers, and providing at least one room control unit provided in any of the one or more computers. ,
Provide one or more devices and applications for each room controller,
Enabling one or more presenters to hold a conference, and each conference setting includes device and application settings by one or more presenters to accommodate multiple presenters, wherein the conference settings include: Stored by the settings storage unit, wherein each presenter setting includes a subset of the one or more devices and applications;
Adjusting the operation of the device and application in the conference by the room controller based on settings of the one or more presenters for the conference;
Meeting task support method including   Adjusting the operation of the device and the application includes performing the adjustment autonomously by the room control unit, and executing the adjustment based on a setting made through an interface by the presenter. The meeting task support method according to claim 1, comprising one of them.   To adjust the operation of the device and the application, the presenter can adjust the presentation when the presentation is performed simultaneously, the adjustment when the presentation is performed continuously, and the adjustment when the presentation is performed by combining simultaneous and continuous presentation. The conference task support method according to claim 1, comprising:   Both allowing one or more presenters to set up a conference and coordinating device and application behavior first authenticate one of the one or more presenters with an authentication device. The meeting task support method according to claim 1, further comprising:   The conference task support method of claim 1, further comprising storing conference information in one or more databases including conference, room, presenter, settings, device, device status, and attribute information. The one or more databases include one of existing commercial databases, custom databases and open source software;
The application includes existing commercial software,
The conference task support method according to claim 5, wherein the device includes one or more of a projector, an electronic whiteboard, a video switch, an audio switch, a video conference device, lighting, an audio device, and an air conditioner.   6. The conference task support method according to claim 5, further comprising making the system independent of database selection by using a database extraction layer. A conference task support system that coordinates operations of hardware devices and software applications to support a specific task such as holding a conference, the system comprising:
One or more computers connected by a network;
At least one setting storage unit provided in any one of the one or more computers;
At least one room controller provided in any of the one or more computers;
One or more devices and applications corresponding to each room controller;
With
The conference is a conference held by one or more presenters, and the conference settings include device and application settings by one or more presenters to accommodate a plurality of presenters, and the conference settings include the conference settings Stored by a settings storage unit, wherein each presenter setting includes a subset of the one or more devices and applications;
Coordination of the device and application operations is performed by the room controller based on settings of the one or more presenters for the conference,
A conference task support system characterized by that.   Claims further comprising a software wrapper that separates the device and application specific differences, such that additional devices and applications are further added to the system while the system is running. Item 9. The meeting task support system according to Item 8.   The conference according to claim 8, wherein the adjustment of the operation of the device and the application includes one of an autonomous adjustment by the room control unit and an adjustment based on a setting made through an interface by the presenter. Task support system.   The adjustment of the operation of the device and application is one of adjustments for simultaneous presentations, adjustments for continuous presentations, and adjustments for simultaneous and continuous presentations by the presenters. The conference task support system according to claim 8, comprising:   An authentication device for authenticating the presenter is further provided, and at least one of the one or more presenters is authenticated by the authentication device in both the conference set by the one or more presenters and the adjustment of the operation of the device and the application. The conference task support system according to claim 8, which is obtained.   9. The conference task support system according to claim 8, further comprising a database for storing conference information including conference, room, presenter, setting, device, device status, and attribute information. The database includes one of an existing commercial database, a custom database, and open source software;
The application includes existing commercial software,
The device comprises one or more of a projector, an electronic whiteboard, a video switch, an audio switch, and a video conferencing device;
The conference task support system according to claim 13.   14. The conference task support system according to claim 13, further comprising a database extraction layer that recognizes a plurality of different databases and provides a common interface so that the system does not depend on database selection. A program for causing a computer to execute a function for supporting a specific task such as holding a meeting by adjusting the operation of a hardware device and a software application,
Providing one or more computers connected to a network;
Providing at least one setting storage provided in any of the one or more computers;
Providing at least one room controller provided in any of the one or more computers;
Providing one or more devices and applications corresponding to each room controller;
Enabling the creation of a conference for one or more presenters, wherein each conference setting includes a device and application configuration by one or more presenters to accommodate multiple presenters, the conference A setting storage unit, wherein each presenter configuration includes a subset of the one or more devices and applications;
Adjusting the operation of the device and application in the conference by the room controller based on the configuration of the one or more presenters for the conference;
A program including executing the program. A user interface device connected to a computer for operating one or more physical devices used for conference room setup,
a) a touch screen that presents the user with various options;
b) a plurality of touch screen controllers that are displayable to the user to allow the user to select any of the options;
Each of the touch screen controllers has target-oriented information that allows the user to select a target, and controls the physical equipment that is required to execute the target based on the selection at the controller It is characterized by executing the process of
User interface device. The goal is at least one control of video conferencing, presentation, whiteboard, and external laptop.
The user interface device according to claim 17. The goals include selecting a video conference attendee from a list of attendees, selecting a presentation from a list of presentations, selecting a set of notes from a list of whiteboard note sets, and selecting a laptop from a list of laptops. Any one of
The user interface device according to claim 17. The goal is any one of the functions of turning pages of lists, presentations, and whiteboard notes.
The user interface device according to claim 19. A program for causing a computer to realize a user interface on a touch screen for operating one or more physical devices used for conference room setting,
Computer
a) the ability to present the user with various options having goal-oriented information that allows the user to select a goal on the touch screen;
b) a function of executing a process for controlling the physical device required to execute a target based on the selection of any of the options by the user;
Computer program for realizing.