JP2002232849A - Interface device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an interface device where time is assigned to an object itself and time is designated by designating an object by a user. SOLUTION: A picture input part 1, an object deciding part 2 and an application 3 are incorporated in a desk 5. A video camera photographing work performed on the desk 5 is equivalent to the picture input part 1. A PC is the object deciding part 2 and is the application 3 is performed. Numeral 4 is a mirror. When a worker works on the desk, physical objects are treated almost in the flat world on the desk. A computer picture projected by a display exists in the almost vertical world with respect to a plane on the desk. The computer picture is projected on the desk and it is considered to have the same dimensions as those of the real world. Thus, the coming and going of the real world and the virtual world become continuous.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an interface device, and more particularly, to an interface device in which time is assigned to an object itself, and when a user designates the object, a time corresponding to the object is designated.

[0002]

2. Description of the Related Art The "computer program generation device and computer program generation method" described in JP-A-11-191061 is a technique for generating a computer program by arranging and operating an object in a physical space. When this conventional technique is used when editing video data and music data, the following is achieved.

When handling video data and music data, it is necessary to quickly and repeatedly perform operations requiring high accuracy, such as specifying a unit of 1/30 second from a time width of several hundred hours or more. is there. In such work, the method described in the above-mentioned gazette, the limit of the size of the physical space to be handled, caused by human work, poor positional accuracy when arranging the object, operation when operating the object Due to poor accuracy, the accuracy required for the work is not satisfied. In addition, if the limitation of the size of the physical space and the inaccuracy of the position and operation accuracy of the human can be avoided by changing the time unit indicated by the position of the object in the physical space, the time unit is changed. Useless work increases by the work. From the above points, it can be said that the conventional method has a significant problem when performing an operation using video data or music data.

[0004]

At present, in the field of multimedia data editing, DTV / DTM (DTV:
Video editing software / animation creation software, DT
M: sequence software / sampling software) Software has become an indispensable tool for computers. Until now, the creation and editing of various types of media required specialized machines, but with the improved specifications of the computing unit, it is now possible to perform integrated editing with DTV / DTM software on a single machine Became. The advent of DTV DTM software has provided a much lower cost means than before. However, while the DTM / DTV software released one after another responded to the high demands of professional use, its functions became more complicated. While this satisfies professional use, it has required a great deal of effort, time and motivation for users.

On the other hand, up to now, DTV / DT
Due to the low cost of M software, the base has expanded to entry (amateur) users. With the increasing functionality year by year, the handling threshold for entry users is high, and those advocating for beginners have come to appear. However, since the degree of difficulty is reduced by reducing the number of functions, the degree of freedom of data operation is lower than that of a specialized application. Although the entry user is not as good as the professional user, due to the demand for functions,
Combined with the growing competition in the entry market due to the lower prices of PCs, DTM / DTM application manufacturers are compelled to respond to conflicting requirements for "function and difficulty."

[0006] While there has been such a movement in the DTV / DTM application market, a technique for easily operating software by operating a physically existing object has recently been proposed. This technology connects the real world object with the virtual world (computer software) and provides the user with ease of understanding and operation. Of course, application with DTV / DTM software is also considered, but these techniques are not simply adopted in DTV / DTM. This is because, when this technology is applied in the field of DTV / DTM, handling of frames (time) becomes a very serious problem. Regardless of the video handled by the DTV or the music handled by the DTM, a large amount of video and music data is handled, and at the same time, an instantaneous frame (time) is searched from a vast time frame with extremely high accuracy. Not be. Broadcasting sites and music production sites are sometimes called for “instantly”, and interfaces that link the real and virtual worlds provide users with easy-to-understand and easy-to-use functions. so,
Achieving this immediacy (improvement) at the same time is of paramount importance.

The present invention has been made in view of the above-mentioned circumstances, and assigns a time to an object itself, and designates a time by a user designating the object. According to the first aspect of the present invention, it is possible to eliminate the inaccuracy of time designation caused by manipulating and positioning an object, and further, to enable an operation with immediacy (improvement). A second object of the present invention is to enable designation of various times without being limited by the number of objects. A third object of the present invention is to change a time frame that can be handled during work using an object by changing a time parameter, so that an operator can express various times. According to the invention of claim 4, for example, when a certain portion is indicated, the object can be flexibly applied to the object without being tied to the time of the data, such as returning to the beginning of the video data or moving to the beginning of the chapter, or moving to the end. The purpose is to set the time. A fifth object of the present invention is to perform a search with a large time interval when performing a rough search, and a small time interval when performing a search within a fine time frame. A sixth object of the present invention is to collectively manage the feature amount of an object.

[0008]

According to a first aspect of the present invention, an object representation includes a time parameter, and includes means for converting a result of observing the object into a signal. Having means to send
An instruction transmission unit, having an object observation unit, having means for receiving a signal as input, having means for generating an object observation result from the received signal, and having means for transmitting an instruction corresponding to the object determination result to an application; , Is provided. Therefore, since the object includes a parameter expressing time, it is possible to accurately specify the time of the object.

The invention of claim 2 is characterized in that, in the invention of claim 1, the object expresses different times depending on its part. Therefore, various times can be expressed by combining a plurality of objects.

A third aspect of the present invention is characterized in that, in the first aspect of the present invention, the time for expressing the object can be changed. Therefore, by changing the time parameter, the time frame that can be handled during the work using the object can be changed, and the operator can express various times.

According to a fourth aspect of the present invention, in the second aspect of the present invention, the time expressed by the part of the object can be changed. Therefore, in one object, the parts a, b, c,. . .
Assuming that there is a site z, the system can represent different times. Among them, for example,
Site a, site b, site c,. . . Site b of site z
By changing the time, only one part,
It is possible to represent the first time and the last time in a continuous value. For example, in the case of a book-based metaphor interface, if a book represents a two-hour video, each section of each line of a page in the book table of contents represents the first time of a chapter, etc. Be able to express.

According to a fifth aspect of the present invention, in the second aspect of the present invention, a unit of time expressed by each adjacent part of the object is changed.
Therefore, by combining multiple objects,
It is possible to express various times. When a rough search is performed, a large time interval is set. When a search is to be performed within a small time frame, the search can be performed with a small time interval.

According to a sixth aspect of the present invention, in the first aspect of the present invention, the command transmitting section has means for receiving information from a database storing information of the object.
The database has means for transmitting information to the command transmitting unit. Therefore, by using the database, the user can easily input the feature amounts of the objects into the database and manage them collectively.

[0014]

FIG. 1 is a diagram showing an example of the system configuration of an interface device according to the present invention.
Is an object, 102 is an object observation unit, 103
Is an application, 104 is a command transmission unit, 105 is a database, an object 101 whose expression includes a time parameter is observed by the object observation unit 102, and a command corresponding to the determination result is transmitted from the command transmission unit 104 to the application 103. The database 105 stores object information.

As an embodiment of the interface device of the present invention, a system to be used and a construction example (work example) will be described below. Here, the application to be used is a DTV application for reproducing video data.

1. System example 1.1. LiveDesk FIG. 2 is a view for explaining an embodiment of LiveDesk according to the present invention. FIG. 2A is a front view, FIG. 2B is a side view, and the desk 5 has an image input unit 1 and The object determination unit 2 and the application 3 are integrated. A video camera that captures work performed on the desk 5 is an image input unit 1
, And the PC is the object determination unit 2, and the object determination unit 2 is also where the application 3 is executed. 4 is a mirror.

The reason why the computer screen is projected on the desk is to consider the fusion of the real and virtual worlds. When workers work on a desk, physical objects are mostly handled in a flat world on the desk. However, the computer screen displayed by the display exists in a world that is almost perpendicular to the surface of the desk. In other words, it can be said that real world objects and virtual world objects are handled in different dimensions. Therefore, the computer screen is projected on a desk, and is regarded as having the same dimension as that of the real world object, so that the continuous flow between the real world and the virtual world is considered. L
In the live Desk, a worker can work on an electronic document by using a book, a file folder, and the like which are usually used.

FIG. 3 is a plan view, a front view, and a side view showing a configuration example of a Live Desk. As shown in FIG.
The part of the desk 5 of veDesk is 60cm in height and 100c in width.
m, height 70cm, made of wood, which is designed based on the size and height of the desk we are used to. The surface 5a of the desk 5 has a shape in which tempered glass is fitted into a wooden frame. By attaching tracing paper to the surface of glass, the surface of the desk is used as a screen instead of the screen, and the screen of the computer is projected on the desk using the projector (application) 3 and the mirror 4. FIG. 4 shows a configuration of a fixed portion of the video camera 1.

1.2. Equipment for LiveDesk The following equipment is used in LiveDesk. Object judgment unit: PC Macintosh (CPU: PowerPC G3, DRAM: 128M) OS Mac OS 8.5 Software development language Max Liquid crystal projector: Proxima Desktop Projector 5500C Digital video camera: Sony Corporation DCR-VX1000

1.3. FIG. 5 is a diagram for explaining a processing example of the PC 2.
2 has a bus standard called IEEE 1394 Fire Wire which receives image information from a digital video camera (image input unit) 1. The digital video camera 1 has a DV terminal, fetches information digitally, and sends it to the PC through a Fire Wire cable. Currently, IEEE1394 Fire Wire
The transfer rate of the bus standard is a maximum of 200 Mbps (25 MB / se
c, using Ultra Wide SCSI HD, it is possible to capture a resolution of 720 × 480 dots up to 30 sheets per second).

1.4. Color Code Recognition To recognize objects on LiveDesk,
In this embodiment, recognition based on a color code using colored paper is incorporated. In color recognition, processing is performed using a 360 × 240 dot 32-bit full color video image. The colors adopted in this embodiment are seven colors of red, blue, yellow, green, pink, yellow-green, and orange. The color recognition method is performed by analyzing an image of 360 × 240 dots by six parameters (R, G, B, Y, Cr, Cb) corresponding to each color. RGB is the three primary colors of light, and YCrCb is the luminance (Y) and saturation (Cr, Cb) when converting from a color image to a black and white image.
It is a component of. The parameters of the seven colors of light set in the present embodiment are as shown in Table 1.

[0022]

[Table 1]

With the above setting, color judgment is made based on how many dots of the same color exist in the image. However, if the detection is performed one dot at a time, the error becomes considerable. Therefore, if the upper, lower, left, and right positions are within the same color range, the detection is performed.

The color conversion from the RGB components to the YCrCb components is performed by multiplying the RGB values for each pixel by an appropriate coefficient. RGB to YCrC
Conversion to b is performed by the following conversion formula.

Y = 0.29900 × R + 0.587000 × G + 0.11400 × B Cb = -0.16874 × R-0.331260 × G + 0.50000 × B Cr = 0.50000 × R-0.418690 × G-0.081310 × B

The color code recognized by the above technique is converted into a signal, and the signal is converted into a signal by a computer (Macintosh).
G3) Transfer to the internal Max (video data processing system) and call the database. Here, a color code is described as an example, but other methods (for example,
2D barcodes, 3D barcodes, etc.).

1.5. Software used 1.5.1. Max is a programming language specialized in multimedia centering on sound, and is based on the C language.
Since 1986, the French IRCAM (Institut de Recherche et
de Coordination Acoustique / Musique) Development started at the National Laboratory and Maci as a general commercial package
Ntosh compatible version will be on sale from 1990. Video data operable functions and EBL (Explanation Based L)
Performing internal processes such as earning (learning based on explanation).

1.5.2. QuickTime Video data file format is Apple Computer's Quick
It is a movie format of kTime4.0, and the playback process is this QuickTime
Use 4.0.

1.6. Data Comparison Next, a description of the object database and instructions that can be specified will be described. The application used in this embodiment is an application that handles video data. In the description of this embodiment, a metaphor (metaphor) determined by the user is used. Here, when a user plays back video data, a case where a book is used as a metaphor and a case where a file cabinet is used as a metaphor are taken up. In addition, the string type interface is taken up for the sake of explanation.

1.6.1. Knowledge used to determine operable functions EBL (Expl) using the characteristics of video data as background knowledge and the characteristics of books and file cabinets as base literals
An operable function is determined by anation based learning (FIG. 6). First, the characteristics of the video data and the book / file cabinet are compared. Next, the common video data characteristic is assigned to the metaphor characteristic, and the operable function is determined by comparing the characteristic with the characteristic required for adding the function prepared for each function. The characteristics described in the database are as follows. FIG. 7 is a generalized proof tree derived from background knowledge that is a necessary condition for enabling a function to be operated.

Target concept: Videodata_physical → load (X) ^ start.stop (X) ^ ff.rev
(X) ^ marking (X)

Background knowledge (characteristics of video data): sort (frame, t), frame (flat, dimention (2)), t (axis) flat (video, frame), dimention_value (video) square (video, frame) , Axis (video, frame, t) send (video, frame)

Base literals (book characteristics): physical (Book), sort (frame, t), frame (flat, dimention (2)) t (axis), flat (Book, Page), dimention_value (Book) square ( Book, Page), axis (Book, Z), send (Book, Page) hold (Book, Page)

Base literals (characteristics of file cabinet): physical (Cabinet), flat (Cabinet, File) dimention_value (Cabinet), square (Cabinet, File) axis (Cabinet, Z), send (Cabinet, File) hold (Cabinet) , File)

Background knowledge (requirements for enabling functions): flat (X, P) ^ frame_dimention_value (X, P, D) ^ square (X, P)
^ equal (D, 2) → load (X) load (X) ^ axis (X, T) ^ over (P, 5) → start.stop (X) start.stop (X) ^ send (X, P) → ff.rev (X) start.stop (X) ^ hold (X, P) → marking (X)

1.6.2. Object Data Format The object data format has the following items. Color code number / object number / object part number / full path movie data name / time parameter (frame; expressed in absolute time [hhmmss] from the beginning of data)

Next, an example of an object corresponding to the color code number 0001 will be described. 0001,0001,001, c: \ data \ movie001.qt, 000000 0001,0001,002, c: \ data \ movie001.qt, 000015 0001,0001,003, c: \ data \ movie001.qt, 000030 0001, 0001,004, c: \ data \ movie001.qt, 000100 0001,0001,005, c: \ data \ movie002.qt, 000000 0001,0001,006, c: \ data \ movie003.qt, 000015 ...

These data are stored in the PC2 shown in FIG.
And is accessed from the software Max described in 1.5.1.

When changing the time information given according to the part of the object, for example, the above format is changed as follows. Example: Data of part 003 of object number 0001 is 000030
→ When changed to 000045 0001,0001,003, c: \ data \ movie001.qt, 000030 → 0001,000
1,003, c: \ data \ movie001.qt, 000045

The merit that the time information assigned to the part of the object can be changed freely is as follows.
For example, in the case of a movie with chapters, for example, the chapters can be centrally arranged at a certain location of an object, and the area around the site can be used as a table of contents.

Example: String type interface FIGS. 8 and 9 show examples of the string type interface.
As shown in the figure, the string-shaped object is divided into each part, and the time parameter of each video data file is assigned to each part. In FIG. 8, the left hand is the beginning of the video frame and the right hand is the last. Chapters exist in this series of video data. FIG.
Is a collection of chapters (chapter) with the beginning of the chapter gathered in the left hand, and the data assignment is changed so that the cue of the video data can be easily performed. The user can easily find the beginning of a chapter only by intensively operating around the left end portion.

1.7. MIDI MIDI (Musical Instrument Digital Interface) is a standard that defines standards for control signals and interfaces of electronic musical instruments. Since the data according to the MIDI standard is a file for simply specifying the timbre, the pitch, the volume, and the sound quality, it can be said that the data is very compact. In the present embodiment, the MIDI data expression is used when inputting information after recognizing an object into a program for operating an application.

2. Working Example Next, a working method of the constructed system will be described.
Here, two examples of a metaphor of video data, “book” and “file cabinet”, will be described.

2.1. In the case of the work method using a book as a metaphor As shown in FIG. 10, a book 6 with a color code added is placed on a Live Desk 5. This book expresses a certain range of time.

The image data from the camera installed on the upper part of the LiveDesk is converted into an object determination unit (PowerMa).
cintoshG3). Next, the object determination unit internally recognizes the color code and determines the object from the color code. Then, data relating to the determined object is extracted from the database of the object, and transmitted to Max mounted in the object determination unit. The correspondence between objects and color code numbers is described in the object database. Max receives the data and displays the data describing the picture and features of the book on the liquid crystal projector connected to the object determination unit (FIG. 11).

Next, an application (screen: virt)
As an ual_object), an application (screen: video) that reproduces video data displays video data from the application database (FIG. 12), compares both data, and displays an assignable video data function ( FIG. 13) Further, in the “FUNCTION View” window, a window of the movie file appears (FIG. 1).
4), [BOOK_WINDOW] rises, and video data can be operated using the book (FIG. 15). This operation includes, for example, the following. Play: Start by placing a book Stop: Specify with the book closed Frame specification: Each page of the book corresponds

Assuming that a book page is composed of seven pages, the entire data can be divided into only seven parts. Therefore, it can be said that it is not suitable for a detailed search, but this problem can be dealt with by making it possible to change the time parameter of an object in the object database by a user operation. That is, it is possible to change the time parameter of each part of the object (for example, each page of the present “object”) in the object database by a user operation.

For example, assuming that the time expressed by each adjacent part of the object is changed, the following is obtained. If the time to represent each page in the book interface is in units of 1 second as follows: 1st page: 1 second 2 pages: 2 seconds 3 pages: 3 seconds 4 pages: 4 seconds 5 pages: 5 seconds Eye : :

At this time, the time expression unit of each page is changed. For example, changing to the unit of 20 minutes is as follows. Page 1: 0 minutes Page 2: 20 minutes Page 3: 40 minutes Page 4: 60 minutes Page 5: 80 minutes ：:

As described above, the time expression unit of each part of the adjacent object is changed. As a result, even when the number of parts of the object that can express different times is small, it is possible to respond to various user requests by changing the unit of the expression time. The user can designate a frame (a certain time of a video) by opening a book. In addition, the user can intuitively understand the playback position of the image by the thickness of the book on the page to be opened, and by the speed of opening the book,
It is also possible to intuitively control the video playback speed.

2.2. In the case of the working method using a file cabinet as a metaphor As shown in FIG. 16, a book 7 with a color code added is placed on a Live Desk. This cabinet expresses a certain period of time. As in the case of the book, the file cabinet is recognized, the data is called from the database of the object (FIG. 17), and after comparing the databases, the characteristics of the object are displayed, and a command that can be specified by the application from the application database is issued. After the acquisition, a window for operating the movie is activated (FIG. 18). The main function is the same as a book. Specific operations include the following. Playback: Cabinet itself supported Stop: Cabinet does not exist Frame specification: Each file in the cabinet is applicable In this example, it is assumed that five files are stored in the file cabinet, 5 whole movie
Divided. Therefore, it can be said that this work method is suitable for a rough search such as when searching for the position of one frame (time) from a plurality of cabinets.

[0052]

As described above, according to the present invention,
By assigning time to the object itself and specifying the time by the user specifying the object, it is possible to eliminate the inaccuracy of time specification caused by manipulating and positioning the object Further, an operation with immediacy (improvement) can be performed.

[Brief description of the drawings]

FIG. 1 is a diagram showing a system configuration example of an interface device according to the present invention.

FIG. 2 is a view for explaining an embodiment of a LiveDesk according to the present invention, wherein FIG.
(B) is a side view.

FIG. 3 is a plan view, a front view, and a side view illustrating a configuration example of a Live Desk.

FIG. 4 is a diagram showing a configuration of a fixed portion of the video camera.

FIG. 5 is a diagram illustrating a processing example of a PC.

FIG. 6 is a diagram for explaining determination of a function operable by the EBL.

FIG. 7 is a generalized proof tree derived from background knowledge.

FIG. 8 is a diagram illustrating an example of a string type interface.

FIG. 9 is a diagram showing another example of a string type interface.

FIG. 10 is a diagram showing a state in which a book with a color code added is placed on a Live Desk.

FIG. 11 is a diagram showing that data is called from an object database.

FIG. 12 is a diagram showing a state in which Video data is displayed.

FIG. 13 is a diagram showing a state in which functions of assignable Video data are displayed.

FIG. 14 is a diagram illustrating a state in which a window of a movie file is displayed.

FIG. 15 is a diagram showing a display state when video data becomes operable.

FIG. 16 is a diagram showing a state in which a book with a color code added is placed on a Live Desk.

FIG. 17 is a diagram showing that data is called from an object database.

FIG. 18 is a diagram illustrating a state in which a window for operating a movie starts up.

[Explanation of symbols]

REFERENCE SIGNS LIST 1 image input unit 2 object determination unit 3 application 4 mirror 5 desk 101 object
102: object observation unit; 103: application; 104: command transmission unit; 105: database.

 ──────────────────────────────────────────────────続 き Continued on the front page F term (reference) 5B087 AA09 AB04 BC11 BC32 CC09 CC11 CC33 DD11 DE05 5C053 FA14 HA29 LA01 LA11 LA15 5E501 AA01 AC15 AC34 BA03 BA12 CA02 DA14 FA14 FA26 FB33 FB45

Claims (6)

[Claims] The object representation includes a time parameter, has a means for converting a result of observing the object into a signal, has an object observation unit having a means for transmitting the signal, and has means for receiving a signal as an input. And an instruction transmission unit having means for generating an observation result of the object from the received signal, and having means for transmitting an instruction corresponding to the object determination result to the application. 2. The interface device according to claim 1, wherein the object expresses a different time depending on its part. 3. The interface device according to claim 1, wherein a time represented by the object can be changed. 4. The interface device according to claim 2, wherein the time expressed by the part of the object can be changed. 5. The method according to claim 2, wherein a unit of time expressed by each adjacent part of the object changes.
An interface device characterized by the above-mentioned. 6. The apparatus according to claim 1, wherein the command transmitting unit has means for receiving information from a database storing information of the object, and the database has means for transmitting information to the command transmitting unit. Interface device characterized.