JP4864968B2 - Recording / playback system - Google Patents

Description

【Technical field】
[0001]
The present invention records and reproduces operation and display history for various data displayed on the screen, such as moving images, still images, audio, drawing and text, and activated applications, while maintaining the functions at the time. It relates to a playback system.
[Background]
[0002]
2. Description of the Related Art Conventionally, an apparatus that records a user's operation history regarding an application program or the like started on a personal computer has been proposed (for example, see Patent Document 1 and Non-Patent Document 1). According to such an apparatus, it is possible to quickly create and draw a document by reproducing the operation history as a moving image or using the operation history with document creation software or drawing software. Similarly, an apparatus that can reproduce the same presentation by using the operation history has been proposed (see, for example, Patent Document 2).
[0003]
On the other hand, many products (hereinafter referred to as composite products) in which a television tuner, audio / video equipment (AV equipment) and the like are combined with a personal computer as a core have been developed recently. According to these composite products, for example, an image of an application program started on a personal computer, an image of a Web site acquired through the Internet, a television broadcast or a video image acquired through a TV tuner or an AV device, and the like are the same. It can be displayed on the screen.
[0004]
As an application example of such a product, there is an electronic blackboard apparatus as shown in FIG. This electronic blackboard device is equipped with a large display unit 200 using a plasma display or the like as a blackboard, and displays images, texts, drawings, etc. using a connected personal computer 301, AV device 401, and pen input device 302.
[0005]
For example, when a presentation or education (such as a lesson in a classroom) is performed using the above electronic blackboard device, an image of an application program started on a personal computer, a video image acquired from a connected AV device, a drawing input by handwriting Many data such as images acquired from websites on the Internet can be displayed on the same screen.
[0006]
The case of education will be specifically described with reference to the display example of FIG. For example, in an arbitrary class in which the application program displays text 211 (such as a textbook) and performs a handwriting input on a blackboard 214, a video image 212 or a Web page image 213 is displayed in another area on the same screen 210 as a teaching material for the class. A situation is assumed. Even in such a composite product, it is preferable that the history of presentation and education can be recorded and the presentation and lesson can be reproduced. For example, in the case of a class or the like, it is possible to perform a more efficient class by utilizing the reproduction of history.
[0007]
However, in the conventional apparatus for recording the operation history disclosed above, the operation history in the personal computer (a plurality of applications) can be referred to, but there is a problem that it cannot be referred (reproduced) until the screen display state. In other words, as explained in the above class example, in the conventional apparatus, the display of application programs and the history of drawing can be recorded, but what is the source of data (content) regarding the display of video images and Web pages played back in the middle? Even if the degree can be recorded, it is not possible to record a history such as which part of the content has been displayed for how long.
[0008]
For example, if you want to reuse a scene that displayed a homepage related to the current day's weather forecast in a past class, the homepage is usually updated, so keep the same state as the previous time and reproduce the class The problem arises that you can't. Therefore, in the conventional apparatus, the past history of operation and screen display cannot be reproduced by specifying the time.
[0009]
On the other hand, as a means for solving this problem, a method of video recording of the lesson situation is conceivable, but in this case, there arises a problem that it is not possible to return to an arbitrary state in the past and restart the lesson. Specifically, for example, when a lesson is performed by reusing an operation history of a past lesson, it is preferable that there is an error in a board writing (handwriting input by a pen input device) in the past lesson. Immediately before an error occurs, the history of operation of various devices and screen display is reproduced, and the application that has been started (text display or pen input) is used as it is from the time of the error, and the board is rewritten and corrected. Is to do.
[0010]
However, in the method of playing back only the recorded video, the application activation state and screen display state are not reproduced, so it is not possible to correct the class (exactly, branching operation for correction) from the middle It was.
[Prior art documents]
[Patent Literature]
[0011]
[Patent Document 1]
JP 2000-163602 A (2nd page-5th page, FIG. 1)
[Patent Document 2]
Japanese Patent Laid-Open No. 5-67150 (page 2 to page 5, FIG. 1)
[0012]
[Non-patent literature]
[Non-Patent Document 1]
My desk Next-generation history information search tool, [online], [Search May 8, 2006], Internet <URL: http // oredesk.com>
SUMMARY OF THE INVENTION
[Problems to be solved by the invention]
[0013]
Problems to be solved by the present invention include linear content that changes in time series such as moving image data supplied from an AV device (video recorder, digital camera, etc.), and text data displayed by a document creation application program When displaying non-linear content that does not change in time series like the above on the same screen for presentations etc., the screen display status and functions of each content can be played back in synchronization with each other. As an example, it is possible to realize a recording / reproducing system capable of appropriately recording the image.
[Means for Solving the Problems]
[0014]
In order to solve the above-described problem, the recording / reproducing system according to claim 1 includes an image including a linear content video whose playback content changes in time series and a non-linear content video whose playback content does not change in time series. Display on the screen of the display device, The video displayed on the screen in chronological order Video recording means for recording, operation history generating means for generating operation history by acquiring operation signals for the linear content and the non-linear content, file use history generating means for generating a history of files used for the non-linear content, Scenario generation recording means for generating and recording a scenario in which the operation history and the file usage history are associated with each other on a time axis, and from a specified restoration point based on the scenario Screen display video recorded in time series on the video recording means Content playback means for starting playback of the content, and the content playback means from the designated restoration point Screen display video recorded in time series on the video recording means If it is not possible to reproduce at least a part of the specified recovery point, it is further determined whether there is another recovery point retroactively from the specified recovery point, and if there is no other recovery point, the specified From the restored point to the video recording means In time series It is characterized in that the recorded screen display video is reproduced.
[Brief description of the drawings]
[0015]
FIG. 1 is a diagram showing a configuration of a conventional electronic blackboard device.
FIG. 2 is a diagram schematically showing a display display example in a conventional electronic blackboard device.
FIG. 3 is a diagram showing a schematic configuration of a recording / reproducing system in an embodiment of the present invention.
FIG. 4 is a diagram showing an internal configuration of a recording / reproducing apparatus of the recording / reproducing system according to the embodiment of the present invention.
FIG. 5 is a diagram showing a process and data flow by extracting a functional part related to an operation history recording operation from the internal configuration of the recording / reproducing apparatus.
FIG. 6 is a flowchart showing an operation history recording operation procedure of the recording / reproducing system in the embodiment of the present invention.
FIG. 7 is a diagram showing a process and data flow by extracting a functional part involved in a screen display reproduction operation from the internal configuration of the recording / reproducing apparatus.
FIG. 8 is a flowchart showing a procedure for reproducing the screen display of the recording / reproducing system in the embodiment of the present invention.
FIG. 9 is a flowchart showing a restoration operation procedure of the recording / reproducing system in the embodiment of the present invention.
FIG. 10 is a diagram showing a configuration and processing of a recording / reproducing system and a data flow when the PC system has a function equivalent to an AV selector.
FIG. 11 is a diagram showing a process and data flow by extracting a functional part related to an operation history recording operation from the internal configuration of the recording / reproducing system when the PC system has a function equivalent to an AV selector. .
FIG. 12 is a diagram showing a process and data flow by extracting a functional part involved in a screen display reproduction operation from an internal configuration of the recording / reproduction system when the PC system has a function equivalent to an AV selector. .
BEST MODE FOR CARRYING OUT THE INVENTION
[0016]
The recording / reproducing system according to the embodiment of the present invention includes the following (1) to ( 4 It is the best mode that is characterized by the provision of the above-mentioned means.
(1) Video recording in which video including linear content video whose playback content changes in time series and non-linear content video whose playback content does not change in time series is displayed on the screen of the display device, and the video is recorded means.
(2) Operation history generation means for acquiring operation signals for the linear content and the non-linear content to generate an operation history.
(3) File usage history generation means for generating a history of usage files of the nonlinear content.
(4) Scenario generation and recording means for generating and recording a scenario in which the operation history and the file use history are associated with each other on a time axis, and the linear content and the non-linear content from a specified predetermined time point based on the scenario Content playback means for starting playback of
[0017]
The recording / reproducing system according to the embodiment of the present invention performs not only recording of a display video but also recording and reproduction of operations and functions associated therewith. Specifically, for example, in addition to recording all display videos from the start to the end of classes and presentations, at the same time, all history related to device operation is recorded in synchronization with video recording. The progress of the used file is recorded at any time, and this progress can be reused.
[0018]
In other words, when the display video is restored and played back, the function of the device that supplies the video is also restored, so that the same situation as at that time in the past is reproduced, and the device can be operated. Furthermore, when the display image is restored, the elapsed time from the screen and the subsequent operation are also recorded as new information. Accordingly, while using the system, display video is always recorded, and reusable content (that is, coherent recording related to lessons and presentations) is accumulated.
[0019]
(Example)
Hereinafter, specific examples of the recording / reproducing system according to the embodiment of the present invention will be described in detail with reference to the drawings.
FIG. 3 is a diagram showing a schematic configuration of an example of the recording / reproducing system according to the embodiment of the present invention. The recording / reproducing system mainly includes a recording / reproducing apparatus 100, a display unit 200 such as a plasma display connected thereto, a personal computer (hereinafter referred to as a PC system) 300, and a peripheral AV device 400.
[0020]
FIG. 4 is a diagram showing an internal configuration of a recording / reproducing apparatus in the recording / reproducing system of the present embodiment. The recording / reproducing apparatus 100 mainly includes an IR signal receiving unit 11, an AV device state acquisition unit 12, an AV device state calculation unit 13, an AV device state restoration unit 14, an AV device state setting unit 15, an IR signal transmission unit 16, and a PC. Operation acquisition unit 21, PC state acquisition unit 22, PC state calculation unit 23, PC state restoration unit 24, PC state setting unit 25, PC operation transmission unit 26, command table 27, scenario creation unit 31, scenario recording unit 32, scenario Recording database (DB) 33, scenario acquisition unit 34, scenario restoration unit 35, video recording unit 41, video recording database (DB) 42, video acquisition / playback unit 43, recording / playback management unit 50, and time generator 60 Composed.
[0021]
The IR signal receiving unit 11 receives an IR signal for operating the peripheral AV device 400. In addition, the AV device state acquisition unit 12 acquires information regarding the state of the peripheral AV device 400.
[0022]
Note that the IR signal is a signal transmitted from a remote control device attached to the AV device, and other types of signals can be obtained as long as the operation information for the peripheral AV device can be acquired or the peripheral AV device can be operated. It can be a signal.
[0023]
In addition, the AV device state calculation unit 13 calculates the state of the peripheral AV device 400 based on the received IR signal and information on the state acquired by the AV device state acquisition unit 12. Then, by recording the received IR signal history together with the acquired AV device state, it functions as an operation history generating means for the peripheral AV device.
[0024]
On the other hand, the AV device state restoration unit 14 generates information for reproducing the state of the peripheral AV device 400 when the screen display is reproduced. The AV device state setting unit 15 sets the information generated by the AV device state restoring unit 14 for the peripheral AV device 400.
[0025]
The IR signal transmitting unit 16 transmits an IR signal for operating the peripheral AV device 400 when reproducing the screen display.
[0026]
The PC operation acquisition unit 21 acquires a signal for operating the PC system 300 input from a mouse or a keyboard. The PC status acquisition unit 22 acquires information regarding the status of the PC system 300.
[0027]
Further, the PC state calculation unit 23 calculates the state of the PC system 300 based on the information regarding the state acquired by the PC state acquisition unit 22.
[0028]
On the other hand, the PC state restoration unit 24 generates information for reproducing the state of the PC system 300 when the screen display is reproduced.
[0029]
The PC state setting unit 25 sets the information generated by the PC state restoring unit 24 for the PC system 300. The PC operation transmission unit 26 transmits a signal for operating the PC system 300 when reproducing the screen display.
[0030]
Note that the PC system status information specifically includes the name of the running application, the display position / display status of the application, the name of the processing file, the status (elapsed) of the file being edited, etc. May be included.
[0031]
The scenario creation unit 31 creates scenario data. The scenario data is configured by associating information on the state of the peripheral AV device and the PC system calculated by the AV device state calculation unit 13 and the PC state calculation unit 23 with time stamp information acquired from the time generation device 60 described later. Information.
[0032]
The scenario recording unit 32 records the created scenario data in the scenario recording database 33.
[0033]
The scenario acquisition unit 34 acquires specific scenario data stored in the scenario recording database 33.
[0034]
The scenario restoration unit 35 generates information for reproducing the state of the peripheral AV device 400 and the PC system 300 from the acquired scenario data when reproducing the screen display.
[0035]
The moving picture recording unit 41 records the video data supplied from the peripheral AV device 400 in the moving picture recording database 42.
[0036]
In this embodiment, the peripheral AV device 400 functions as an AV selector, and an image displayed on the screen by an application running on the PC system 300 can also be recorded as a moving image via the peripheral AV device 400. Recorded in part 41.
[0037]
That is, the video (display video) displayed on the display unit 200 is recorded as it is in time series. When reproducing the screen display, the video acquisition / playback unit 43 obtains the video data of the corresponding part recorded in the video recording database 42 based on the scenario data for reproduction generated by the scenario restoration unit 35 described above. It is acquired and sent to the peripheral AV device 400 that functions as an AV selector. The transmitted video data is reproduced as reproduced video on the display unit 200 via the moving image recording unit 41. The video reproduced as the reproduced video is recursively recorded in the moving image recording database 42.
[0038]
The recording / playback management unit 50 performs overall management of the recording / playback operations of the respective units of the recording / playback apparatus 100 described above. A recording start / end request command for screen display data input from the outside via an input unit (not shown) to the recording / playback apparatus 100 and a playback start / end request command for recorded screen display data are recorded / playback managed. This is transmitted to the moving image recording unit 41 and the scenario recording unit 32 via the unit 50.
[0039]
The time generating device 60 generates information related to the absolute time or the relative time from when the first time generation request is made. By linking this time information to the recorded display screen data, it is possible to accurately reproduce the operation and state of the peripheral AV device 400 as well as the PC system 300 when reproducing the screen display.
[0040]
Next, the operation of the recording / reproducing system of this embodiment will be described.
First, the operation history recording operation will be described. FIG. 5 is a diagram showing a process and data flow by extracting a functional part related to an operation history recording operation from the internal configuration of the recording / reproducing apparatus.
[0041]
FIG. 6 is a flowchart showing the operation history recording operation procedure of the recording / reproducing system in the present embodiment.
First, when the PC operation acquisition unit 21 acquires a trigger operation signal such as keyboard input, mouse input, or timer processing (step S101), the PC state calculation unit 23 checks whether there is a common interface (status 102). If there is a common interface, the status of the running application is acquired through the common interface (step S103).
[0042]
Acquisition of information relating to the state is performed by the PC state acquisition unit 22. The PC state calculation unit 23 transmits the acquired information to the scenario creation unit 31. The scenario creating unit 31 links the transmitted information with the information related to the state of the AV device transmitted from the AV device state calculating unit 13, and creates scenario data in association with the time information acquired from the time generating device 60. (Step S104). The created scenario data is stored in the scenario recording database 33.
[0043]
On the other hand, if there is no common interface in the determination in step S102, the PC state calculation unit 23 next determines whether a method for acquiring an individual state is stored in the command table (step S105).
[0044]
If it is stored, the status of the application running by the method described in the command table is acquired (step S106). Acquisition of information relating to the state is performed by the PC state acquisition unit 22. The PC state calculation unit 23 transmits the acquired information to the scenario creation unit 31.
[0045]
The scenario creating unit 31 links the transmitted information with the information related to the state of the AV device transmitted from the AV device state calculating unit 13, and creates scenario data in association with the time information acquired from the time generating device 60. (Step S107). The created scenario data is stored in the scenario recording database 33.
[0046]
On the other hand, if the command is not saved in the determination in step S105, the PC state calculation unit 23 next determines whether or not there is an operation input from an IR device such as a keyboard, a mouse, or a remote control device of an AV device (step S108). ).
[0047]
If there is an operation input, scenario data related to the operation input is created (step S109). If there is no operation input, the thread for recording the operation history is terminated (step S110).
[0048]
Next, the screen display reproduction operation for reproducing the screen display in the past will be described. FIG. 7 is a diagram showing a process and data flow by extracting a functional part related to the reproduction operation of the screen display from the internal configuration of the recording / reproducing apparatus.
[0049]
FIG. 8 is a flowchart showing the procedure for reproducing the screen display of the recording / reproducing system in the present embodiment. First, the recording / playback management unit 50 receives a request for a restoration point serving as a base point for starting playback of the screen display (step S201), and determines whether there is information that can be directly restored to the restoration point (step S202). If there is information that can be directly restored, the recording / playback management unit 50 performs a restoration process described later (step S203).
[0050]
If there is no information that can be directly restored, the recording / playback management unit 50 further determines whether there is a restoration point retroactively (step S204). If there is no restoration point, the recording / reproduction management unit 50 performs a moving image reproduction process from the restoration point (step S205). The moving image reproduction process is a process of reproducing a screen display image recorded in the moving image recording database.
[0051]
The recording / playback management unit 50 causes the moving image acquisition / playback unit 43 to acquire the moving image data stored in the moving image recording database via the scenario acquisition unit 34 and the scenario restoration unit 35.
[0052]
On the other hand, if there is a restoration point in the determination in step S204, restoration processing is performed from the restoration point (step S206). Then, the input process is reproduced (step S207), and the input process is repeated until the original restoration point is reached (step S208). If the restoration point is reached in step S208, the process ends.
[0053]
Next, the details of the restoration process will be described. FIG. 9 is a flowchart showing the restoration processing procedure of the recording / reproducing system in the present embodiment.
[0054]
First, the recording / playback management unit 50 receives the designation of the restoration point (step S301), and activates the process recorded in the scenario recording database 33 (step S302). Next, the presence / absence of a used file is determined (step S303), and if a used file exists, the process notifies the process of the used file (step S304).
[0055]
If it is determined in step S303 that there is no file to be used, it is next determined whether the information can be reproduced by the common interface (step S305). If the information is reproducible, the display state of the display page is notified to the process (step S306), and the restoration process is completed.
[0056]
On the other hand, if the information is not reproducible in the determination in step S305, an individual application state transition command is acquired (step S307), the display state of the display page is notified to the process (step S308), and the restoration process is performed. Complete.
[0057]
Specifically, the screen display is reproduced by the scenario acquisition unit 34 acquiring the corresponding scenario from the scenario recording database 33 and transmitting the scenario to the scenario recovery unit 35, the scenario recovery unit 35, the AV device state recovery unit 14, and the PC state. A screen display in which the states of the PC system 300 and the peripheral AV device 400 are restored via the restoration unit 24 is reproduced and performed on the display unit 200.
[0058]
FIG. 10 is a diagram showing the configuration and processing of the recording / playback system and the data flow when the PC system has a function equivalent to an AV selector.
[0059]
FIG. 11 is a diagram showing a process and data flow by extracting a functional part related to the recording operation of the operation history from the internal configuration of the recording / reproducing system when the PC system has a function equivalent to an AV selector. It is.
[0060]
Further, FIG. 12 is a diagram showing a process and data flow by extracting a functional part related to the reproduction operation of the screen display from the internal configuration of the recording / reproducing system when the PC system assumes a function equivalent to the AV selector. It is.
[0061]
Here, since the function and operation of each unit are the same as when the peripheral AV device 400 has the function of an AV selector, detailed description thereof is omitted.
[0062]
In the above embodiment, the recording mode is always switched to the reproduction mode, but each recording operation is continued. The input switching of the display unit 200 monitor is controlled by the PC system 300, but can also be switched by a remote control device attached to the peripheral AV device 400.
[0063]
In the above embodiment, the moving image recording unit 41 and the moving image recording database 42 play a role as video recording means. In addition, the IR signal receiving unit 11, the AV device state obtaining unit 12 and the AV device state calculating unit 13 serve as operation history generation means for linear content, and the PC operation obtaining unit 21 serves as operation history generation means for non-linear content. The unit 22 plays a role as a file usage history generation unit. Furthermore, the scenario creation unit 31 and the scenario recording unit 32 play a role as scenario generation and recording means.
[0064]
Further, a scenario acquisition unit 34, a scenario restoration unit 35, an AV device state restoration unit 14, a PC state restoration unit 24, an AV device state setting unit 15, an IR signal transmission unit 16, a PC state setting unit 25, and a PC operation transmission unit 26 are provided. It plays a role as a content reproduction means.
[0065]
As described above in detail in the above embodiment, the recording / playback system according to the embodiment of the present invention has a video of linear content (such as moving image data) whose playback content changes in time series and a playback content of time. A video including a video of non-linear content (such as text data) that does not change in series is displayed on the screen of the display device, and a video recording unit 41 that records the video, and an operation signal for the linear content and the non-linear content are displayed. A PC operation acquisition unit 21 that acquires and generates an operation history, a PC state acquisition unit 22 that generates a history of the usage file of the nonlinear content, and a scenario in which the operation history and the file usage history are associated on a time axis. And the scenario creation unit 31 for recording, and based on the scenario, the linear copy is performed from a designated predetermined time point. Scenario acquisition unit 34, scenario restoration unit 35, AV device state restoration unit 14, PC state restoration unit 24, AV device state setting unit 15, IR signal transmission unit 16, PC state setting unit for starting reproduction of the content and the nonlinear content 25, a content reproduction function unit including a PC operation transmission unit 26.
[0066]
Thereby, it is possible to faithfully reproduce the past screen display of the content in a presentation or a lesson and reuse it. It is also possible to reproduce the screen display from the state at an arbitrary time in the past.
[0067]
That is, since not only the application in the personal computer but also the usage history of the linear content provided from the AV device or the like is held, it is possible to return to the past state and start again (branch).
[Explanation of symbols]
[0068]
11 IR signal receiver
12 AV equipment status acquisition unit
13 AV equipment status calculator
14 AV equipment state restoration unit
15 AV equipment state setting section
16 IR signal transmitter
21 PC operation acquisition unit
22 PC status acquisition unit
23 PC status calculator
24 PC state restoration part
25 PC status setting section
26 PC operation transmitter
31 Scenario creation department
32 Scenario recording part
33 Scenario record database (DB)
34 Scenario acquisition unit
35 Scenario Restoration Department
41 Movie recording unit
42 Video recording database (DB)
43 Movie acquisition / playback unit
50 Recording / playback management section
60 Time generator
100 Recording / reproducing apparatus
200 display section
300, 301 PC system
302 Pen input device
310 Input using mouse or keyboard
400, 401 AV equipment
410 IR input

Claims (1)

Display the video including linear content video whose playback contents change in time series and non-linear content video whose playback contents do not change in time series on the screen of the display device, and display the video displayed on the screen in time series Video recording means for recording;
An operation history generating means for acquiring an operation signal for the linear content and the non-linear content and generating an operation history;
File usage history generation means for generating a history of the usage file of the non-linear content;
Scenario generation recording means for generating and recording a scenario in which the operation history and the file usage history are associated with each other on a time axis;
Based on the scenario, content playback means for starting playback of the screen-displayed video recorded in time series on the video recording means from a designated restoration point;
With
If the content reproduction means cannot reproduce at least a part of the screen-displayed video recorded in time series on the video recording means from the designated restoration point, the content reproduction means further starts from the designated restoration point. It is determined whether or not there is another restoration point retroactively, and if there is no other restoration point, the screen-displayed video recorded in time series on the video recording means from the designated restoration point is displayed. A recording / reproducing system for reproducing.

Images