JPH03160664A - Picture recording/display device - Google Patents

Abstract

PURPOSE:To easily retrieve a recorded scene and to facilitate reproduction by analyzing character display in a program, recording it in a storage device at the time of recording the television program, retrieving a character code and reproducing a desired scene at the time of reproduction. CONSTITUTION:When a user operates a remote controller 26 and gives the instruction of the recording of VTR, the resetting of an intra-recording counter is executed and recording is started. Then, character display in a video is analyzed at every five seconds, and a character recognition result is recorded in an external storage device 12. When the user depresses the video display push button of the remote controller 26, a display 6 comes to a video display mode, and processings such as reproduction, rewinding and fast-forwarding are executed. When computer display is instructed, the character code recorded in the device 12 is displayed in the display 6, and the user selects the desired scene on a list. Thus, the recorded scene can easily be retrieved and reproduction is facilitated.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to an image recording and display device having, for example, a function of recording and reproducing television programs and the like and a character recognition function. [Prior Art] In recent years, with the spread of video table recorders (VTR),
Various systems have been proposed in combination with personal computers (PCs), workstations, and the like. [Problems to be Solved by the Invention] However, in the above-mentioned conventional examples, no device has been proposed that recognizes characters on the screen of a television program recorded on a VTR and records the results as a file. For this reason, it has not been possible to easily search for and playback a desired scene of a recorded television program, for example. The present invention was developed to solve the above problems, and
To provide an image recording/displaying device capable of easily searching and reproducing recorded scenes. [Means and operations for solving the problem] In order to achieve the above object, the image recording and display device of the present invention has the following configuration. That is, it is an image recording and display device having a recording/playback function for television programs, etc., and a character recognition function, comprising: a character recognition means for recognizing characters from character signals included in scenes of the television program; The apparatus comprises a recording means for recording a recognized character code, and a display means for displaying according to the character code recorded by the recording means, and preferably searches for a desired scene based on the character code on the display means. By doing so, you can easily search for and play back recorded scenes. [Embodiment] Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings. <Description of Configuration> FIG. 1 is a schematic block diagram showing the configuration of the image recording and display apparatus in this embodiment. When divided broadly, the left half of the area surrounded by the dotted line is the workstation side, and the right half is the VTR side. To explain the details of each, first, on the workstation side, 1 is the central processing unit (hereinafter referred to as C) that controls the entire device.
2 is a read-only memory (hereinafter referred to as ROM) that stores programs and character fonts (to be described later); 3 is a random access memory that includes a work area and various tables used by the CPU 1 when executing processing; Memory (hereinafter referred to as RAM) 4, video RAM (hereinafter referred to as VRA) for displaying images on a CRT display 6
It is composed of 5 (referred to as M). 6 is a CRT display that displays menus, etc., which will be described later;
7 is a left speaker, 8 is a right speaker, and 9 is a printer for printing. 10 is a keyboard for data entry, 11
is a mouse as a pointing device. 1
2 is an external storage device, a floppy disk device l3
It consists of a hard disk drive l4. A large amount of programs and data are stored in the external storage device l2, and are loaded into the RAM 4 of the memory 2 to perform a large number of processes. Next, each device on the VTR side in this embodiment will be explained. 20 is the main body of the VTR device, and the CRT display 6
Record and play back images displayed on the screen. Audio recording/
It also plays. 21 is a channel selection circuit, which selects a television (TV) according to instructions from a VTR controller 22, which will be described later.
Select each channel of the signal. A VTR controller 22 controls the VTR device main body 20 in accordance with control signals output from the CPU of the workstation. Note that the details of the internal structure will be described later. In addition, VTR device 2
0 is connected to a VTR remote control 26, which will be described later, through an infrared light emitting section 27 and a light receiving section containing an infrared sensor. 23 is a signal conversion circuit that converts the signal of one screen of TV video into a digital signal that can be processed on the workstation side. 24 is a signal switching device, VT
According to instructions from the R controller 22, each signal from the channel selection circuit 2l, the VTR device main body 20, or the CPU 1 is switched in three ways and output to the CRT display 6, left speaker 7, and right speaker 8. 25 is a modem that converts digital information on the workstation side into an audio signal so that it can be recorded on a VTR. Of course, during playback, the information converted into an audio signal is converted into digital information and sent to the workstation. Note that this information is recorded in the audio (monaural) area on the VTR tape, details of which will be described later. FIG. 2 is an external view of the entire apparatus in this embodiment. In the figure, 30 is the main body of this embodiment which contains most of the workstation side and VTR side devices shown in FIG. 1, 31 is the floppy disk insertion slot, and 32 is the VT
33 is a message display section of the VTR; and 28 is a light receiving section for receiving infrared pulses from the remote control 26. Other components are a CRT display 6, a left speaker 7, a right speaker 8, a printer 9, a keyboard 10, and a mouse 11 shown in FIG. FIG. 3 is an external view showing the main body of the remote control 26 in this embodiment. As shown in the figure, at the top of the remote control 26,
There is a light emitting section 27, which uses infrared pulses to send an instruction signal to the VTR controller 22 toward the sensor of the light receiving section 28. In addition, the remote control 26 body first has a CRT.
A TV display 40 button, a video display 4L button, for the user to select what to display on the display 6;
There are 42 buttons on the computer display. Immediately after the power is turned on, the display on the CRT display 6 is in the computer display mode on the workstation side. For example, when the user pushes the TV display button 40, the CRT display 6 switches to display the TV program received from the TV signal, and from then on, the user can change channels, When starting, interrupting, restarting, or ending recording, or when recording, it simultaneously analyzes character data in the program, turns on/off the mode for file recording, resets the tape counter, etc. The user can also make a recording reservation using a group of buttons in 44 areas. Further, when the user pushes the video display 41 button, the CRT display 6 displays the VT of the VTR device body 20.
Display the reproduced image of the R tape. The user can play, rewind, fast forward, stop, still image, slow, etc. using buttons in 45 areas on the remote controller 26. Furthermore, if the CRT display 6 is in the computer display state and a list of playback start positions is displayed immediately before the user pushes the video display 41 button, then after pushing the video display 41 button, the user presses the remote control 26 Each time the search next candidate button or search pre-candidate button in the upper area 46 is pressed, the VTR head moves to the playback start position on the VTR tape specified on the list, and still image playback is executed. The details will be explained later with reference to FIGS. 9, 10, and 11. Note that when the user pushes the computer display button 42, the display on the CRT display 6 becomes the computer display on the workstation side. FIG. 4(a) is a conceptual diagram showing the positional relationship between the rotary head 50 inside the VTR apparatus main body 2o and the recording area 52 on the VTR tape 5l. Further, FIG. 4(b) is a conceptual diagram showing the types of signals recorded on the VTR tape 5l, with a video signal 53 and an audio signal (left) 54, respectively.
, audio signal (right) 55, audio signal (monaural) 56, etc. The audio signal (left) 54 is the left speaker 7.
, the audio signal (right) 55 is reproduced as a rattling sound from the right speaker 8, respectively. However, the audio signal (left) 54 includes a computer synthesized sound,
Microphone sounds are recorded in the audio signal (right) 55, respectively. Stereo audio is not recorded like normal home video. Furthermore, in the audio signal (monaural) area 56, digital information from the workstation is converted into an audio signal through the modem 52 and recorded. Specifically, search data such as the value of an in-record counter and an identification number are recorded, but the details will be described later. FIG. 5 shows the message display section 3 in the VTR device main body 20.
In the display example 3, the user selects the video display 4 on the remote control 26.
After pressing the l button, press the play button to start the VTR.
This is an example during playback. In the figure, 33-1 is a video display mode, indicating that the VTR is being reproduced. 33-2 records search data and other files when recording the program being played.
The recognition number of that file is No. This shows that l 2 3 4. 33-3 is an in-recording counter value indicating the position currently being played, assuming that the beginning of the recorded program is "O" seconds. Further, 33-4 is a tape counter value indicating the position currently being played, assuming that the beginning of the VTR tape is "0" seconds. The details of the identification number and the recorded counter will be explained later in FIG. 12. FIG. 6 is a block diagram showing the internal structure of the VTR controller 22 explained in FIG. 1. This is also a type of computer, with a CPU 22-1, memory 22-2, RO
It is composed of M22-3, RAM22-4, etc. The function of each of these parts is explained on the workstation side in Figure 1.
The functions are the same as those of PU 1 , memory 2, ROM 3, and RAM 4, and the explanation will be omitted here. This CPU 22-1 includes, in addition to the memory 22-2, a CPU on the workstation side, a VTR device body 20, a channel selection circuit 21, a signal switching device 24, an infrared light receiving section 2
8, and according to instructions from the workstation or instructions from the remote control 26, the VTR device main body 2
Control 0 etc. FIG. 7 is a software configuration diagram, and in this embodiment there is news search software 73;
further includes a recording program 75 and a list/playback program 76. The recording program 75 further includes a file recording program 75-1, a tape recording program 75-2, and a character recognition (OCR) program 75-3. Here, the file recording program 75-1 and the tape recording program 75-2 each operate in a multi-task manner, and execute recording of search data, etc. to the VTR or file independently of the list/playback program 76. There is. Note that the character recognition (OCR) program 75-3 operates as a subroutine of the file recording program 75-1. Incidentally, in this embodiment, a news program is taken up as an example of a TV program because the screen display of example characters as a news title is large, the type of characters is almost constant, and the characters are easy to recognize. FIG. 8 is a diagram showing a screen when a startup command for the news search software 73 is input. FIG. 9 shows a screen display when the news search software 73 is started after the power of this device is turned on. Immediately after startup, the display on the CRT display 6 is in the computer display mode on the workstation side, and the process until the "News List" display shown in FIG. 9 is reached is as follows. First, When the news search software 73 is started, the VTR
It is determined where the rotary head of the VTR is located on the recorded program on the tape, and at that position, the identification number recorded on the audio (monaural) band on the tape is read. Second, a record file corresponding to the identification number is searched and read from the external storage device l2, and third, search data including character examples in the record file is displayed on the CRT display 6, as shown in FIG. The “news list” shown in
Proceed as follows. Further, 80 is a character cursor, and 8l is a field for explaining various key operations, and the user starts various operations according to these explanations. For example, if the number of items listed in Figure 9 is too large and the user cannot easily find the desired title item by simply going through the list page by page by pressing the "Next item: 2" key, the user can select "Title search". ＝S″
Push the key. Then, the title search screen shown in FIG. 10 appears. As shown in the figure, the search items are:
There are three locations: a number 82, an in-record counter 83, and a title field 84. Then, when the user sets the desired items and presses the "search execution=W" key, if there is data that even partially matches each of the set items, all the data is extracted and displayed. An example of the display result is shown in FIG. In FIG. 10, the user selects the title field 84
This is a case where a search is instructed for data containing the word "cherry blossoms", and as a result, three news titles are searched and displayed: "Yaezakura Festival", "Sakura Viewing Party", and "Cherry Blossom Front", as shown in FIG. Even in the state shown in Fig. 11, if the number of items listed is too large, if the user presses the "Recheck: S゜key", the title search screen shown in Fig. 10 will be displayed again, and if the search is executed, the title search screen in Fig. 11 will be displayed. You can further narrow down the list number shown in the figure. This re-examination can be repeated as many times as you like. Also, use the "Restore list: ESC" button shown in operation explanation column 8l in Figures 10 and 11. The key is used to return to the "News list" display screen shown in FIG. 9. Similarly, the "End: ESC" key shown in FIG. Display the 8th
This key is used to return to the graphical display of the OS shown in the figure. By the way, when the user pushes the video display button 41 on the remote control 26 in the state shown in FIG. 9 or FIG. 11, the VTR enters the video playback image display mode. next,
When the user presses the "search next candidate" button in area 46 on the remote controller 26, the screen shown in FIG.
1, the rotary head automatically moves to the position of the recording counter value on the VTR tape indicated by the character cursor 80, and a still image is displayed at that position. Furthermore, when the user pushes the "search next candidate" button, the character cursor 80 in FIG. 9 or 11 changes to
At the same time as moving to the position of the next data number on the list, the rotary head also automatically moves to the position on the VTR tape showing the next data, and a still image of the recorded news program is displayed. There is data on the list, and the search display of still images on the CRT display 6 continues each time the user presses the "Search Next Candidate" button. Note that when the user presses the "Pre-search candidate" button on the remote control 26, the character cursor 80 moves to the position of the previous data number on the list shown in FIG. 9 or FIG. At the same time, the V pointing at that position
The rotary head moves to the position of the recorded counter value on the TR tape and displays a still image of the recorded news program. After this, each time the user presses the "Pre-Search Suggestions" button, the character cursor on the list will move to 8 as long as the list continues.
0 moves to the previous data, a search is made for the corresponding position on the VTR tape, and a still image is displayed. In addition, the user uses the play button, rewind button, fast forward button, slow button, etc. in area 45 on the remote control 26 from the above-mentioned still image display state to play back the news program recorded on the VTR tape. be able to. When the playback operation by the user is completed, the user presses the computer display button 42 on the remote control 26, and the display on the CRT returns to the "news list" display shown in FIG. 9 or FIG. 11. Of course, the above-mentioned pressing of the "Next Search Candidate" button or the "Previous Search Candidate" button after pressing the video display 4l button is omitted, and the user can suddenly press the play button and start playing the VTR tape. However, in this case, please refer to Figure 9 or 10.
VT to the character cursor position 80 displayed in the list in the figure
There is no automatic movement of the rotating head on the R tape. Furthermore, although it has been explained that "when you push the next search candidate button or the previous search candidate button, the character cursor 80 moves on the list," when the CRT display 6 is in the video display mode, the character cursor 80 moves at the same time. It is not possible to check the movement directly on the CRT display 6. In other words, if the user
2 button to change the display of the CRT display 6 to computer display mode and display the screen shown in FIG. 9 or 11.
The movement of the character cursor 80 can be confirmed for the first time when the state shown in the figure is returned. Next, FIG. 12 is a diagram illustrating the relationship between the recorded content on the VTR and search data. FIG. 12 shows that the user selects the television display 40 on the remote control
When recording a news program by pressing the button to display the TV signal on the CRT display 6, press the file recording ON/OFF button in area 43 on the remote controller 26. 1
0N'' state, the temporal positional relationship of the audio (monaural) band of the VTR tape and the search data written to the recording file in the external storage device 12 is explained. While you are there, listen to the audio on the VTR tape (
In the monaural) band, the counter value and identification number in the record are recorded every second, while in the record file, the results of OCR analysis of the character display of the news program are recorded in the record at that time at intervals of about 5 seconds. Recorded together with the counter value. In this case, if the OCR analysis result from 5 seconds ago matches the current OCR analysis result, the recording file will contain the following information:
It is configured not to record, and there are no duplicate records in the recording file. Then, the search data in the recorded file (recorded counter values and OCR analysis results) is displayed as a "news list" shown in FIGS. 9 and 11. Description of Processing Procedure> Next, the processing procedure in this embodiment will be described below with reference to the flowcharts shown in FIGS. 13 to 17. FIG. 13 is a flowchart showing an outline of the operation from startup of the device and various processes of the "news search software" to power-off. First, in step S1, when the user turns on the power of this device, the VTR device main body 20 and each part on the workstation side are started up (step S2
, S3). Next, in step S4, the signal switching device 24
is switched to computer display, and the news search software 73 is started in step S5. Then, in step S6, each process is executed, and the end of the process is determined in step S7. Here, if there is a key human power to stop the process, the process advances to step S8, ends the running program, and turns off the power to each device (step S9, SIO).
．． FIG. 14 is a flowchart illustrating the operation of the list/playback program 76 included in the news search software 73. As shown in FIG. 8, when the startup command fNEWssRcJ of the news search software 73 is input, the list/playback program 76 starts operating, and the recording program 75 is started from within the list/playback program 76 (step S11). Note that, as described above, the recording program 75 includes the file recording program 75-1 and the tape recording program 75-1.
2. Consists of a character recognition (OCR) program, including a file recording program 75-1 and a tape recording program 7.
5-2 operates with multitask formation, and after startup, the list/
It operates independently of the playback program 76. However, the command to terminate the recording program 75 is issued from within the list/playback program 76 (step S18). Step S15
``News list/display'' corresponds to FIG. 9, and ``Title search screen display'' in step S2l corresponds to FIG. 10.
"Search result display" in step S25 corresponds to FIG. 11, respectively. Note that when the user pushes the video display 4l button on the remote control 26 while waiting for the user's key input in step S16 or step S26, the display on the CRT display 6 changes to the video display mode for VTR playback. Here, when the user performs VTR playback processing using the playback, rewind, fast forward buttons, etc. on the remote control 26, and then pushes the computer display 42 button on the remote control 26, the display on the CRT display 6 changes to step S16 or step The screen returns to the news list display in S26, and returns to the state of waiting for the user's input. Next, FIG. 15 is a flowchart illustrating the processing of the file recording program 75-1 included in the recording program 75. This program is started from the list/playback program 76 as described above, and also
A termination command is received from (step S31). and,
The user operates the remote control 26 to watch TV news VT.
When R recording starts (step S32), it is determined whether the file recording mode is ON (step S34). Here, YE
If S, create a new recording file (step S35)
, reset the recording counter, etc. (step S3)
6). Recording is started (step S37). Then, every 5 seconds, the character display in the news video is analyzed (steps 538 to 343), and the character recognition results are recorded in a file in the external storage device 12 (step S44). When the end of recording is commanded (step 846), the recording file is closed (step S47) and the process ends. Note that the program of steps S39 to S41 is a subroutine included in the character recognition program 75-3. Processing such as character region extraction using the ``edge extraction method'' or character recognition using ``pattern matching'' is a general technique that is described in many literatures related to character recognition, and will not be explained here. FIG. 16 is a flowchart illustrating the operation of the tape recording program 75-2 included in the recording program 75. This program is also the file recording program 75-
1, it is started from the list/playback program 76,
A termination command is received from the list/playback program 76 (step S51). As a process, when the user starts recording TV news on a VTR using the remote control 26 (step S5
2) Resetting the recording counter and recording the identification number, etc. (steps S55, 856). Then, every second (step S57), the internal recording counter value and the identification number are recorded in the audio (monaural) band on the VTR tape (step S59). Thereafter, in response to a command to end recording (step S58), a predetermined end mark, etc. is recorded on the VTR tape, and the process ends (step S60).
. FIG. 17 is a flowchart for explaining the operation of the VTR controller 22. In this program, the user uses the remote control 26 to
It shows each process when each of the TV display 40 button, video display 4l button, and computer display 42 button is pushed. Here, the user presses the TV display 40 button (step S6).
2) When one of the video display 41 button (step S65) or the computer display 42 button (step S68) is pressed, the display on the CRT display 6 is switched to the designated display (steps S63, S66, S6).
9) Then, the user issues instructions regarding TV display recording (step S64), video playback (step S67), or computer processing on the remote controller 26 (
Step S70) are respectively processed. Note that in step S64, step S67, or step S70 of this flowchart, instructions from a large number of users are described.
button, the video display 4l button, or the computer display 42, the process proceeds to step S6.
The processing continues as if it had returned to immediately before step 2. As explained above, according to this embodiment, when a television program, for example, a news program, is recorded on a VTR on a personal computer or workstation, at the same time,
This has the advantage of being able to analyze the characters displayed in a program and record the character data resulting from the analysis into a file in an external storage device. Furthermore, when the user later wants to play back a recorded program on a VTR, the user can search for character data recorded in a file in the external storage device, select the desired scene on the list, Based on the search data, the head can be automatically moved to the playback start position on the VTR tape and playback can be executed. In addition, in this embodiment, a news program was explained as an example of a television program, but the text captions used for news are large in font size, and the type of characters is almost constant, making it easy to recognize the characters. This is because. In the above example. Although a news program was taken up as an example of a TV program, foreign movies with Japanese subtitles may also be targeted. In that case, each performer's lines are subjected to character recognition processing and stored. However, the drawback in this case is that after recording. Even if an attempt is made to display a list of lines on the CRT display 6 and search for the starting position of VTR tape playback, the line that will surely evoke the desired scene is not always arranged. Note that the time interval for character recognition processing is every 5 seconds in the above-described embodiment, but in the case of a movie, it is every 2 seconds. This is because scenes change faster in movies than in news programs. [Effects of the Invention] As explained above, according to the present invention, by character-recognizing the character information of a television program and recording the results,
This has the advantage that recorded scenes can be easily searched and played back.

[Brief explanation of the drawing]

FIG. 1 is a schematic block diagram showing the structure of the image recording and display device in this embodiment, FIG. 2 is an external view of the image recording and display device in this example, and FIG. 3 is an external view of the remote control in this example. Fig. 4(a) is a diagram explaining the relationship between the rotating head and the VTR tape, Fig. 4(b) is a diagram showing the signals recorded on the VTR tape, and Fig. 5 is a display example of the VTR message display section. Figure 6 is a schematic block diagram showing the configuration of the VTR controller, Figure 7 is a diagram showing the software configuration, Figure 8 is a diagram explaining input of the start command of news search software, Figure 9 is a news list. Figure 10 is a diagram showing the title search screen, Figure 11 is a diagram showing the search result screen, Figure 12 is a diagram explaining the relationship between the recorded content on the VTR and search data, and Figure 13 is a diagram showing the screen. Figure 14 is a flowchart of the startup procedure, Figure 14 is a flowchart of the list/playback program, Figure 15 is a flowchart of the file recording program, Figure 16 is a flowchart of the tape recording program, Figure 17 is a flowchart of the VTR controller program. be. In the figure, 1...CPU, 2...Memory, 3...RO
M, 4...RAM, 5...VRAM, 6...CR
T display, 7...Left speaker, 8...Right speaker, 9...Printer, 10...Keyboard, 11
...Mouse, 12...External storage device, l3...Floppy disk device, l4...Hard disk device, 20...VTR device body, 2l...Channel selection circuit, 22...VTR controller , 23... signal conversion circuit, 24... signal switching device, 25... modem, 26... remote control, 27... light emitting unit, 28...
・It is a light receiving part. Patent applicant Canon Inc. No. 42 (0) 5l Figure 4 (b) Figure 5 Figure 7 Whistle 8Ll1 Figure 15 (b)

Claims (2)

[Claims] (1) An image recording and display device having a recording/playback function for television programs, etc. and a character recognition function, comprising a character recognition means for recognizing characters from character signals included in scenes of the television program; and the character recognition means. An image recording/displaying device comprising: a recording means for recording a character code recognized by the recording means; and a display means for displaying according to the character code recorded by the recording means. (2) The image recording apparatus according to claim 1, wherein a desired scene is searched and reproduced based on a character code on the display means.