JP3960646B2 - Information storage device and information storage method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an information storage device and an information storage method for storing and storing information such as conversation voices in conferences and interviews and images of conferences and interviews, such as conference record recording systems and interview recording systems.
[0002]
[Prior art]
When recording key points such as conferences, lectures, interviews, interviews, conversations using telephone and videophone, and video images, recording is usually performed by a recorder. For example, when recording a meeting, one of the attendees of the meeting serves as a clerk and records the remarks of all the meeting participants one by one, or selects and records only important items. However, there are often situations where it is difficult to create meeting minutes later because the notes that you wrote down do not recall the details of the meeting or how it was spoken, or you did not record who spoke. .
[0003]
Therefore, records such as conferences, lectures, interviews, interviews, telephone and videophone conversations, TV images, surveillance camera images, etc. have been stored and stored in digital disks, digital still cameras, video tapes, semiconductor memories, etc. An apparatus for reproducing has been proposed. When information is stored using these information storage devices, it is possible to record the input information, such as voice and images, without leaking, compared to the method in which the recording person writes only the main points of the information to be recorded. There is.
[0004]
These devices include those that record a digital signal transmitted via a computer network on a storage medium, those that record an analog input signal from a video camera or microphone as it is on a storage medium, or those that are encoded into a digital signal. There are things to convert and record.
[0005]
However, there is a problem that it is difficult to record a long-time audio signal and / or image signal in a storage medium having a limited recording capacity. This is because, in general, the storage capacity required for recording time-series data such as sequentially input audio signals or image signals over a long period of time becomes enormous.
[0006]
To solve this problem, for example, as in Video for Windows ("Microsoft Video for Windows 1.0 User's Guide" pp. 57-59, pp. 102-108), audio signals and image signals are always compressed and stored in a storage medium. A method of storing has been proposed. In that case, however, all input audio signals or image signals are generally stored at the same compression rate. For this reason, there is a problem that a relatively large amount of relatively unimportant information that is less likely to be referenced after recording is recorded, or that the information cannot be recorded in good quality due to the storage capacity despite the important information. It was.
[0007]
For example, when interview scenes are recorded for a long time using the Video for Windows, the compression rate is set so that only one frame of image signals is stored for 5 seconds in order to save storage capacity. Suppose that At this time, even if the recording person wants to reproduce the part that is important at the time of recording later, only one frame of image signal can be reproduced in 5 seconds. ), Talking, and subtle nuances cannot be reproduced.
[0008]
However, conversely, when trying to store all input image signals in 30 frames per second, storing a long interview requires a huge amount of storage capacity, which is very difficult to implement. It is.
[0009]
Therefore, Japanese Patent Application Laid-Open No. 5-64144 and Japanese Patent Application Laid-Open No. 5-134907 disclose an old frame of image information already stored when the amount of data stored in the image storage medium exceeds a predetermined amount. An information storage device that attempts to save storage capacity by sequentially compressing or thinning out frames is described. By regarding the information stored later as important information, the previously stored information is overwritten with new input information, or the information stored earlier is increased in compression rate. It is a device that saves storage capacity.
[0010]
Japanese Patent Laid-Open Nos. 2-305053 and 7-15519 recompress audio information that has already been stored when it is recognized that the free capacity of the storage medium has fallen below a certain amount. Thus, an audio information storage device that secures a free space in a storage medium is described.
[0011]
In addition, the moving image recording apparatus described in Japanese Patent Laid-Open No. 6-149902 performs automatic scene change detection, and considers that a longer scene is a more important scene. This is an apparatus that extracts in order from scenes having a high importance so as to be long.
[0012]
If only the scene included in the digest generated by the apparatus described in this publication is left and the scene not included in the digest is deleted, the storage capacity can be saved without losing important information.
[0013]
On the other hand, Japanese Patent Application Laid-Open No. 3-90968 and Japanese Patent Application Laid-Open No. 6-149902 have proposed devices that automatically generate a video digest so as to have a time length specified by a user. In the apparatus described in Japanese Patent Laid-Open No. 3-90968, when a user inputs an importance level for each scene in advance from an editor and generates a digest, the high importance level is set so that the time length specified by the user is reached. This is an apparatus for extracting in order from a scene having Also in this apparatus, if only the scene included in the generated digest is left, the storage capacity can be saved without losing important information.
[0014]
[Problems to be solved by the invention]
However, the devices described in Japanese Patent Laid-Open Nos. 2-305053 and 7-15519 are devices that re-compress stored audio information at the same compression rate throughout and record it. There is a problem that it is not possible to record the high-quality sound by partially reducing the compression rate of only the important part of the content.
[0015]
Further, in an information storage device for storing and storing conferences, lectures, interviews, interviews, etc., as described in Japanese Patent Laid-Open No. 5-64144 or Japanese Patent Laid-Open No. 5-134907, new records are simply recorded as important information. If the old record is erased as unnecessary information, the record of important meetings and important interviews will be overwritten by new input information just because it was recorded first. There is. This is because, in general, it is not possible to determine the importance of the content of the meeting or the content of the coverage based only on the date and time when the conference or the coverage was held.
[0016]
In addition, regarding the device described in Japanese Patent Laid-Open No. 6-149902 that determines the importance of a scene based on the length of the scene, when a meeting or lecture is taken with an unmanned camera, the scene can be changed by a cut change or a scene change. There is a problem that it is very difficult to cut and the length of the scene cannot be detected. In addition, if you are shooting a meeting or lecture, important statements may be included even in a short scene, so based on the length of the scene alone, There is a problem that the importance cannot be determined.
[0017]
Furthermore, the device described in Japanese Patent Laid-Open No. 3-90968, in which the user inputs the importance level for each scene from the editor in advance, can be used by a cut change or a scene change when a conference or a lecture is taken with an unmanned camera. There is a problem that it is very difficult to separate scenes. In addition, the task of inputting the importance level from the editor after shooting is very troublesome, and there is a problem that it is not suitable for the purpose of recording a meeting or a lecture.
[0018]
By the way, as a known technique, there is known an apparatus that performs selection of information at the time of recording and records only information recognized as important or records by changing the compression rate. For example, Japanese Patent Application Laid-Open No. 7-129187 describes an apparatus that records audio before and after a voice capture key is pressed for a predetermined time. In addition, some commercially available tape recorders have a silent section detection function in which silent sections do not store voice.
[0019]
However, since these devices do not have means for re-compressing information once recorded, the compression rate is changed stepwise depending on the length of the information storage period, or the free storage capacity of the storage medium is reduced. There has been a problem that the compression efficiency cannot be changed dynamically according to the change, and the compression efficiency is very poor compared to the method of recompressing the stored image or audio information.
[0020]
Further, in the method of selecting information in real time at the time of recording as in the apparatus described in Japanese Patent Application Laid-Open No. 7-129187, for example, the person who has made the most speech in the conference is identified, and the speech of the identified person is identified. For example, to save only part of audio information or image information with high quality, or to create a digest by extracting in order from the scene with high importance so that the time length specified by the user is reached I can't. In other words, the device described in this publication cannot compress audio information or image information based on information obtained for the first time after recording of audio information or image information, or information that cannot be obtained while recording. There is a problem.
[0021]
Further, as described in Japanese Patent Laid-Open No. 7-129187, in order to record time series information just before the trigger is detected, the recording is performed for temporarily recording the input time series information. Since a buffer memory is required, there is a problem that the apparatus is complicated and expensive.
[0022]
In view of the above-described problems, the present invention is a storage medium having a limited storage capacity for only an audio signal or an image signal in an important period in which a characteristic event occurs among input audio or image signals. The purpose is to store a large number of audio signals or video signals other than the important part, and to store them for a long time with a small amount of data, and to ensure that the important part is reproduced from the beginning to the end. Yes.
[0023]
[Means for Solving the Problems]
In order to solve the above problems, in the information storage device according to the present invention,
Information input means for inputting audio information and / or image information to be stored;
The audio information and / or the image information input from the information input means Compress Time series information storage means for storing;
Condition matching section detection means for detecting a condition matching section in which the voice information stored in the time series information storage means matches a predetermined condition set in advance;
Time information indicating the time when the audio information and / or the image information is stored in the time-series information storage unit is stored, and an elapsed time from the time indicated by the stored time information is set in advance. Elapsed time measuring means for outputting a compression processing start instruction when the time is over,
The time series is started by the compression processing start instruction from the elapsed time measuring means, and the condition matching section detected by the condition matching section detecting means and another section are changed in compression rate or compression method. Stored in the information storage means The amount of compressed audio information and / or image information is Compression means for compressing;
It is characterized by providing.
[0024]
The condition coincidence detection operation in the condition coincidence section detecting means can be performed when the voice information is stored in the time series information storage means, or the time series information is compressed when the stored information in the time series information storage means is compressed. Audio information can also be read from the storage means.
[0025]
In the former case, as in the invention of claim 2, section information indicating the condition matching section detected by the condition matching section detecting means, and the time series of the audio information or the image information corresponding to the section information Correspondence relation storage means for storing correspondence relations with storage positions in the information storage means is provided.
[0026]
The compression process can be started at an arbitrary time after the audio information or the image information is stored in the time-series information storage means, but can also be automatically executed at a predetermined time. In that case, as in the invention of claim 12, there is provided time information storage means for storing time information indicating the time at which audio information or image information is stored in the time-series information storage means, and the compression means When the elapsed time from the time determined by the time information stored in the storage means becomes equal to or longer than a predetermined time, the compression process is executed.
[0027]
Further, the compression can be executed when the used capacity of the storage medium of the time-series information storage means exceeds a predetermined amount, or when the free capacity of the storage medium becomes less than the predetermined amount.
[0028]
[Action]
In the present invention having the above-described configuration, the audio information or the image information once stored in the time-series information storage unit is compressed at a later time to reduce the use capacity of the storage medium. Audio information or image information in a condition matching section that matches the predetermined condition is compressed with high quality. For this reason, it is possible to reduce the used capacity of the time-series information storage means while preserving important information with high quality.
[0029]
【Example】
Embodiments of the present invention will be described below with reference to the drawings.
[First Embodiment]
The first embodiment is a case where the information storage device according to the present invention is applied to conference recording.
[0030]
In general, it is very unlikely to refer to a meeting that took place a month ago compared to the possibility of referring to a meeting that took place a few days ago. Keeping conference information such as video information that is less likely to be referenced in high quality is extremely inefficient in terms of saving memory space, and information is stored at an appropriate timing. It is desirable to reduce the amount of information by performing deletion or thinning compression.
[0031]
However, even in the case of old conference records, there are demands to reproduce the movements made by the speaker while speaking (gestures, etc.), talking, and subtle nuances. Therefore, it is required that an audio signal or an image signal in an important period in which such a characteristic event occurs is stored in high quality.
[0032]
In the first embodiment, audio information and video information are recorded for a conference, and when one month has elapsed from the recording time, the conference that was actively discussed in the recorded conference video is important. A description will be given of an example in which compression processing is performed in which only the video of a part is left in high quality and the other part is compressed at a high compression rate.
[0033]
According to the first embodiment, as will be described later, when a video of a part that has been actively discussed is reproduced, a high-quality moving image with smooth motion is reproduced, and the other part is When it is played back, it becomes a so-called frame dropping and a moving image with awkward movement. However, since the insignificant part other than the scene in which discussions were actively exchanged can be compressed at a high compression rate, the amount of information to be stored and stored becomes very small.
[0034]
FIG. 2 shows a meeting scene in the case of this embodiment. A microphone 15 is provided for each of a plurality of conference participants 16, and the uttered voice of each conference participant 16 is the respective microphone 15. Sound is picked up. The video camera 17 captures a scene of discussion among a plurality of conference participants 16 and a paper document provided as conference material.
[0035]
Reference numeral 10 denotes an information storage device. In the case of this embodiment, the information storage device 10 includes an audio signal analyzer 11, a conference information storage processing device 12, a storage medium 13, and a playback unit 14.
[0036]
The conference information storage processing device 12 stores the image signal from the camera 17 that captures the conference scene and the audio signal from the microphone 15 in a storage medium 13 made of, for example, a disk or a semiconductor memory. The audio signal or the image signal stored in can be compressed. In this embodiment, in order to make the explanation easy to understand, the conference information storage processing device 12 compresses only the image information stored in the storage medium 13 and does not compress the audio information stored in the storage medium 13. To do. The conference information storage processing device 12 may be a personal computer.
[0037]
The conference information storage processing device 12 includes an audio input terminal and an image input terminal. In this embodiment, the speech signals of the speeches of a plurality of conference attendees 16 collected by the microphone 15 are once input to the speech signal analyzer 11, and the output of the speech signal analyzer 11 is stored as conference information. Input to the audio input terminal of the processing device 12.
[0038]
The audio signal analyzer 11 analyzes the audio signal input from the plurality of microphones 15, identifies which microphone the input audio signal is input from, and outputs the identification result together with the audio signal to the conference information storage processing device 12. Is output.
[0039]
Also, the paper document photographed by the camera 17 and the image signal of the meeting scenery are input to the image input terminal of the meeting information storage processing device 12.
[0040]
The conference information storage processing device 12 includes a user interface (not shown). In response to a playback request from the user via this user interface, an image based on the image signal stored in the storage medium 13 is displayed on the playback unit 14. In addition to being displayed on the screen, it also has a function of emitting reproduced sound based on the sound signal stored in the storage medium 13 from a speaker attached to the playback unit 14.
[0041]
When the conference information storage processing device 12 is configured by a personal computer, the conference audio information and image information are simultaneously shared between remote locations by connecting to the ISDN network via this personal computer, for example. It is also possible to realize an environment where a meeting is held in the same room.
[0042]
FIG. 1 is a block diagram showing the information storage device 10 of this embodiment, focusing on its functions. That is, the information storage device according to the present embodiment includes a voice information input unit 1, an image information input unit 2, a condition matching interval detection unit 3, a time series information storage unit 4, a correspondence relationship storage unit 5, The compression unit 6, the time information storage unit 7, the reproduction unit 8, and the control unit 9 are connected to each other. In this example, the voice information input unit 1 is also connected to the condition matching section detection unit 3. The control unit 9 controls the entire processing operation.
[0043]
Each part may be configured as a separate block, or one block may be configured to include several parts. Moreover, one part may be divided into several blocks and mounted.
[0044]
The voice information input unit 1 receives the voice signal from the microphone 15, converts it into a digital voice signal, sends it to the system bus, and sends it to the condition matching section detection unit 3.
[0045]
The image information input unit 2 receives an image signal from the video camera 17. If the image signal from the video camera 17 is a digital signal, it is received and sent to the system bus. If the input image signal is not a digital signal, the image information input unit 2 converts the input image signal into a digital image signal and outputs it to the system bus.
[0046]
The audio signal from the audio information input unit 1 is supplied to the condition matching section detection unit 3 as a digital signal. The condition matching section detection unit 3 monitors a voice signal input thereto, and detects a voice section that matches a predetermined condition. In this embodiment, a condition matching section is detected on the condition that there is an input of an audio signal of a predetermined level or higher and an active conversation pattern is detected from the input audio signal. Thereby, the section in which the conference participants actively exchanged discussions is detected as the condition matching section. In the condition matching section detection unit 3, the audio signal analyzer 11 and a part of the conference information accumulation processing unit 12 play a role.
[0047]
As a method for detecting the presence / absence of an audio signal of a predetermined level or higher, as shown in FIG. 3, the condition matching section detection unit 3 detects that the input audio level is higher than the predetermined level and detects a speaker. And a detection function for recognizing the end point of the speaker's speech by detecting that the voice level is below a predetermined threshold level.
[0048]
However, as shown in FIG. 3, if the speech level change point F101 itself at which the speech level intersects the threshold level is the speech start point or end point, the first part and the last part of the speech are included in the condition matching section. Therefore, when the speech level changes from the high level to the low level, the time point F100 before the change point F101 when the audio level changes from the low level to the high level is set as the speech start point, and the audio signal level changes from the high level to the low level. The point of time F102 after a certain time T2 from the change point F101 of the above is defined as the speech end point.
[0049]
In this embodiment, the sound level at a certain time is a value obtained by smoothing the sound level before and after the time, for example, an average value of instantaneous sound levels for 2 seconds before and after the certain time.
[0050]
In this embodiment, as shown in FIG. 2, a microphone 15 is installed for each speaker, and the voice signal analyzer 11 compares the voice input level from the microphone of each speaker. 11 specifies the speaker who has transmitted the input voice signal.
[0051]
In addition to this, as a method for specifying a speaker, a speaker may be specified from the characteristics of an audio signal (such as a voiceprint), or the speaker may be specified from a face or mouth movement based on image information. . In that case, it is not necessary to provide a plurality of microphones corresponding to all the meeting attendees, and one or a plurality of microphones smaller than the number of meeting attendees may be used. It is also possible to install a plurality of microphones, analyze the phase difference of the audio signals input from these microphones, detect the position of the sound source, and specify the speaker.
[0052]
The condition matching section detection unit 3 determines that the active conversation is performed as the time from when one speaker ends speaking until the other speaker starts speaking is shorter. Also, if another speaker starts speaking before one speaker finishes speaking, it is determined that an active conversation is taking place.
[0053]
FIG. 4 illustrates a process in which the condition matching section detection unit 3 recognizes a section where conversation is active. This figure shows a case where it is determined that an active conversation is taking place as the time from when one speaker finishes speaking until another speaker starts speaking is shorter. A speech signal of a predetermined level or more from each speaker is recognized as the speech section SP of the speaker, and the speech section SP is short among a plurality of speakers so as to be surrounded by a dotted circle in FIG. Patterns that change over time are detected as active conversation patterns.
[0054]
The condition matching section detection unit 3 detects the pattern in which the speaker is changed in such a short time in this manner, and the speaker within a predetermined set time after one speaker finishes speaking. Detect if has changed. For example, the set time is 0.5 seconds. This set time may be changed by the user.
[0055]
In this embodiment, a pattern in which speech sections SP partially overlap because another speaker starts speaking before one speaker finishes speaking is also detected as an early speaker change pattern.
[0056]
Then, the condition matching section detecting unit 3 recognizes a section where the conversation is active depending on whether the pattern of early speaker change has continued a predetermined number of times, for example, three times or more. For example, in the example shown in FIG. 4, since the pattern of early speaker change continues four times in the section PP, this section PP is detected as an active section of dialogue. That is, the beginning F200 of the speech section including the early speaker change pattern that continues four times is set as the starting point of the active conversation section, and the end F201 of the speech section including the early speaker change pattern is set as the active conversation. The end point of the section.
[0057]
The time-series information storage unit 4 is a storage unit that accumulates audio information and image information, and uses, for example, a disk storage medium or a semiconductor memory as a storage medium.
[0058]
The correspondence relationship storage unit 5 includes each of the active conversations detected by the condition matching interval detection unit 3, and voice information and images stored in the time-series information storage unit 4 corresponding to the respective active conversations. The information is stored in association with the storage address in the time-series information storage unit 4. The correspondence relationship storage unit 5 is also composed of, for example, a disk storage medium or a semiconductor memory.
[0059]
In this embodiment, the compression unit 6 performs data compression of the image information stored in the time series information storage unit 4. In this case, the compression unit 6 is configured to be able to dynamically change the data compression rate or the data compression method based on the information indicating the condition matching section from the correspondence relationship storage unit 5.
[0060]
In this embodiment, the compression unit 6 assumes moving image information, and handles the moving image information as a single processing unit with a predetermined time length or a predetermined number of frames. For example, compression processing is performed on a continuous 10-frame image sequence as one unit partial image sequence, but the image information of the sections other than the condition matching section leaves only the first frame in the 10 frames, The thinning compression process of discarding information of other frames is performed. On the other hand, in the condition matching section, the thinning process for image information is not performed, and all the 10 frames are stored.
[0061]
Therefore, when the image information of the sections other than the condition matching section is reproduced, the moving image is a so-called frame dropping and awkward movement, but the amount of information is very small. On the other hand, when the image information in the condition matching section is reproduced, a high-quality moving image with smooth movement is reproduced.
[0062]
The time information storage unit 7 is used to store the time when the input audio signal and image signal are recorded in the time-series information storage unit 4, and is configured by, for example, a disk storage medium or a semiconductor memory.
[0063]
Further, the time information storage unit 7 has a function of measuring an elapsed time from the recording start time. Therefore, the time information storage unit 7 is supplied with current time information from a clock circuit unit (not shown). In this embodiment, the time information storage unit 7 uses the compression unit 6 to store the image of the time series information storage unit 4 when the elapsed time from the recording start time is equal to or longer than a predetermined time. A compression trigger timing signal that triggers the start of compression of information as described above is output.
[0064]
The reproduction unit 8 is a functional unit that reproduces an audio signal and an image signal stored in the time-series information storage unit 4 by the monitor device 14 of FIG.
[0065]
The control unit 9 controls the overall processing operation of the information storage device 10.
[0066]
[Operation during recording]
Next, an operation during recording in the information storage device 10 having the above configuration will be described. FIG. 5 is a diagram for explaining the recording operation in this embodiment, together with the flow of various information and the flow of output of each unit at that time.
[0067]
When the conference starts and the audio signal from the microphone 15 and the image signal from the camera 17 are supplied to the information storage device 10, the audio signal and the image signal are sequentially stored in the time-series information storage unit 4. The audio signal is input to the condition matching section detection unit 3.
[0068]
As described above, the condition matching section detection unit 3 compares the voice level of the voice information from the microphone 15 with a predetermined threshold level, detects the speech start point and the speech end point of the conference attendee, Is set as a speaker's speech section SP. Then, a short turn or partial overlap between a plurality of conference attendees in the speech section SP is detected, and a section where conversation is active is detected as a condition matching section. Then, information on the start point and end point of the detected condition matching section is supplied to the correspondence storage unit 5.
[0069]
FIG. 6 is a flowchart for explaining the operation of the condition matching section detection unit 3.
[0070]
When the voice signal from the voice information input unit 1 is supplied as a digital signal to the condition matching section detection unit 3, in step S100, the above-described speech section SP is detected and the speaker is specified. As described above, the speaker identification method is performed by comparing the voice input level from the microphone installed for each speaker with the voice signal analyzer 11.
[0071]
After step S100, in step S101, when a pattern in which a speaker is changed in a short time including a partial overlap is recognized and an early speaker change pattern is detected, the process proceeds to step S102. It is determined whether or not the pattern has continued for a predetermined number of times. As described above, when an early speaker change pattern is detected three or more times in succession, a condition is set in advance so as to recognize a speech section including the pattern as an active conversation section. If it has been set, in the example of FIG. 4 described above, the section PP is detected as a section where the conversation is active, and the process proceeds to step S103.
[0072]
In step S103, the section detected as the section where the conversation is active is specified as the condition matching section. That is, in the example of FIG. 4, for example, the beginning of the section where the dialogue is active is the beginning F200 of the section PP, the end of the section where the conversation is active is the end F201 of the section PP, and the section PP is the section where the dialogue is active ( (Condition matching section). Note that the information specifying the condition matching section may be information on one of the beginning or end of the section and information on the length of the section.
[0073]
Subsequently, in step S104, the condition matching section specified in step S103 is output to the correspondence storage unit 5, and then the process returns to step S100 to start detection of a new condition matching section. In step S102, when it is recognized that the pattern of early speaker change is equal to or less than the predetermined number of times, the process returns to step S100 and detection of a new condition matching section is started.
[0074]
When the correspondence relationship storage unit 5 receives information specifying the condition matching section from the condition matching section detection unit 3 as described above, that is, in this example, information on the start and end of the condition matching section, it will be described below. In this way, these pieces of information are stored in association with the storage addresses of the time-series information storage unit 4 corresponding to the condition matching section.
[0075]
FIG. 7 is a flowchart for explaining the operation of the correspondence storage unit 5. In FIG. 7, the steps involved in the above-described recording operation are steps S200 to S203. Steps S204 and S205 are related to the operation at the time of compression described later.
[0076]
That is, at the time of recording, in step S200, it is detected whether or not the information indicating the condition matching section is input from the condition matching section detection unit 3. If the input of the condition matching section is not detected, The process returns to step S200 via S204, and the presence or absence of input of information indicating a condition matching section is detected.
[0077]
If the input of the condition matching section from the condition matching section detection unit 3 is detected in step S200, the process proceeds to step S201. In step S201, in order to obtain the storage address in the time series information storage unit 4 of the voice information or the image information stored in the time series information storage unit 4 corresponding to the condition matching section, the time series information storage unit 4, a storage address inquiry request is output together with information indicating the condition matching interval, and a response is awaited in step S202.
[0078]
When a response from the time series information storage unit 4 is returned, the process proceeds to step S203, where information specifying the condition matching section, voice information stored in the time series information storage unit 4 corresponding to the condition matching section, and The image information is stored in association with the storage address in the time-series information storage unit 4.
[0079]
After step S203, the process returns to step S200 via step S204, and the presence or absence of input of information indicating the next condition matching section is detected.
[0080]
FIG. 8 shows a storage address in the time-series information storage unit 4 of the condition matching section which is the detection result of the condition matching section detection unit 3 and the voice information and the image information input at the time specified by the condition matching section. FIG.
[0081]
As shown in FIG. 8, the start time F200 of the section where the conversation is active corresponds to the storage address F300 of the time-series information storage unit 4, and the end time F201 of the section where the dialog is active is the storage address F301. It corresponds to.
[0082]
The correspondence relationship storage unit 5 stores this correspondence relationship in the form of a table as shown in FIG. In this example, the start time F200 and the end time F201 are relative times starting from the storage start time.
[0083]
For example, as shown in the example of FIG. 9, the start of the active section of the dialog is detected at 210 seconds after the start of the storage of the voice information and the image information, and the active section of the dialog is also at the time of 240 seconds. If the end of is detected, audio information and image information stored between 210 seconds and 240 seconds after the start of the storage become corresponding storage information. In the example of FIG. 9, addresses 420 to 480 of the time-series information storage unit 4 are storage addresses for audio information and image information corresponding to this condition matching section.
[0084]
In FIG. 9, ID is an identifier for identifying each of the detected condition matching sections, and is a three-digit number in this example.
[0085]
Note that the storage format of the correspondence relationship storage unit 5 is not limited to the table format of the example of FIG. 9, and may be a list structure, a stack structure, or the like.
[0086]
Next, the operation of the corresponding time-series information storage unit 4 at this time will be described with reference to the flowchart of FIG.
[0087]
That is, FIG. 10 is a flowchart for explaining the operation at the time of recording of the time-series information storage unit 4. First, when this recording operation is started, the time-series information storage unit 4 outputs the recording start time of the audio information and the image information to the time information storage unit 7 for recording in step S300. Next, the process proceeds sequentially to step S301 and step S302, and the input image information and audio information are received and stored sequentially.
[0088]
Then, in the next step S303, it is determined whether or not a request for a storage address corresponding to the condition matching section has arrived from the correspondence storage unit, and when it is detected that the request has arrived, the process proceeds to step S304. In step S <b> 304, the storage address of the audio information and the image information corresponding to the condition matching section is returned to the correspondence storage unit 5.
[0089]
After it is determined in step S303 that a request for a storage address corresponding to the condition matching section has not arrived, and after step S304, the process returns to step S301 to continue storing image information and audio information.
[0090]
The time information storage unit 7 receives the information of the storage start time by the process in the step S300 of the time series information storage unit 4, and stores the storage start time.
[0091]
FIG. 11 is a flowchart for explaining the operation of the time information storage unit 7, and FIG. 12 is a diagram for explaining the storage structure of the time information storage unit 7. In FIG. 11, steps S400 and S401 are processing at the time of recording. In step S400, the storage start time of audio information and image information supplied from the time-series information storage unit 4 is detected, and this storage is performed in step S401. The start time is stored in the time information storage unit 7.
[0092]
As will be described later, the time information storage unit 7 is configured such that the elapsed time (that is, the information storage time) since the audio information and the image information are recorded in the time-series information storage unit 4 is equal to or longer than a predetermined time. A compression processing start instruction is output to the correspondence relationship storage unit 5. Steps S402 and S403 in FIG. 11 are processing units, and the compression start instruction process will be described later.
[0093]
The time information storage unit 7 manages the relationship between the name of the file storing the audio information and the image information and the storage start time using a table as shown in FIG. In this example, one conference record is recorded in one file. The file name is a file name given to each conference record, and the ID in FIG. 12 is an identifier (number in this example) for identifying each conference record file.
[0094]
The storage format of the storage start time is not limited to the table format, and may be a list structure or a stack structure. Further, information specifying the storage start time may be stored in a file or file name storing audio information and image information.
[0095]
Furthermore, the time to start the compression process may be automatically changed according to file attributes such as file size, file name, and file creator, and directory attributes such as directory creator and directory name.
[0096]
For example, as shown in FIG. 13, when the file size exceeds 5 Mbytes, the time to start the compression process is 1 month. When the file size is less than 5 Mbytes, the time to the compression process to be started is 2 months. To do. If the file extension is .AVI, the time to start compression processing is 1 month. If the file extension is .mpg, the time to start compression processing is 2 months. And In these cases, there is no need to specify the time until the compression process is started for each file, and there is an effect that the labor of the user can be saved.
[0097]
As described above, in this embodiment, when a conference is started and conference recording is started, the time at the start time is stored in the time information storage unit 7 and the conference start time (storage start). The image information and the sound information are stored in the time-series information storage unit 4 from the time point).
[0098]
Then, for the audio information in progress of the conference, the condition matching section detecting unit 3 detects the condition matching section as a section where the conversation is active, and the correspondence storage unit 5 corresponds to the information specifying the section. Stored in association with the storage address of the sequence information storage unit.
[0099]
[Operation during compression]
Next, the operation during compression will be described. In the first embodiment, the image information and / or audio information stored in the time-series information storage unit 4 is information-compressed so that the degree of importance is reduced when a predetermined period has elapsed since storage, and the time-series information is stored. An empty capacity is formed in the memory of the information storage unit 4, but the condition matching section is an important section, and this section is not compressed, or the compression rate is lowered to maintain high quality.
[0100]
FIG. 14 is a diagram for explaining the operation at the time of information compression in this embodiment, together with the flow of various information and the flow of output of each unit at that time.
[0101]
The time information storage unit 7 instructs the correspondence storage unit 5 to start the compression process when the elapsed time since the audio information and the image information are recorded in the time-series information storage unit 4 exceeds a predetermined time. Is output.
[0102]
That is, in step S402 of the processing routine of the time information storage unit 7 in FIG. 11, the current time supplied from a clock circuit unit (not shown) is compared with the storage start time stored in the time information storage unit 7, and the information It is determined whether or not a storage time of a predetermined time has passed. When it is determined that the predetermined time has elapsed, the process proceeds to step S403, and the correspondence storage unit 5 is requested to start the compression process.
[0103]
And after issuing this request | requirement or when it determines with predetermined time not having passed by step S402, it returns to step S400.
[0104]
For example, when the predetermined time is determined to be one month, the compression processing start request is generated one month after the storage start time, and the information newly accumulated in the time series information storage unit 4 is 1 After a month, compression will be applied. For example, the audio information and image information of the file name “file10” recorded at 13:30 on April 25, 1996 shown in FIG. 12 are subjected to the above-described compression processing at 13:30 on May 25, 1996. Will be given.
[0105]
In this example, the time until the compression processing is executed is fixedly given to the time information storage unit 7, but this time can be changed by the user. The timing of starting the compression process may be in the vicinity of the set time, and the compression process may be performed after the system enters the idling state. Further, the time until compression is performed may be set differently for each file.
[0106]
When a compression start instruction is input from the time information storage unit 7, the correspondence relationship storage unit 5 detects the input in step S <b> 204 in FIG. 7. If a compression start instruction is detected, the process proceeds to step S205, where each of the information specifying the condition matching section indicating the active section of the dialogue and the time series information storage unit corresponding to each condition matching section 4, the audio information and the image information stored in the time series information storage unit 4 are output to the compression unit 6. That is, the contents of one table in the table shown in FIG.
[0107]
Of course, a set of information specifying each condition matching section and a storage address corresponding to the condition matching section may be sequentially output to the compression unit 6 one by one. Further, in the file of the time-series information storage unit 4 storing the sound information and the image information, each piece of information for specifying the condition matching section and the sound stored in the file corresponding to each condition matching section are stored. You may comprise so that the storage address in this file of information and image information may be memorize | stored.
[0108]
The compression unit 6 that has received the input from the correspondence relationship storage unit 5 performs data compression of the image information stored in the time-series information storage unit 4. In this case, the compression unit 6 executes compression by dynamically changing the data compression rate or the data compression method based on the information indicating the condition matching section from the correspondence relationship storage unit 5.
[0109]
In the case of this embodiment, the condition matching section information is maintained at a high quality without being subjected to data compression, and data compression is performed for image information in sections other than the condition matching section. For this reason, as shown in FIG. 14, the compression unit 6 acquires a partial image sequence in a section other than the condition matching section from the time-series information storage unit 4, compresses the data, and compresses the compressed image sequence. Is written back to the time-series information storage unit 4.
[0110]
FIG. 15 is a flowchart for explaining the operation of the compression unit 6.
When receiving the compression start request from the correspondence storage unit 5, the compression unit 6 detects this in step S500, and proceeds to step S501. In step S501, information specifying each of the condition matching sections indicating active sections of the dialogue input from the correspondence storage unit 5 and stored in the time series information storage unit 4 corresponding to each condition matching section. The stored voice information and image information in the time-series information storage unit 4 are input and stored in a work memory (not shown) of the compression unit 6. The work memory uses, for example, a semiconductor memory as a storage medium.
[0111]
The compression unit 6 compresses the image information stored in the time series information storage unit 4 with reference to a plurality of sets of condition matching section and storage address information stored in the work memory.
[0112]
In step S502, the partial image sequence of 10 frames other than the condition matching section is sequentially read from the time series information storage unit 4 to the compression unit 6 as one unit partial image sequence. In this embodiment, in order not to compress the image information corresponding to the condition matching section, only the image information other than the condition matching section is read and compressed. Needless to say, when image information corresponding to a condition matching section is also compressed, it is necessary to read and compress the image information including the condition matching section.
[0113]
In step S503, frame thinning processing is performed in which only the first frame in this example of the 10-frame partial image sequence is left and the other 9 frames are deleted. In the next step S504, the compressed image sequence after the frame thinning is written back to the time-series information storage unit 4.
[0114]
In the next step S505, it is determined whether or not the compression process for the file in which the conference record is accumulated is completed. When the compression process for the entire file is completed, the process of the compression unit 6 is terminated. If there remains a portion to be compressed, the process returns to step S502 to repeat the compression process.
[0115]
FIG. 16 is a diagram for explaining the operation of the compression unit 6. In FIG. 16A, if the memory areas designated by the storage addresses of the time-series information storage unit 4 corresponding to the condition matching section are a3 to a6 and a11 to a12, the compression unit 6 compresses , Storage memory areas a1 to a2 and storage memory areas a7 to a10. Here, each of the storage memory areas a1, a2,... Has a capacity for a predetermined number of frames, for example, 10 frames.
[0116]
As described above, even after compression, the image sequences in the condition matching section accumulated in the storage memory areas a3 to a6 and the storage memory areas a11 to a12 are not changed compared to before compression.
[0117]
On the other hand, since the image sequences stored in the storage memory areas a1 to a2 and the storage memory areas a7 to a10 are subjected to frame thinning compression, the contents of the storage memory areas a1, a7, and a8 are compressed image sequences. Replaced by Then, as the amount of information decreases, free memory areas a2, a9, and a10 are generated in the storage medium of the time-series information storage unit 4 as shown in FIG. 16B.
[0118]
If it is desirable that the time series data is continuously stored in the storage medium, the generated empty memory portion is filled with the previous and next time series data, for example, so that there is no gap in the memory. Can be eliminated.
[0119]
Next, an operation when reproducing recorded or compressed audio information or image information in the first embodiment will be described. As described above, this reproduction is performed in the reproduction unit 8 under the control of the control unit 9, and is executed based on a reproduction instruction through a user interface (not shown).
[0120]
When playing back recorded audio information or image information, it is often necessary to change the playback speed or to play back slowly by rewinding a little, so the fast forward function, rewind function, slow playback function, pause function, The conference information storage processing device 13 is provided. In addition, a slide bar corresponding to the time axis is provided on the screen of the display monitor, and a pointer indicating the current playback time is displayed on the slide bar, or the playback position is set by sliding the bar by a user input operation. It can be specified.
[0121]
In addition, regarding the speed to be reproduced, it is not always necessary to reproduce it according to the recorded time information, it is possible to reproduce by increasing the speed while keeping only the recorded order relation, and only the condition matching section is reproduced. You can also search for automatic playback. For example, the section from time F200 to F201 in FIG. 8 is played back at the same speed as the stored speed, and the other sections are played back at double speed.
[0122]
In addition, information indicating the condition matching section may be displayed together on the slide bar on the time axis so that the condition matching section can be seen at a glance. Further, the detection result detected by the condition matching section such as the name of the speaker detected by the condition matching section or the face photograph of the speaker may be displayed on the slide bar on the time axis.
[0123]
In the first embodiment described above, when a predetermined number or more of early speaker change patterns are continuously detected, both ends of the speech section including the patterns are set as both ends of the condition matching section. However, a point in time before the start point of the speech segment including the early speaker change pattern may be set as the start of the condition matching interval, or a speech segment before the predetermined number of speech segments including the early speaker change pattern It is good also as a condition coincidence section including.
[0124]
Also, the end point of the condition matching section may be a time point after a predetermined time after the end time of the speech section including the early speaker change pattern, or the speech section after the predetermined number of the speech sections including the early speaker change pattern may be selected. It may be included as a condition matching section.
[0125]
In addition, a single voice signal such as “door closing sound” can be detected by the condition matching section detection unit 3. In this case, a time point that is a predetermined time before the point in time when the single sound signal is detected is detected as the start point of the condition matching section, and a time point that is a predetermined time after the point in time when the single sound signal is detected is the condition match. It is configured to detect as the end point of the section.
[0126]
Further, in this embodiment, the case where the condition matching section detection unit 3 detects an active dialogue pattern from the input voice signal has been described, but other features such as a laughter pattern and a clapping pattern are also included. It is also possible to register various voice patterns, recognize these patterns from the input voice signal, and detect a section including these patterns as a condition matching section. In this case, the condition matching section detection unit 3 includes pattern recognition means for performing pattern recognition using a known pattern recognition technique, for example, a technique for analyzing the temporal transition of the power or frequency component of an audio signal. Provided.
[0127]
In this embodiment, the compression unit 6 is configured to perform thinning-out compression of the image. However, the configuration of the compression unit 6 includes a storage time, a compression rate of intra-frame compression, and a frame when compressing image information. Any device that dynamically changes at least one of the compression rate of intermittent compression, the time interval of intermittent recording, the color information thinning rate, the luminance information thinning rate, and the like may be used.
[0128]
In particular, methods for compressing moving image information include intra-frame compression methods and inter-frame compression methods, and intra-frame compression methods include a method using vector quantization and a method using discrete cosine transform. There is. As a compression method between frames, there is a method of recording only a difference between image information of previous and subsequent frames. That is, any device that converts the information amount per unit time into a smaller information amount corresponds to the compression unit 6 in the present invention.
[0129]
In the first embodiment, image information of sections other than the condition matching section is configured to be stored as a frame dropping video with a small amount of information. You may make it delete the image information or audio | voice information of an area from a storage medium.
[0130]
Further, the information on the section of the condition matching section and the information on the section other than the condition matching section may be stored separately in different storage media. For example, when recording information, the information in the condition matching section and the information in the section other than the condition matching section are stored on the same magnetic disk, and only the information in the condition matching section is stored when the information is compressed. It is configured to leave the information on the magnetic disk and move the information of the sections other than the condition matching section to the magneto-optical disk or the magnetic tape. In general, magneto-optical disks and magnetic tapes have a feature that a large amount of information can be stored at a low cost although the access speed to information is slower than that of magnetic disks. This is suitable for storing information of sections other than the condition matching section.
[0131]
Furthermore, in this embodiment, the audio information is not compressed. However, when the audio information is compressed, at least one of the storage time, the sampling frequency, and the number of encoded bits is set when the audio signal is compressed. It should be changed dynamically.
[0132]
Note that the time-series information input to the apparatus according to the above-described embodiment may be an analog signal input from a camera / microphone / video deck / tape recorder / sensor, or a digital signal obtained by encoding the analog signal. Further, it may be a digital signal input through a computer network / computer bus. That is, any information that is sequentially input with the passage of time corresponds to time-series information in the present invention.
In the first embodiment described above, the condition matching section detection unit 3 detects a condition matching section from the voice information, and based on the detection result, the image information of the condition matching section has a higher image quality than the other sections. So that the compression rate is dynamically changed and the image information stored in the time-series information storage unit 4 is compressed so that a characteristic phenomenon occurs in the image information. Many important portions can be stored in a limited storage medium, and even image information other than the important portions can be stored for a long time with a small amount of data.
Further, since the presence / absence of the input audio signal or the audio signal level is detected by the condition matching interval detection unit 3, the audio or image information of the interval in which the audio is being emitted is recorded from the beginning to the end. There is an effect that it can be stored with a small amount of data even if it is voice or image information of a section that can be stored with high image quality and no voice is emitted.
Also, since the condition matching section detection unit 3 detects the input voice sender or the change of the sender, the voice or image information of a specific sender can be obtained from the beginning to the end with high sound quality / high image quality. In addition, there is an effect that even a voice or image information of other callers can be stored with a small amount of data.
[0133]
[Second Embodiment]
Also in the second embodiment, in the same manner as described above, the following description will be made assuming that only the image is the compression target in order to simplify the description.
[0134]
In the second embodiment, when the input image information is stored in the time-series information storage unit 4, it is stored for each frequency band, such as a low frequency band and a high frequency band, and a time information storage unit is stored. When the compression start instruction is received from 7, the image information is compressed by deleting the high frequency band of the image. This second embodiment uses the fact that the high frequency band of an image is a component related to so-called image detail, and even if this is deleted, there is little influence on grasping the basic image content. It is.
[0135]
In the first embodiment, as shown in FIG. 14, the partial image sequence in the time-series information storage unit 4 is read out, subjected to compression processing by the compression unit 6, and then written to the time-series information storage unit 4 again. However, in the case of the second embodiment, the partial image sequence is read from the time-series information storage unit 4, is subjected to image compression processing, and is written back to the time-series information storage unit 4. Since it is not necessary, the load on the system during the compression process can be reduced.
[0136]
In the second embodiment, when the input image information is stored for each frequency band, the condition matching section and the sections other than the condition matching section are stored in the time-series information storage unit 4. It is configured to change the way of dividing the frequency band.
[0137]
Specifically, only the image information of the sections other than the condition matching section is stored for each frequency band, and the image information of the condition matching section is stored by a normal method that does not store by frequency band.
[0138]
FIG. 17 is a diagram for explaining the recording operation in the second embodiment, together with the flow of various information and the flow of output of each unit. The configuration of the second embodiment is different from the configuration of FIG. 1 and FIG. 5 described in the first embodiment in that a frequency band-specific image generation unit 21 is added as a component. .
[0139]
In this example, the frequency band-specific image generation unit 21 includes a high-pass filter and a low-pass filter. In the case of the second embodiment, the condition matching section detection unit 3 detects the condition matching section from the input voice information in the same manner as in the first embodiment, and the condition matching section is detected. Information to be specified is supplied to the correspondence storage unit 5 and is also supplied to the frequency band-specific image generation unit 21.
[0140]
The frequency band-specific image generation unit 21 receives the information for specifying the condition matching section from the condition matching section detection unit 3 and outputs to the time-series information storage unit 4 in the condition matching section and other sections. Change the signal.
[0141]
FIG. 18 is a flowchart for explaining processing in the frequency band-specific image generation unit 21 in the second embodiment.
[0142]
As shown in FIG. 18, the image generating unit 21 classified by frequency band receives input of audio information or image information in step S600. Then, in the next step S601, information specifying the condition matching section from the condition matching section detection unit 3, that is, in this example, information at the beginning and end of the condition matching section, It is determined whether it is within the condition matching section.
[0143]
Regarding the image information of the section determined to be within the condition matching section, the process proceeds from step 601 to step S603, and the image information is output to the time-series information storage unit 4 as it is, and the input image information is divided into normal, frequency bands. It is made to memorize | store in the time-sequential information storage part 4 with the storage format which cannot be divided.
[0144]
On the other hand, if it is determined in step S601 that it is a section other than the condition matching section, the process proceeds to step S602, where the input image information is divided into high frequency band information and low frequency band information. A process of generating different image information is executed. The generated frequency band-specific image information is output to the time-series information storage unit 4 and stored in step S603. Thereafter, steps S600 to S603 are repeated.
[0145]
FIG. 19 is a diagram for explaining the storage state of the time-series information storage unit 4 before compression of image information in the second embodiment. As shown in FIG. 19, the time-series information storage unit 4 in the case of the second embodiment includes a memory unit 4Ma that stores image information of a condition matching section, and image information of sections other than the condition matching section. A memory unit 4Mb for storage. These memory units 4Ma and 4Mb may be separate storage media, or may be obtained by dividing the memory area of one storage medium.
[0146]
In the time series information storage unit 4, the memory unit 4 Ma that stores the image information of the condition matching section stores the image information without dividing the frequency band. The memory unit 4Mb that stores the image information of the section other than the condition matching section is further divided into a high-frequency storage memory and a low-frequency storage memory. Frequency band components are stored in association with each other. That is, in FIG. 19, the storage contents of the areas a1 to a6 of the high band storage memory and the storage contents of the areas a1 to a4 of the low band storage memory are the high frequency component of the image signal and the low frequency of the same section. Ingredients are shown. The time-series information storage unit 4 also manages the correspondence between the frequency band components.
[0147]
In FIG. 19, “·” is given in each of the memory areas a 1, a 2,... Indicates that an image sequence is stored, and a memory area without “·” is an empty memory area. I mean.
[0148]
Also in the second embodiment, the time information storage unit 7 monitors the storage period of the stored contents of the time series information storage unit 4, and when the storage period has passed a predetermined period such as one month, for example, The time information storage unit 7 outputs a compression processing start instruction and causes image information compression to be executed.
[0149]
FIG. 20 is a diagram for explaining the operation at the time of information compression in the second embodiment, together with the flow of various information and the flow of output of each unit at that time. FIG. 21 is a flowchart for explaining the compression processing in the second embodiment.
[0150]
That is, in the case of this embodiment, the compression start instruction from the time information storage unit 7 is directly supplied to the compression unit 6. Then, as shown in the flowchart of FIG. 21, the compression unit 6 receives this compression start instruction, and the elapsed time since the audio information and the image information are recorded in the time-series information storage unit 4, that is, the information storage time. If it is detected in step S700 that the predetermined time has elapsed, the process proceeds to step S701. In step S701, an instruction to perform processing for deleting the high frequency band is sent from the memory storing the sections other than the condition matching section to the time-series information storage unit 4.
[0151]
In this example, the time-series information storage unit 4 receives the high frequency component deletion instruction from the compression unit 6 and stores the image information of the section other than the condition matching section in the high frequency storage memory of the memory unit 4Mb. Delete all contents.
[0152]
FIG. 22 is a diagram illustrating the storage state of the time-series information storage unit 4 after image information compression. When the storage state before the compression process is as shown in FIG. 19 described above, the time-series information storage unit 4 receives the high frequency component deletion instruction from the compression unit 6 and the time series information storage unit 4 All of the image information is deleted from the areas a1 to a6 of the high frequency band storage memory for storing the high frequency band image components in the memory unit 4Mb for storing the image information of the other sections. As a result, in the time-series information storage unit 4, areas a1 to a6 indicated by halftone dots in FIG. 22 are generated as free memory areas.
[0153]
The generated free memory area may be reused as a storage memory area for storing a section other than the condition matching section, or may be used as a storage memory area for storing the condition matching section.
[0154]
After performing compression processing by the above-described processing, if the video of the part that was actively discussed is played back, the high-quality video with smooth motion is played back, and the other parts are played back. In this case, the video is a so-called low-quality video and low-quality video. However, since only the insignificant parts other than the scenes that have been actively discussed can be compressed with a high compression rate, the amount of information to be stored is very small.
[0155]
In the above description of the second embodiment, the method of storing input image information for each frequency band has been described. For example, as described in Japanese Patent Laid-Open No. 6-178250, input image information is time-sequentially described. When accumulating in the information storage unit 4, the image signal is divided into a luminance signal component and a color signal component such as a color difference signal or a carrier color signal (color subcarrier signal) and stored in separate areas, When a compression start instruction is generated from the information storage unit 7, only the color signal component may be deleted. In this case, since it is not necessary to read out or write back the partial image sequence from the time-series information storage unit 3, the system load during the compression process can be reduced, and the speed of the compression process can be increased.
[0156]
Further, when the input image information is accumulated in the time-series information storage unit 4 for each frequency band, the image information of the condition matching section and the low frequency band component of the image information of the section other than the condition matching section are continuously stored in the storage medium. The high frequency band component of the image information in the section other than the condition matching section may be stored in another area of the storage medium. In this case, even if the high frequency band component of the image information in the section other than the condition matching section is deleted during compression, the time-series data after compression continues in the storage medium, so that the reproduction speed is prevented from being lowered. Can do.
[0157]
Furthermore, in the second embodiment, the audio information is not compressed, but the audio information can be compressed in the same manner. For example, as described in Japanese Patent Application Laid-Open No. 7-15519, when the input voice information is stored in the time-series information storage unit 4, it is stored for each frequency band, and the compression unit 6 is connected to the time information storage unit 7. When a compression start instruction is received from, a high frequency band of audio may be deleted. In this case as well, it may be configured to preferentially delete the high frequency band component of the audio information in the section other than the condition matching section.
In the second embodiment described above, the input audio information or the input image information is stored for each frequency band when accumulating in the time-series information storage means, and the high frequency band is deleted during compression. This eliminates the need to read information from the time-series information storage means for writing or to write back information to the time-series information storage means, and thus has an effect of reducing the load on the system during the compression process.
Further, when the input audio information or the input image information is stored in the time-series information storage unit by frequency band, the frequency band is divided into the condition matching section detected by the condition matching section detection unit and the section other than the condition matching section. In the case of storing differently, only the image information of the section other than the condition matching section is stored for each frequency band, and the image information of the condition matching section is stored in a normal method (not stored for each frequency band). As described above, since it is possible to reduce the processing for dividing the input voice information or the input image information by frequency band, there is an effect that the load on the system can be reduced.
[0158]
[Third Embodiment]
In the first embodiment and the second embodiment, the elapsed time since the audio information or the image information was recorded in the time-series information storage unit 4 has become a predetermined time or more, for example, one month. In this case, the example in which the compression process is performed only once has been described.
[0159]
However, there is a case where the storage medium can be saved more effectively by performing the compression in a plurality of steps in stages rather than performing the compression process only once. For example, when recording a meeting, it is less likely to refer to a meeting that took place a month ago than to a meeting that took place a week ago, and similarly, Compared to the possibility of referring to a meeting held later in the year, the possibility of referring to a meeting held six months ago is low. As described above, when the possibility of being referred to later becomes lower, it is possible to effectively save the storage medium by storing the information with a smaller amount of information.
[0160]
In the third embodiment, an example will be described in which the compression rate or the compression method is changed in accordance with the elapsed time since the image information is recorded in the time-series information storage unit 4, and the information is compressed stepwise. . However, even in the case of an old conference recording video, since it is necessary to store the image signal in high quality for important scenes, it is the same as in the first embodiment or the second embodiment. In this example, in the recorded conference video, only the video of the important part of the conference that has been actively discussed remains in high quality, and the other part is compressed at a high compression rate.
[0161]
Also in the third embodiment, the input image information is stored for each frequency band when the input image information is stored in the time-series information storage unit 4. For this reason, as in the second embodiment, an image generating unit 21 for each frequency band is provided. In the case of the third embodiment, the frequency band includes a high frequency band and a medium frequency band. Then, the image information is divided into three bands, a low frequency band, and stored in the time-series information storage unit 4. The frequency band-specific image generation unit 21 in this case includes a high-pass high-pass filter, a mid-band pass-pass filter, and a low-pass low-pass filter.
[0162]
Further, in the third embodiment, the image signal is stored with the frequency band divided into three bands without distinction between the condition matching section and the section other than the condition matching section.
[0163]
FIG. 23 is a diagram for explaining the storage state of the time-series information storage unit 4 when image information is recorded (before image information compression). That is, in this example, each of the memory unit 4Ma that stores the image information of the condition matching section of the time-series information storage unit 4 and the memory unit 4Mb that stores the image information of the section other than the condition matching section are as shown in FIG. In addition, a high-frequency storage memory, a mid-frequency storage memory, and a low-frequency storage memory are stored, and the high-frequency component, the mid-frequency component, and the low-frequency component of the image information of the corresponding section are stored in each memory region It is what is done.
[0164]
Also in the third embodiment, the time information storage unit 7 monitors the passage of time from the time of storage, and outputs a compression start instruction to the compression unit 6 when a predetermined time has elapsed. However, the compression start instruction is output at each of a plurality of preset elapsed times, for example, one week later, one month later, and six months later. At this time, each compression start instruction is added to the compression unit 6 by adding data indicating which compression start instruction is at which time point and which frequency band component is to be compressed.
[0165]
FIG. 24 is a diagram illustrating an example of a compressed time management table stored in the time information storage unit 7. As shown in FIG. 24, for example, the image data to be erased first is a high frequency band portion of a section other than the condition matching section, and is erased when one week elapses after the information is recorded. In addition, when one month has passed since the information was recorded, the middle frequency band portion other than the condition matching section and the high frequency band portion of the condition matching section are deleted. In addition, when half a year has passed after recording the information, the low frequency band portion of the section other than the condition matching section and the middle frequency band section of the condition matching section are deleted. It should be noted that the low frequency band portion of the condition matching section is not automatically deleted unless an explicit deletion instruction is given from the user.
[0166]
When the compression unit 6 receives the compression start instruction from the time information storage unit 7, the compression unit 6 analyzes the content of the instruction, and determines the content of any storage memory to the time-series information storage unit 4 based on the analysis result. Issue compression instructions for deleting. The time series information storage unit 4 executes stepwise compression processing of image information in response to the compression instruction. More specifically, the contents of each storage memory are erased according to the erase time of the table shown in FIG.
[0167]
25, 26, and 27 show time-series information storage units when one week, one month, and half a year have passed since the image information is recorded as shown in FIG. 23 before image information compression. FIG. In FIG. 23, FIG. 25, FIG. 26, and FIG. 27, “.” Is given in each of the memory areas a1, a2,... Indicates that an image sequence is stored. A memory area without "" means an empty memory area.
[0168]
That is, when one week has elapsed, as shown in FIG. 25, the time-series information storage unit 4 erases all the contents of the high-frequency storage memory of the memory unit 4Mb that stores image information of sections other than the condition matching section. The free memory area.
[0169]
In addition, when one month has elapsed, as shown in FIG. 26, the contents of the high-frequency storage memory of the memory unit 4Ma that stores the image information of the condition matching section and the image information of the sections other than the condition matching section are stored. All the contents of the middle area storage memory of the section 4Mb are erased.
[0170]
Furthermore, when half a year has elapsed, as shown in FIG. 27, the memory unit 4Ma that stores the image information of the condition matching section and the memory unit that stores the contents of the middle area memory of the memory 4Ma and the image information of the sections other than the condition matching section All contents of the 4Mb low-frequency storage memory are erased. As a result, only the content of the low-frequency storage memory of the memory unit 4Ma that stores the image information of the condition matching section remains as the storage content of the time-series information storage unit 4.
[0171]
In this way, as shown in FIGS. 25 to 27, the time-series information storage unit 4 accumulates image information with a smaller amount of information as time elapses.
[0172]
In the above-described example of the third embodiment, a table is used for managing the compression time of the time information storage unit 7. Of course, a list or stack structure may be used instead of the management table. Absent.
[0173]
Furthermore, in the time information storage unit 7, instead of managing the compression time and the compression target in a table or list, the compression rate of information at an arbitrary time is calculated by mathematical calculation using the information storage time as a parameter, Information regarding the compression rate may be sent to the compression unit 6 to perform information compression.
[0174]
For example, when y is an information amount retention rate (%) and x is time (elapsed days), the time information storage unit 7
y = 90exp (-Ax) +10 (1)
Where A is a constant and A> 0.
The information amount retention rate at a specific time is obtained by the above-described arithmetic expression (1), and information on the information amount retention rate is supplied to the compression unit 6 as information on the compression rate. Here, the information amount retention rate refers to the ratio of the information amount at a specific time to the information amount when the information is first recorded.
[0175]
The compression unit 6 sets a compression rate based on the information amount retention rate from the time information storage unit 7 and compresses the image information accumulated in the time-series information storage unit 4 with the compression rate. In this case, the time information storage unit 7 repeatedly generates a compression start instruction in the cycle so as to execute recompression step by step in a certain cycle.
[0176]
As in the third embodiment described above, the compression processing is performed when the elapsed time (that is, the information storage time) from when the audio information or the image information is recorded in the time-series information storage means becomes a predetermined time or more. If it is configured so as to start, recent audio or image information that is likely to be referenced can be stored with high sound quality / high image quality, and even audio or image information recorded in the past is small There is an effect that can be memorized by the data amount.
In addition, when the audio information or the image information is configured to be compressed in stages according to the elapsed time (that is, the information storage time) since it was recorded in the time-series information storage unit, it is referred to later. Since the information that is less likely to be stored can be stored with a smaller amount of information, the storage medium can be more effectively saved.
Further, since the compression amount or the compression method can be set so that the data amount of the audio information or the image information fits in a predetermined storage capacity, the compressed data is included in the input image information. There is an effect that a digest of a desired storage size is obtained, in which only the important part is stored with high image quality.
In addition, although this 3rd Embodiment was demonstrated as a deformation | transformation of 2nd Embodiment, of course, it can also implement as a deformation | transformation of 1st Embodiment.
[0177]
[Fourth Embodiment]
In the fourth embodiment, the detection condition in the condition matching section detection unit 3 is that a pre-registered keyword appears in an input audio signal or a pre-registered voice pattern appears. This is the case.
[0178]
First, a case will be described in which the detection condition in the condition matching section detection unit 3 is that a keyword registered in advance appears in the input audio signal.
[0179]
In this case, the condition matching section detection unit 3 compares the speech recognition means, the memory for storing the registered keyword, the speech recognition result and the keyword registered in advance in the memory, and detects the match between them. Detecting means. A keyword is registered in advance in the memory by a user.
[0180]
At the time of information recording, the condition matching section detection unit 3 sequentially converts the input voice signal into character string information by the voice recognition means, and extracts words from the character string information by performing morphological analysis. Then, the extracted word / phrase is compared with character string keywords registered in advance in the memory, such as “homework”, “action item”, “task”, “conclusion”, “decision”, “important”, and “summary”.
[0181]
When the word / phrase extracted from the input speech signal matches any of the character string keywords registered in advance, the time point when the character string keyword is detected becomes the start point of the condition matching section.
[0182]
In order to determine the end point of the condition matching section, the condition matching section detection unit 3 in this example uses an image signal for how long the image signal is detected for each keyword character string after the keyword is detected. 28 is set in a table as shown in FIG. 28. The keyword validity period is assigned to a longer time for an important keyword.
[0183]
When the image information stored in the time-series information storage unit 4 is compressed after a predetermined time as described above, the image information of the section from the start point to the end point of the condition matching section is high-quality. The other section is compressed at a high compression rate. As the compression method, any one of the first to third embodiments can be adopted.
[0184]
Further, if the importance level can be set for each keyword character string, the image signal can be compressed at different compression rates depending on the importance level of each keyword character string.
[0185]
Next, a case will be described in which the condition matching section detection unit 3 detects a condition matching section on the assumption that a pre-registered voice pattern appears in the input voice signal.
[0186]
Even when it is difficult to detect keywords by voice recognition, these patterns may be recognized if they are characteristic voice signal patterns such as laughter patterns, applause patterns, and active conversation patterns. . Therefore, the condition matching section detection unit 3 also detects the appearance of this characteristic voice pattern as a detection condition.
[0187]
In this case, the condition matching section detection unit 3 is provided with a memory in which characteristic voice signal patterns such as a laughter pattern, a clap pattern, and an active conversation pattern are registered and stored in advance. Then, there is provided pattern recognition means for performing pattern recognition using a known pattern recognition technique, for example, a technique for analyzing the temporal transition of the power or frequency component of an audio signal.
[0188]
Compare the pattern of the characteristic audio signal registered in advance with the pattern of the audio signal extracted from the sequentially input audio signal, and recognize the characteristic pattern from the coincidence or similarity To do. In order to increase the recognition rate of pattern recognition, a voice pattern may be registered for each speaker.
[0189]
When it is determined that the pattern of the audio signal extracted from the input audio signal matches one of the characteristic audio signal patterns registered in advance, the detection time of the audio signal pattern is the start of the condition matching section It becomes a point.
[0190]
In addition, in order to determine the end point of the condition matching section, the condition matching section detection unit 3 in this example increases the image signal for how long from the time when the pattern is detected for each audio signal pattern. The audio signal pattern effective period for determining whether to save the image quality is set in a table as shown in FIG. 29, and the image information of the section from the start point of the condition matching section to the end point is stored with high image quality. It is determined that it should be information.
[0191]
When the image information stored in the time series information storage unit 4 is compressed after a predetermined time as described above, the image information of the section from the start point to the end point of the condition matching section is high-quality. The image information of other sections is compressed so as to greatly reduce the amount of information. As the compression method, any one of the first to third embodiments can be adopted.
[0192]
In this example, the time point when it is determined that the sound signal pattern extracted from the input sound signal matches one of the pre-registered characteristic sound signal patterns is determined as the start point of the condition matching section. However, it is possible to store the image information including the image information before the time when the pattern of the audio signal is detected with high image quality. In such a case, for example, there is usually a cause for the appearance of the pattern before the time when the pattern of laughter or applause appears. You can make it save with image quality.
[0193]
In this case, by setting the time point a predetermined time before the time point when the characteristic audio signal pattern appears as the start point of the condition matching section, the event causing the pattern appearance can be stored with high image quality. The other sections are configured to be compressed at a high compression rate.
As described above, according to the fourth embodiment, it is configured so that the condition matching section detecting means detects that a keyword or pattern registered in advance appears in the input voice information. Voice or image information stored during a period in which keywords or patterns registered in advance frequently appear can be stored with high sound quality / high image quality from the beginning to the end. There is an effect that data can be stored with a small amount of data.
[0194]
[Fifth Embodiment]
In the fifth embodiment, the condition matching section detection unit 3 detects a predetermined state change by an external sensor. That is, in this embodiment, when a condition matching section is detected on the condition that it is difficult to detect the condition matching section from the audio signal, a state change occurs in information not included in the input audio signal. An external sensor is provided in order to detect a condition matching section on the condition.
[0195]
In this embodiment described below, a case where an external sensor detects a location will be described. In other words, in the following example, importance levels are given according to the conference rooms, such as boardrooms, reception rooms, and general meeting rooms. Like that.
[0196]
The information on which conference room the meeting was held at where the audio signal or the image signal was input is obtained by analyzing position information output from a position measuring device such as a GPS (Global Positioning System). can get. When GPS is used, the latitude / longitude of the place where the audio signal or the image signal is input is measured, and the information is compared with the latitude / longitude stored in advance where each conference room exists. The conference room to which the audio signal or the image signal is input can be specified.
[0197]
In addition to GPS, the infrared transmission / reception system described in Japanese Patent Application Laid-Open No. 7-141389, in which an infrared transmitter that oscillates a bit pattern specific to each location, is installed in an arbitrary location such as a conference room or hallway. It can also be used. In this case, when an audio signal or an image signal is input, a bit pattern oscillated by a nearby infrared transmitter is received, and a conference room is identified from the pattern.
[0198]
In the example described below, a case where an infrared transmission / reception system is used will be described. In this case, the condition matching section detection unit 3 includes an infrared signal recognition means, a memory that stores the registered location name, a location name determined from the result of recognizing the infrared signal, and a location name registered in the memory in advance. And a place coincidence detecting means for detecting coincidence of the two. A user registers a location name in advance in the memory.
[0199]
At the time of recording information, the condition matching section detection unit 3 converts the input infrared signal into a place name by the infrared signal recognition means. Then, the converted place name is compared with a place name registered in advance in the memory. Then, when detecting the place, the condition matching section detection unit 3 detects the beginning of the period recognized as staying at the same place as the start point of the condition matching section, and recognizes that it remains at the same place. The end of the set period is detected as the end point of the condition matching section.
[0200]
The correspondence storage unit 5 stores the start point and end point of the section and the place name as information for specifying the condition matching section. A corresponding identifier may be stored instead of the place name. Further, the correspondence relationship storage unit 5 stores each condition matching section and the storage addresses of the audio signal and the image signal stored in the time series information storage unit 4 in association with the section.
[0201]
In this example, the time information storage unit 7 outputs a compression start instruction to the correspondence relationship storage unit 5 when the storage retention period is equal to or longer than a predetermined period. In response to this compression start instruction, the correspondence relationship storage unit 5 sends information on the section and a place name or a place identifier to the compression unit 6 as information for specifying the condition matching section.
[0202]
The compressing unit 6 includes a table that stores a conference room name (location name) registered in advance and an importance level of each conference room in association with each other. FIG. 30 is an example of this table. The compression unit 6 refers to this table using the place name or identifier from the correspondence relationship storage unit 5 and detects the meeting room name in the condition matching section. Then, the importance assigned to the conference room name is extracted, and the image signal in the corresponding condition matching section is compressed at a compression rate corresponding to the importance. In other words, information recorded at a location of higher importance is stored with higher quality during compression.
[0203]
In this way, for example, the recorded video of an important meeting held in a boardroom meeting room is stored for a longer time with higher image quality than the meeting video held in another meeting room. Can do.
[0204]
Although the case where a place is detected by an external sensor has been described above, the sensor may determine a person. For example, a weak wireless transmitter is attached to a conference attendant and a wireless receiver is attached to a conference room. A period during which a conference attendee enters the conference room is detected by the wireless receiver, and only this period is stored with high image quality.
[0205]
Furthermore, if the weak wireless transmitter transmits a different signal for each attendee of the conference so that it is possible to identify who has entered the room, only the period during which a specific person has entered the room can be increased. It can also be configured to save with image quality.
[0206]
In addition, not only the physical location and the name of the person, but also from the event obtained by combining the detection results of multiple sensors such as "I was attending a certain meeting" and "I was with a certain person" A section may be specified.
[0207]
Furthermore, when a single sensor input signal (trigger) such as “opening / closing of a door” is detected by the condition matching section detection unit 3, a time point a predetermined time before the trigger is detected is set as a condition matching section. It is configured to detect as a start point, and to detect a time point after a predetermined time after the trigger is detected as an end point of the condition matching section. In order to detect opening / closing of the door, it can be implemented by attaching an opening / closing detection sensor to the door.
As described above, in the case of this fifth embodiment, when the place where the audio signal or the image signal is input is detected by the condition matching section detecting means, an important meeting is designated as a specific meeting room. For example, if the sound or image of an important event taken at an important place can be saved with high sound quality / high image quality, and the sound or image information is taken at any other place, There is an effect that data can be stored with a small amount of data.
In addition, when configured to detect a specific person by an external sensor, the sound or image information of the specific person can be stored from the beginning to the end with high sound quality / high image quality, and the sound of other persons or Even image information can be stored with a small amount of data.
[0208]
[Sixth Embodiment]
In the sixth embodiment, the condition matching section detection unit 3 detects the motion of the video camera 17 (hereinafter referred to as camera work).
[0209]
For example, when a person is photographed with zoom-in, important images are often taken, and the audio signal or image signal during the period when the camera 17 is zoomed in has high sound quality / high image quality. I often want to remember. Therefore, in the example described below, a section that is captured at the same magnification is detected as a condition matching section together with the magnification. Then, assuming that the condition matching section with a higher magnification is the more important image, the importance is determined, and subsequent information compression is performed so that the quality becomes high according to the magnification. Thereby, the image of the section where the camera 17 is zoomed up with high magnification is maintained in high quality.
[0210]
An example in the case of this sixth embodiment will be described below.
[0211]
In this example, the video camera 17 can set three magnification modes of 1 ×, 5 ×, and 10 × as the camera magnification, and outputs information indicating the magnification as camera operation information in accordance with the operation of the zoom ring. To do. This camera operation information is supplied to the condition matching section detection unit 3. As described above, the condition coincidence section detection unit 3 detects, from this camera operation information, a section with the same camera magnification as the condition coincidence section, together with the magnification.
[0212]
That is, when detecting the camera work, the condition matching section detecting unit 3 detects the time of the change point of the magnification of the camera operation signal as the start point of the condition matching section, and then the magnification of the camera operation signal is changed. The time is detected as the end point of the condition matching section. Therefore, the end point of this condition matching section is the same time as the start point of the next condition matching section. The information on the condition matching section and the magnification information are stored in the correspondence storage unit 5 in association with the storage addresses of the time-series information storage unit for the image information and the voice information of the condition matching section.
[0213]
FIG. 31 is a diagram illustrating the relationship between the camera magnification and the condition matching section. In FIG. 31, T0, T1, T2, and T3 are condition matching sections. Sections T0 and T3 where the magnification is 1 are normal magnification sections where the zoom ring is not operated. In the example of FIG. 31, the zoom ring is operated at time t1, and the zoom-in operation starts. In the beginning section T1, the magnification is 5 times, the magnification is increased to 10 times at time t2, and the magnification is increased at time t3. The zoom-in operation ends when the magnification becomes 1.
[0214]
In this embodiment, the compression unit 6 sets the image thinning compression rate in each magnification mode to 1 frame / second for three magnification modes of 1 ×, 5 ×, and 10 × of the camera magnification. 5 frames / second and 10 frames / second are set.
[0215]
Similar to the above-described embodiment, when the elapsed time since the image information is recorded in the time-series information storage unit 4 becomes a predetermined time or more, the correspondence relationship storage unit 5 compresses the time information from the time information storage unit. A start instruction is given and executed. At this time, the correspondence relationship storage unit 5 sends a pair to the compression unit 6 for each condition matching section information, magnification, and storage address of the time-series information storage unit. In this case, the compression unit 6 compresses the section T1 in FIG. 23 at 5 frames / second, the section T2 at 10 frames / second, and the other sections T0 and T3 at 1 frame / second. To do.
[0216]
As described above, in the case of this embodiment, information is stored by changing the compression rate of the image signal of the important scene and the image signal of the unimportant scene according to the camera work or the change of the camera work. Can do.
When the camera operation signal or the change in the camera operation signal is detected by the condition matching section detection unit 3, the camera is zoomed in when an important sound or image is shot up by the camera. There is an effect that the sound or image of the period can be stored with high sound quality / high image quality, and even the sound or image information of the other period can be stored with a small amount of data.
[0217]
The condition matching section detection unit 3 is not limited to detecting camera work or its change from the operation information of the camera, but can also detect from the image signal from the camera.
[0218]
As camera work that can be detected from the image signal from the camera, there are panning, tilting, zooming, booming, trimming, draining, cutting start, cutting end, etc., when detecting these camera work, An input image signal is detected by image recognition. Of course, these camera works may also be detected by detecting operation signals such as buttons used for camera operation as described in JP-A-6-165209 and JP-A-7-245754.
[0219]
[Seventh Embodiment]
In the seventh embodiment, the compression rate or the compression method is changed based on the reference state whether or not the audio information or the image information stored in the time-series information storage unit 4 is referenced (accessed) by the user. A case where information is compressed will be described.
[0220]
In general, frequently referenced information is important information, so image information of frequently referred sections is stored with high image quality, and image information that is less frequently referenced is compressed with a high compression rate, Try to save with a small amount of information.
[0221]
A reference state indicating how often the audio information or image information stored in the storage medium is accessed by the user is stored, and the compression rate is changed based on the reference state. For this purpose, the seventh embodiment includes a reference state storage unit for storing the reference frequency from the user.
[0222]
In this embodiment, the reference state storage unit stores the section of the image information stored in the time-series information storage unit 4 that has been reproduced by the user and the number of times the image has been reproduced. Store as a reference state.
[0223]
FIG. 32 is a diagram for explaining the storage state of the time-series information storage unit 4. In the figure, sections T2, T4, and T6 are condition matching sections detected by the condition matching section detection unit 3 of some embodiments described above. That is, for example, in the case of the first embodiment, it is an active section of dialogue. The other sections T1, T3, T5, and T7 are sections other than the condition matching section. Information regarding these sections is stored in the correspondence storage unit 5 as described above.
[0224]
FIG. 33 is a diagram illustrating an example of the storage state of the reference state storage unit. The reference state storage unit stores how many times the image information of the sections T1 to T7 has been accessed from the time when the image information is stored in the time-series information storage unit 4 until the present time, that is, how many times the video of the section is reproduced. I remember what was done.
[0225]
When an elapsed time since the image information is recorded in the time-series information storage unit 4 (that is, information storage time) is equal to or longer than a predetermined time, when a compression start instruction is generated from the time information storage unit 7, The reference state storage unit sends the information of the reference count table in FIG. 33 to the compression unit 6. The compression unit 6 includes a compression rate setting table shown in FIG. This compression rate setting table is a correspondence table of compression rates set in each of the condition matching section and sections other than the condition matching section with respect to the reference count.
[0226]
The compression unit 6 refers to the compression rate setting table and determines the image compression rate for each section. Then, the compression unit 6 compresses the image information stored in the time series information storage unit 4 at this compression rate.
[0227]
For example, the section T1 in FIG. 32 is a section other than the condition matching section, and the number of times of reference is 0 as shown in the table of FIG. 33. Therefore, based on the compression rate setting table of FIG. The compression rate is set to 90%. That is, since the section T1 is a section other than the condition matching section and has not been accessed by the user, it can be understood that the section T1 is an unimportant section. Therefore, at the time of compression, it is compressed at a high compression rate of 90%.
[0228]
On the other hand, a section T6 in FIG. 32 is a condition matching section, and the number of times of reference is five as shown in the table of FIG. 33. Therefore, based on the compression ratio setting table of FIG. Set to 10%. That is, the section T6 is a condition matching section and is a section that has been accessed five times by the user, and thus can be regarded as a very important section. Therefore, at the time of compression, image information is stored with high image quality with little compression.
[0229]
In the seventh embodiment, the reference state storage unit stores the relationship between the section and the reference count in the form of a table, but of course, it may be stored in another form such as a list or a stack. The numerical values in the compression rate setting table may be set by the user.
Then, as in the seventh embodiment described above, based on the reference state of how often the audio information or image information stored in the time-series information storage unit 3 is referenced (accessed) by the user, When information is compressed by changing the compression amount or compression method, frequently referenced information is important information, so the audio information or image information in the frequently referenced section is of high quality. The audio information or the image information that has been stored in and referenced less frequently can be compressed with a high compression rate and stored with a small amount of information.
[0230]
[Eighth Embodiment]
In the first, second, third, fourth, fifth, sixth, and seventh embodiments described above, the elapsed time since the audio information or the image information was recorded in the time-series information storage unit 4. The compression process is started when the information storage time (ie, the information storage time) is equal to or longer than a predetermined time. In the eighth embodiment, the free area in the time-series information storage unit 4 falls below a certain value. The compression process is started at the timing when it is recognized that the storage amount has been reached, or when the storage amount in the time-series information storage unit 4 is recognized to be greater than or equal to a certain value.
[0231]
Therefore, the processing operation at the time of storage is the same as that of the above-described embodiments, but the operation at the time of information compression is different.
[0232]
FIG. 35 is a diagram for explaining the operation at the time of information compression in the eighth embodiment, together with the flow of various information and the flow of output of each unit at that time. The information storage device of this embodiment includes a storage amount detection unit 31. When the storage amount detection unit 31 detects that image information has been recorded in excess of a pre-registered storage capacity, A compression processing start instruction is output to the correspondence relationship storage unit 5. The operation after this compression processing start instruction can be performed in the same manner as in the above-described embodiments.
[0233]
FIG. 36 is a flowchart of the process of the storage amount detection unit 31 in the eighth embodiment. If it is detected in step S800 that the information storage amount exceeds the predetermined amount, the process proceeds to step S801, and a compression processing start instruction is output to the correspondence relationship storage unit 5. For example, if the storage amount detection unit 31 is set to execute the compression process when information is recorded exceeding 90% of the storage capacity of the storage medium, the storage amount is stored. When 90% of the medium is reached, the storage amount detection unit 31 outputs a compression processing start instruction.
[0234]
The correspondence storage unit 5 that has received the compression start instruction from the storage amount detection unit 31 includes the condition matching sections and the image information stored in the time-series information storage unit 4 corresponding to each condition matching section. The storage address in the time series information storage unit 4 is output to the compression unit 6. Similarly to the above, the compression unit 6 performs data compression of the image information stored in the time-series information storage unit 4. Of course, in this case, the compression process may be executed in the background while recording a new image signal.
[0235]
In the case of this embodiment, the compression rate or the compression method may be set so that the data amount of the image information fits in a predetermined storage capacity. For example, when information is recorded to exceed 90% of the storage capacity of the storage medium, the use amount of the storage medium is reduced to 30% by the compression process. From this set value, the compression rate of the condition matching section and sections other than the condition matching section are calculated.
[0236]
For example, it is assumed that 10,000 frames of uncompressed images are accumulated in the time-series information storage unit 4. The 10,000 frames are divided into 2,000 frames for the condition matching section and 8000 frames for sections other than the condition matching section.
[0237]
At this time, a case where the frame thinning compression processing is performed so as to reduce the image information to 3000 frames will be described. Further, as a condition, it is assumed that the ratio between the compression rate of the condition matching section and the compression ratio of the sections other than the condition matching section is set to 1:10 in advance.
[0238]
In this case, if the compression rate of the condition matching section is a, the compression ratio of the sections other than the condition matching section is 10a.
[0239]
2000a + 8000 × 10a = 3000
Since the compression rate a that satisfies the condition is 0.0366, the compression rate of the condition matching section and the compression rate of the sections other than the condition matching section are 3.66% and 36.6%, respectively.
[0240]
If the uncompressed image 10000 frames stored in the time-series information storage unit 4 are divided into a condition matching section and a section other than the condition matching section, and frame compression is performed at each compression rate, the desired 3000 frames It is possible to reduce the image information.
As described above, according to the eighth embodiment, when audio or image information that is sequentially input exceeds the storage capacity of the storage medium, there is a value in which there is a free area in the time-series information storage unit. Since the compression process is started at the timing when it is recognized as below or when the amount of storage in the time-series information storage means is recognized as being greater than or equal to a certain value, the newly input voice or image Even when information is input exceeding the storage capacity of the time-series information storage means, there is an effect that input can be continued.
[0241]
[Ninth Embodiment]
As a known technique, there is known an apparatus that selects information at the time of recording, records only information recognized as important, or changes the compression rate. For example, Japanese Patent Application Laid-Open No. 7-129187 describes an apparatus that records audio before and after a voice capture key is pressed for a predetermined time. In addition, some commercially available tape recorders have a silent section detection function in which silent sections do not store voice.
[0242]
However, in the method of selecting information at the time of recording as in the apparatus described in Japanese Patent Application Laid-Open No. 7-129187, for example, the person who makes the most speech in the conference is specified, and the voice of the specified person's speech is specified. It is not possible to store only information or image information with high quality, or to create a digest by extracting in order from scenes with high importance so as to have a time length specified by the user. That is, there is a problem that it is impossible to compress audio information or image information based on information obtained for the first time after recording of audio information or image information, or information that cannot be obtained while recording.
[0243]
In this embodiment, a case where a compression method and a compression rate are set based on information obtained for the first time after recording of audio information or image information will be described.
[0244]
For example, in a meeting scene, a speaker's utterance continues for a long time, a scene in which communication items are being communicated, a scene in which a group of opinions are expressed, or a summary of discussions. It is often a scene where important remarks are stated. Therefore, after shooting a single meeting, the highest importance is assigned in order from the scene with the longest speech time, and when the information is compressed, the speech part to which the high importance is assigned is stored with high sound quality / high image quality, and the importance is low. The other parts are compressed at a high compression rate.
[0245]
As another example, for example, it is stored how long a pre-registered voice keyword has been used in a meeting, and a higher importance is assigned in order from a keyword with a longer usage time. It may be. For example, in a conference scene, discussions that have been discussed for a long time are often important discussions. Therefore, a keyword that can estimate the content of the discussion is registered in advance, and the keyword is detected from the input voice signal.
[0246]
Then, by detecting that a specific keyword has been used for a long time, it is recognized that the discussion corresponding to the keyword has been made for a long time, and this keyword appearance section is regarded as an important part. At the time of information compression, a section assigned a high importance is stored with high sound quality / high image quality, and other sections with a low importance are compressed with a high compression rate.
As in the seventh embodiment described above, the importance level of the audio information or the image information is determined by combining the detection results detected by the condition matching section detection unit 3, and the condition matching section and other information are determined based on this importance level. When the compression amount or compression method is changed for each section and data compression of audio information or image information is performed, the compression rate or intermittent recording according to a complex event that combines various events There is an effect that audio or image information can be stored at time intervals.
[0247]
[Tenth embodiment]
In the first to ninth embodiments described above, a condition matching section is detected by the condition matching section detection unit 3 when information is input, and the detection result and each condition matching section are supported. The audio information or the image information stored in the time-series information storage unit 4 is associated with the storage position in the correspondence storage unit 5 and the compression unit 6 is based on the correspondence information stored in the correspondence storage unit 5. The audio information or the image information stored in the time series information storage unit 4 is compressed.
[0248]
In the tenth embodiment, the condition matching section is not detected when information is input, and the condition matching section is detected by the condition matching section detection unit 3 when information is compressed. In the case of this embodiment, the correspondence storage unit 5 is not necessary.
[0249]
FIG. 37 is a diagram for explaining the recording operation in the tenth embodiment together with the flow of various information and the flow of output of each unit at that time. At the time of storage in the case of the tenth embodiment, input voice information and image information are sequentially stored in the time-series information storage unit 4. The time series information storage unit 4 stores the storage start time in the time information storage unit 7.
[0250]
Comparing FIG. 3 for explaining the operation at the time of storage in the first embodiment and FIG. 37 for explaining the operation at the time of storage in the tenth embodiment, in FIG. In addition, since the condition matching section detection process is not performed at the time of storage, the process is greatly simplified.
[0251]
FIG. 38 is a flowchart of processing during recording in the tenth embodiment. First, in step S900, the recording start time is stored in the time information storage unit 7, and the process proceeds to step S901. In step S901, audio information and image information are input by the audio information input unit 1 and the image information input unit 2, and in step S902, the input audio information and image information are stored in the time-series information storage unit 4. . Then, steps S901 and S02 are repeated.
[0252]
FIG. 39 is a diagram for explaining the operation at the time of compression in the tenth embodiment, together with the flow of various information and the flow of output of each unit at that time. In the tenth embodiment, for example, when a predetermined time elapses from the time of storage and a compression processing start instruction is generated from the time information storage unit 7, the condition matching section detection unit 3 receives audio from the time series information storage unit 4. Information is read and the condition matching section as described above is detected. In this embodiment, as in the first embodiment, a section in which dialogue is active is detected as a condition matching section. Then, the condition matching section detection unit 3 sends information on the detected condition matching section to the time series information storage unit 4 and also to the compression unit 6.
[0253]
The time-series information storage unit 4 receives the condition-matching section information from the condition-matching section detection unit 3 and the time information of the audio information or the image information stored in the time-series information storage unit 4 corresponding to the condition matching section. The storage address in the series information storage unit 4 is calculated. Then, the storage address is output to the compression unit 6.
[0254]
The compressing unit 6 receives the condition matching interval information from the condition matching interval detection unit 3 and the storage address from the time series information storage unit 4, and based on these input information, other than the condition matching interval and the condition matching interval The compression rate of the image information in the section is determined. That is, the compression rate of the image information in the condition matching section is determined to be low and the compression ratio of the image information in the section other than the condition matching section is determined to be high. For example, the condition matching section is not compressed, and the compression unit 6 receives the partial image sequence of the section other than the condition matching section from the time-series information storage unit 4 and compresses it to 1/10 as in the first embodiment. To do. Then, the compressed partial image sequence is written back to the time-series information storage unit 4.
[0255]
FIG. 40 is a flowchart of processing during compression in the case of the tenth embodiment. The processing shown in this flowchart corresponds to the processing content executed by the conference information accumulation processing unit 12 in FIG.
[0256]
That is, in the tenth embodiment, it is detected in step S1000 that the elapsed time since the audio information and the image information are recorded in the time-series information storage unit 4 has become a predetermined time or more. This is processing performed in the time information storage unit 7 in the functional block of FIG. When the information storage period exceeds a predetermined time, the process proceeds to step S1001, where a compression processing start instruction is generated and supplied to the condition matching section detection unit 3 as described above.
[0257]
In step S1002, the condition matching section detection unit 3 that has received the compression processing start instruction detects a condition matching section. In step S1003, it is determined whether a condition matching section is detected, and the determination result is sent to the time-series information storage unit 4. When the condition matching section detection unit 3 detects a condition matching section, the process proceeds to step S1004, and compression is performed at a low compression rate. As in the first embodiment, the information on the condition matching section in the time-series information storage unit 4 may not be compressed.
[0258]
On the other hand, when it is determined in step S1003 that it is a section other than the condition matching section, the process proceeds to step S1005, and the image information of the section other than the condition matching section stored in the time series information storage unit 4 is read, and the high compression rate For example, compression by thinning is performed to leave only the first frame of 10 frames.
[0259]
After steps S1004 and S1005, the process proceeds to step S1006, and the compressed image information is written back to the time-series information storage unit 4. Then, the process proceeds to step S1007, where it is determined whether or not the compression process has been completed for all the information stored in the time-series information storage unit 4. If not completed, the process returns to step S1003, and this step S1003 and subsequent steps are repeated. If the compression process has been completed for all information, this compression process routine is terminated.
[0260]
[Other variations]
In the first to tenth embodiments described above, in addition to the aspects specified in the description of each embodiment, the information storage device of the present invention is appropriately combined with each embodiment. It is also possible to configure.
[0261]
For example, the stepwise compression method described in the third embodiment is used in combination with the reference state storage method described in the seventh embodiment, so that the image is stepwise according to the reference state of the image information. The compression rate of information can be increased.
[0262]
Moreover, although 3rd Embodiment was demonstrated as a deformation | transformation of 2nd Embodiment, you may implement as a deformation | transformation of 1st Embodiment.
[0263]
Furthermore, the storage amount detection unit described in the eighth embodiment is combined with the first embodiment to the seventh embodiment, and the ninth embodiment to the tenth embodiment. Can be implemented.
[0264]
Further, the embodiments from the first embodiment to the ninth embodiment can be configured not to include the correspondence storage unit as described in the tenth embodiment.
[0265]
【The invention's effect】
As described above, according to the information storage device of the first aspect of the present invention, it is possible to store a large number of important portions of audio information or image information in which characteristic events occur in a limited storage medium. Moreover, there is an effect that even audio information or image information other than the important part can be stored for a long time with a small amount of data.
[0266]
In the case of the invention of claim 2, the condition matching section detection operation at the time of information compression becomes unnecessary, and there is an effect that the load on the system when compressing audio information or image information can be reduced.
[0267]
Further, in the case of the invention of claim 3, even when an event in which it is difficult to detect a change in state of an audio signal occurs or when a state change occurs in information not included in the input audio signal, A large number of important parts of information or image information in which characteristic events occur can be stored in a limited storage medium, and audio data or image information other than important parts can be stored in a small amount of data. It has the effect of memorizing for a long time.
[0268]
Further, in the case of the invention of claim 4, the condition matching section detection operation at the time of information compression becomes unnecessary, and there is an effect that it is possible to reduce the load on the system when compressing audio information or image information.
[Brief description of the drawings]
FIG. 1 is a functional block diagram showing the entirety of a first embodiment of an information storage device according to the present invention;
FIG. 2 is a diagram illustrating an outline of a system to which an embodiment of an information storage device according to the present invention is applied.
FIG. 3 is a diagram for explaining the sound level detection operation of the condition matching section detection unit according to the first embodiment.
FIG. 4 is a diagram for explaining an operation of detecting an active conversation section by the condition matching section detection unit according to the first embodiment.
FIG. 5 is a diagram showing a flow of operation during information recording according to the first embodiment.
FIG. 6 is a flowchart of the operation of a condition matching section detection unit in the first embodiment.
FIG. 7 is a flowchart of the operation of a correspondence relationship storage unit in the first embodiment.
FIG. 8 is a diagram for explaining a correspondence relationship between a detection result of a condition matching section detection unit and a memory address of a time-series information storage unit in the first embodiment.
FIG. 9 is a table for managing the correspondence between the detection result of the condition matching section detection unit and the memory address of the time-series information storage unit in the first embodiment.
FIG. 10 is a flowchart of the operation of the time-series information storage unit in the first embodiment.
FIG. 11 is a flowchart of the operation of a time information storage unit in the first embodiment.
FIG. 12 is a diagram illustrating a storage structure of a time information storage unit in the first embodiment.
FIG. 13 is a diagram illustrating another example of the storage structure of the time information storage unit in the first embodiment.
FIG. 14 is a diagram illustrating a flow of operation during information compression according to the first embodiment.
FIG. 15 is a flowchart of the operation of the compression unit in the first embodiment.
FIG. 16 is a diagram for comparing and explaining storage states of a time-series information storage unit before compression and after compression in the first embodiment.
FIG. 17 is a diagram illustrating a flow of operations during information recording according to the second embodiment.
FIG. 18 is a flowchart of processing performed by a frequency band-specific image generation unit according to the second embodiment.
FIG. 19 is a diagram illustrating a memory storage state before compression processing of a time-series information storage unit in the second embodiment.
FIG. 20 is a diagram illustrating an operation flow when compressing information according to the second embodiment;
FIG. 21 is a flowchart of compression processing in the second embodiment.
FIG. 22 is a diagram illustrating a memory storage state after compression processing in a time-series information storage unit according to the second embodiment.
FIG. 23 is a diagram for explaining a memory storage state at the time of information recording in the time-series information storage unit in the third embodiment;
FIG. 24 is a compression time management table for managing the time for executing stepwise compression in the third embodiment.
FIG. 25 is a diagram illustrating a memory storage state after one week has elapsed in the time-series information storage unit according to the third embodiment.
FIG. 26 is a diagram illustrating a memory storage state after one month has elapsed in the time-series information storage unit according to the third embodiment.
FIG. 27 is a diagram for explaining a memory storage state after six months in the time-series information storage unit in the third embodiment;
FIG. 28 is a table for managing keyword validity periods when it is detected that a keyword registered in advance in an audio signal appears in the fourth embodiment.
FIG. 29 is a table for managing a pattern valid period when it is detected that an audio signal pattern registered in advance in an audio signal appears in the fourth embodiment.
FIG. 30 is a table for managing the location and the importance of the location in association with each other when the condition matching section detection unit detects the location in the fifth embodiment.
FIG. 31 is a diagram illustrating compression rate setting processing when the condition matching section detection unit detects camera work in the sixth embodiment.
FIG. 32 is a diagram illustrating a condition matching section of a time-series information storage unit in the seventh embodiment.
FIG. 33 is a table for managing the storage state of the reference state storage unit in the seventh embodiment;
FIG. 34 is a table for managing the storage state of the compression rate setting table in the seventh embodiment.
FIG. 35 is a diagram illustrating a flow of operation during information compression according to the eighth embodiment.
FIG. 36 is a flowchart of processing of a storage amount detection unit in the eighth embodiment.
FIG. 37 is a diagram showing a flow of operations at the time of information recording in the tenth embodiment.
FIG. 38 is a flowchart of information recording processing in the tenth embodiment;
FIG. 39 is a diagram showing a flow of operations during information compression in the tenth embodiment.
FIG. 40 is a flowchart of information compression processing in the tenth embodiment.
[Explanation of symbols]
1 Voice information input section
2 Image information input section
3 Condition matching section detector
4 Time series information storage
5 Correspondence storage unit
6 Compression unit
7 Time information storage
8 Playback section
9 Control unit
10 Information storage device
11 Audio signal analyzer
12 Conference information storage processing device
13 Storage media
14 Monitor device
15 Microphone
16 meeting participants
21 Image generator by frequency band
31 Memory detection unit

Claims (22)

Information input means for inputting audio information and / or image information to be stored;
Time-series information storage means for compressing and storing the audio information and / or the image information input from the information input means;
Condition matching section detection means for detecting a condition matching section in which the voice information stored in the time series information storage means matches a predetermined condition set in advance;
Time information indicating the time when the audio information and / or the image information is stored in the time-series information storage unit is stored, and an elapsed time from the time indicated by the stored time information is set in advance. Elapsed time measuring means for outputting a compression processing start instruction when the time is over,
The time series is started by the compression processing start instruction from the elapsed time measuring means, and the condition matching section detected by the condition matching section detecting means and another section are changed in compression rate or compression method. Compression means for recompressing the data amount of the compressed audio information and / or image information stored in the information storage means;
An information storage device comprising: Information input means for inputting audio information and / or image information to be stored;
Time-series information storage means for compressing and storing the audio information and / or the image information input from the information input means;
Condition matching section detection means for detecting a condition matching section in which the voice information input from the information input means matches a predetermined condition set in advance;
The correspondence relationship between the section information indicating the condition matching section detected by the condition matching section detection means and the storage position of the audio information and / or the image information corresponding to the section information in the time-series information storage means. Correspondence storage means for storing;
Time information indicating the time when the audio information and / or the image information is stored in the time-series information storage unit is stored, and an elapsed time from the time indicated by the stored time information is set in advance. Elapsed time measuring means for outputting a compression processing start instruction when the time is over,
Based on the correspondence between the section information stored in the correspondence storage means and the storage position in the time series information storage means, activated by the compression processing start instruction from the elapsed time measurement means, the condition matching section And the other section, the compression rate or the compression method is changed, and the data amount of the compressed audio information and / or image information stored in the time series information storage unit is recompressed,
An information storage device comprising: Information input means for inputting audio information and / or image information to be stored;
Sensor information detecting means for detecting information from an external sensor;
With compression and storing the voice information and / or the image information inputted from said information input means, and the time-series data storing means for storing the sensor information from the sensor information detecting means,
Condition matching section detection means for detecting a condition matching section where the sensor information stored in the time series information storage means matches a predetermined condition set in advance;
Time information indicating the time when the audio information and / or the image information is stored in the time-series information storage unit is stored, and an elapsed time from the time indicated by the stored time information is set in advance. Elapsed time measuring means for outputting a compression processing start instruction when the time is over,
The time series is started by the compression processing start instruction from the elapsed time measuring means, and the condition matching section detected by the condition matching section detecting means and another section are changed in compression rate or compression method. stored in the information storage means, the data amount of the audio information is the compressed and / or image information, a compression means for recompressing,
An information storage device comprising: Information input means for inputting audio information and / or image information to be stored;
Time-series information storage means for compressing and storing the audio information and / or the image information input from the information input means;
Sensor information detecting means for detecting information from an external sensor;
Condition matching section detection means for detecting a condition matching section in which sensor information from the sensor information detection means matches a predetermined condition set in advance;
The correspondence relationship between the section information indicating the condition matching section detected by the condition matching section detection means and the storage position of the audio information and / or the image information corresponding to the section information in the time-series information storage means. Correspondence storage means for storing;
Time information indicating the time when the audio information and / or the image information is stored in the time-series information storage unit is stored, and an elapsed time from the time indicated by the stored time information is set in advance. Elapsed time measuring means for outputting a compression processing start instruction when the time is over,
Based on the correspondence between the section information stored in the correspondence storage means and the storage position in the time series information storage means, activated by the compression processing start instruction from the elapsed time measurement means, the condition matching section Compression means for recompressing the data amount of the compressed audio information and / or image information stored in the time-series information storage means by changing the compression rate or the compression method in the other sections When,
An information storage device comprising: In claim 1, claim 2, claim 3, or claim 4,
The information storage is characterized in that the image information of the condition matching section stored in the time series information storage means is subjected to data compression that maintains higher quality than the image information of other sections by the compression means. apparatus. In claim 1, claim 2, or claim 5,
The condition matching section detecting means compares a voice signal level of the voice information with a predetermined threshold, and detects a start point or an end point of the condition matching section based on the comparison result. Information storage device. In claim 1, claim 2, or claim 5,
The condition matching section detecting means detects a specific sender of voice or a change of sender in the voice information, and detects a start point or an end point of the condition matching section based on the detection result. Characteristic information storage device. In claim 1, claim 2, or claim 5,
The condition matching section detecting means detects a specific keyword or a specific pattern predetermined in the voice information, and detects a start point or an end point of the condition matching section based on the detection result. Information storage device. In claim 1, claim 2, or claim 5,
The sensor information detection means detects information related to a location where the audio information and / or the image information is input or a location where the sensor information is detected,
The image forming apparatus according to claim 1, wherein the condition matching section detecting unit detects a start point or an end point of the condition matching section based on the sensor information. In claim 3, claim 4, or claim 5,
The sensor information detection means detects a specific person by the external sensor,
The condition matching section detecting means detects a start point or an end point of the condition matching section based on the sensor information. In claim 3, claim 4, or claim 5,
The sensor information detecting means detects a camera operation signal or a change in the camera operation signal,
The condition matching section detecting means detects a start point or an end point of the condition matching section based on the sensor information. In claim 1, claim 2, claim 3, claim 4 or claim 5,
When the compression means is recognized that the free space in the time series information storage means has become a certain value or less, or when it is recognized that the storage amount in the time series information storage means has become a certain value or more the information storage apparatus characterized by performing the recompression process in the compression means. In claim 2, claim 4, or claim 5,
The compression unit is configured to store the audio information in the condition matching section or other sections stored in the correspondence storage unit so that the data amount of the audio information and / or the image information is within a predetermined storage capacity. And / or setting a compression rate or compression method of the image information. In claim 1, claim 2, claim 4, or claim 5,
The time series information storage means stores the audio information and / or the image information input from the information input means for each frequency band,
The compression means, at the time of compression, at least a high frequency band of a section other than the condition matching section of the audio information and / or the image information stored in the time series information storage means. An information storage device for erasing from a storage medium. In claim 2, claim 4, or claim 5,
The time series information storage means, when storing the audio information and / or the image information input from the information input means for each frequency band, a condition matching section detected by the condition matching section detection means, and a condition matching Change the way of dividing the frequency band in the sections other than the section and store it,
The compression means deletes at least a high frequency band in a section other than the condition matching section of the audio information and / or the image information stored in the time-series information storage means at the time of compression. Storage device. In claim 1, claim 2, claim 3, claim 4 or claim 5,
The time-series information storage means stores the image information input from the information input means separately into a luminance signal and a color signal,
The information storage device, wherein the compression unit deletes at least the color information in a section other than the condition matching section in the image information stored in the time-series information storage unit during compression. In claim 1, claim 2, claim 3, claim 4 or claim 5,
A reference state storage means for storing information that distinguishes a section that has been referred to by a user a predetermined number of times within a predetermined time and a section that has not been referred to by the user more than the predetermined number of times within the predetermined time;
Based on the information stored in the reference state storage unit, the compression unit is referred to by the user a predetermined number of times within the predetermined time, and has not been referred to by the user the predetermined number of times within the predetermined time. The information is characterized in that the data amount of the compressed audio information and / or image information stored in the time-series information storage means is recompressed by changing the compression rate or the compression method for each section. Storage device. In claim 1, claim 2, claim 3, claim 4 or claim 5,
The compression means determines the importance of the audio information and / or the image information by combining the detection results detected by the condition matching section detection means, and based on this importance, the detection by the condition matching section detection means The amount of compressed audio information and / or image information stored in the time-series information storage means is changed by changing the compression amount or compression method between the interval determined from the result and another interval. An information storage device characterized by compressing . A time-series information storage step of compressing voice information and / or image information input through the information input means and storing the compressed information in the time-series information storage means;
A condition matching section detecting step for detecting a section in which the voice information stored in the time series information storing step matches a predetermined condition set in advance;
A time information storage step of storing time information indicating a time at which the audio information and / or the image information is stored in the time-series information storage unit;
An elapsed time measuring step of measuring whether an elapsed time from the time indicated by the time information stored in the time information storage step is equal to or greater than a preset time;
The elapsed time measurement step is activated when the elapsed time is detected to be equal to or longer than the preset time, and the compression rate is reduced between the condition matching section detected in the condition matching section detection step and another section. Alternatively, a compression step of changing the compression method and recompressing the data amount of the compressed audio information and / or image information stored in the time-series information storage means,
An information storage method comprising: A time-series information storage step of compressing voice information and / or image information input through the information input means and storing the compressed information in the time-series information storage means;
A condition matching section detecting step for detecting a section in which the input voice information matches a predetermined condition set in advance;
A correspondence relationship between the section information indicating the condition matching section detected in the condition matching section detection step and the storage position in the time-series information storage unit of the audio information and / or the image information corresponding to the section information is stored. A correspondence storage process to
A time information storage step of storing time information indicating a time at which the audio information and / or the image information is stored in the time-series information storage unit;
An elapsed time measuring step of measuring whether an elapsed time from the time indicated by the time information stored in the time information storage step is equal to or greater than a preset time;
When the elapsed time measuring step detects that the elapsed time is equal to or greater than the preset time, the section information stored in the correspondence storage step and the storage position in the time series information storage means The compressed audio information and / or image stored in the time-series information storage means by changing the compression rate or the compression method between the condition matching section and another section based on the correspondence relationship of A compression process for recompressing the amount of information data ;
An information storage method comprising: A sensor information detection step for detecting information from an external sensor;
A time-series information storage step for storing in the time-series information storage means information obtained by compressing audio information and / or image information input through the information input means, and sensor information detected in the sensor information detection step;
A condition matching section detecting step for detecting a section in which the sensor information stored in the time-series information storage means matches a predetermined condition set in advance;
A time information storage step of storing time information indicating a time at which the audio information and / or the image information is stored in the time-series information storage unit;
An elapsed time measuring step of measuring whether an elapsed time from the time indicated by the time information stored in the time information storage step is equal to or greater than a preset time;
The elapsed time measurement step is activated when the elapsed time is detected to be equal to or longer than the preset time, and the compression rate is reduced between the condition matching section detected in the condition matching section detection step and another section. Alternatively, a compression step of changing the compression method and recompressing the data amount of the compressed audio information and / or image information stored in the time-series information storage means,
An information storage method comprising: A time-series information storage step of compressing voice information and / or image information input through the information input means and storing the compressed information in the time-series information storage means;
A sensor information detection step for detecting information from an external sensor;
A condition matching section detecting step for detecting a condition matching section in which the sensor information detected in the sensor information detecting step matches a predetermined condition set in advance;
A correspondence relationship between the section information indicating the condition matching section detected in the condition matching section detection step and the storage position of the audio information and / or the image information corresponding to the section information in the time-series information storage unit. A correspondence storing step for storing;
A time information storage step of storing time information indicating a time at which the audio information and / or the image information is stored in the time-series information storage unit;
An elapsed time measuring step of measuring whether an elapsed time from the time indicated by the time information stored in the time information storage step is equal to or greater than a preset time;
When the elapsed time measuring step detects that the elapsed time is equal to or greater than the preset time, the section information stored in the correspondence storage step and the storage position in the time series information storage means The compressed audio information and / or image stored in the time-series information storage means by changing the compression rate or the compression method between the condition matching section and another section based on the correspondence relationship of A compression process for recompressing the amount of information data ;
An information storage method comprising:

Images