JP4883171B2 - Information processing system - Google Patents

Description

  The present invention relates to an information processing system that is applied to accumulate audiovisual information and biological information related to an event experienced by an individual.

  As the amount of digital information such as digital image information possessed by an individual increases, a service for storing the digital information of each individual is being put into practical use. In that case, a service that makes it easy for an individual to create his / her own history (self history) is proposed rather than simply storing data (see Patent Document 1). In the self-history construction system described in this document, a user terminal and an administrator (server computer) are connected via the Internet, and the administrator has a database group. The database group includes a user database for user management, a personal event database, a history event database, an event selection database, and a chronological table database. The user activates a WWW browser on the user terminal and performs a process of creating a chronology as his / her own history using the information in the database described above. By providing a historical event database, it is possible to easily create a personal history by selecting a historical event to be compared with an individual event.

Japanese Patent Laid-Open No. 11-66049

  In the document described above, personal event information is centered on characters, and the associated image and audio information is not information that the user has seen or heard. Therefore, the information related to the personal event is not much different from that presented by the electronic album except that it is configured as a chronological table. Further, when trying to reproduce the user's experience later, it was insufficient as information. Furthermore, since the administrator himself / herself has a database, there is a problem that the cost for constructing and managing the database is high.

  Therefore, the object of the present invention is to enable the accumulation of audiovisual information that the user sees and hears and the biometric information of the user, and to reproduce the user's experience later, and has been made public. An object of the present invention is to provide an information processing system capable of enriching information contents using a database.

In order to solve the above problems, the present invention is worn by the user, the audio visual information signal acquisition means for acquiring audiovisual information signal comprising at least one of visual information and auditory information,
A biological information signal acquisition unit that is attached to the user and acquires a biological information signal corresponding to the biological information of the user;
At least audiovisual information signal determined to be valid information by a change of the biological information signal selectively writes in the storage unit, control means for write unnecessarily controlling write attribute data associated with the audiovisual information signal input by the input means When,
A part of the attribute data by searching the external database as a keyword and related information obtaining means for obtaining related information from an external database,
An information processing system comprising: editing means for editing a stored audiovisual information signal using related information correlated with the audiovisual information signal among the related information acquired by the related information acquisition means .

  According to the present invention, an audiovisual signal and a biological signal when a user actually experiences are acquired, and an impressive signal among them is selectively accumulated. Biometric information is used for this selection. In addition, since audiovisual signals and biological signals are accumulated in the database in the present invention, it is possible to reproduce them with a sense of experience at the time of re-experience, and a realistic reproduction that cannot be obtained with photographs or video data. Experience is possible. Furthermore, according to the present invention, information related to an external database can be acquired in addition to data acquired by an individual, and the contents of the database can be complemented. Therefore, the contents of the database can be rich and the quality can be improved.

It is a block diagram which shows the outline of the whole system of one Embodiment of this invention. It is a block diagram of an example of the signal acquisition and encoding part in one Embodiment of this invention. It is a block diagram which shows an example of the integrated encoding part which can be used for this invention. It is a basic diagram used for description of integrated encoding. It is a block diagram which shows an example of the decoding apparatus of integrated encoding. It is a block diagram which shows an example of the accumulation | storage part in the signal acquisition apparatus in one Embodiment of this invention. It is a block diagram used for description of the recording trigger detection part in a storage part. It is a block diagram used for description of the recording trigger detection part in a storage part. It is a block diagram used for description of the recording trigger detection part in a storage part. It is a block diagram used for description of the recording trigger detection part in a storage part. It is a block diagram of an example of a rental integrated database in one embodiment of this invention. It is a flowchart for demonstrating an example of the data transmission process with respect to a rental integrated database from a signal acquisition part. It is a flowchart for demonstrating the other example of the data transmission process with respect to a rental integrated database from a signal acquisition part. It is a flowchart for demonstrating the further another example of the data transmission process with respect to a rental integrated database from a signal acquisition part. It is a basic diagram which shows the format of the attribute data in one Embodiment of this invention. It is a basic diagram for demonstrating the specific example of attribute data. It is a flowchart for demonstrating operation | movement of the rental integrated database in one Embodiment of this invention. It is a flowchart for demonstrating operation | movement of the rental integrated database in one Embodiment of this invention. It is a block diagram which shows the structure of an example of the integrated signal presentation apparatus in one Embodiment of this invention. It is a flowchart for demonstrating operation | movement of the integrated signal presentation apparatus in one Embodiment of this invention.

  Hereinafter, an embodiment of the present invention will be described. First, a schematic configuration of a signal acquisition, storage, and presentation system will be described with reference to FIG. Reference numeral 1 denotes a signal acquisition unit for each individual (user), which includes a signal acquisition and encoding unit 2 and a storage unit 3. The signal acquisition and encoding unit 2 collects the audiovisual information and biological information of the user from a plurality of sensors attached to the user, and integrally encodes the collected signals. The signal obtained by integration coding is referred to as an integration signal. The signal acquisition and encoding unit 2 has a portable configuration, that is, a so-called mobile terminal configuration so that the user can carry it. The integrated signal is stored in a removable storage medium such as a card-like semiconductor memory.

  The storage unit 3 can be realized by, for example, a personal computer installed in the user's home. A storage medium is inserted into the storage unit 3, and the integrated signal is read from the storage medium. Then, the storage unit 3 selects valid information from the integrated signal based on comprehensive judgment, and temporarily stores the selected valid information in a storage medium such as a hard disk. The integrated signal may be transmitted from the signal acquisition and encoding unit 2 to the storage unit 3 by wired or wireless communication instead of the storage medium. Thus, the signal acquisition and encoding unit 2 and the storage unit 3 are connected via a personal network.

  The signal acquisition unit 1 (storage unit 3) is connected to the rental integrated database 6 via the communication terminal 4 and the public network 5. The communication terminal 4 is a wired or wireless communication terminal. The integrated signal is encrypted to ensure security during transmission. The rental integrated database 6 includes an individual database with a predetermined capacity allocated to each user. The user confirms the communication connection with a specific authentication means from the communication terminal 4, transmits the encrypted integrated signal to the rental integrated database 6, and accumulates the integrated signal in the database assigned to the user. The accumulated data is the transmitted encrypted integrated signal and attribute data. The company that manages the rental integration database 6 manages the accumulated data by concluding individual contracts with a plurality of users. For example, by collecting usage fees according to the contract period and data amount from the users, the rental integration The database 6 is operated.

  The rental integrated database 6 is connected to various public databases 7 and is automatically linked to related information by a method described later. The public database 7 means an accessible database, and can be free of charge for use. In addition, the rental integrated database 6 and the integrated signal presentation device 8 are connected. The integrated signal presentation device 8 is installed in the re-experience center. When the user desires to re-experience the past experience, the user can re-experience with the integrated signal presentation device 8.

  In practice, the system shown in FIG. 1 is often operated by a plurality of companies. Profits can be gained by manufacturing and selling the signal acquisition and encoding unit 2. Profits can be obtained by manufacturing and selling the hardware or software of the storage unit 3. Furthermore, a usage fee can be obtained based on a usage contract for the rental integrated database 6. When the rental integrated database 6 is linked to the public database 7, an information fee is paid to the management company of the public database 7. The integrated signal presentation device 8 can obtain a usage fee by providing a re-experience. It is practical that a system is operated by a plurality of companies from the viewpoint that these business benefits can be obtained.

  Each component constituting the system described above will be described in more detail. First, an example of the signal acquisition and encoding unit 2 of the signal acquisition unit 1 will be described with reference to FIG. In order to acquire an audiovisual signal and a biological signal, an image input unit 11 using a CCD or the like, an audio input unit 12 using a microphone, a temperature detection unit 13 for measuring body temperature, and a muscle for detecting myoelectric potential A potential detection unit 14 and a perspiration detection unit 15 that detects perspiration are provided. You may provide the detection part for inputting biological information other than what was shown in figure. For example, information such as heart rate and blood pressure may be acquired.

  Sensors constituting these detection units are attached to the user. For example, the CCD of the image input unit 11 is attached to a frame of glasses or the like so as to generate an image signal related to an image that is as close as possible to the image that the user actually sees. In particular, a CCD that can perform wide-angle imaging is desirable. Other maccrophones and sensors of the detection unit can also be attached to the frame of the glasses.

  The output signals of the input unit and the detection unit described above are supplied to the A / D converters 16, 17, 18, 18, and 20 through amplifiers, and converted into digital signals. Digital signals from the A / D converters 16 to 20 are supplied to the signal processing units 21, 22, 23, 24, and 25, respectively. Each of the signal processing units 21 to 25 performs processing such as optimum noise reduction processing, removes unnecessary signal components such as noise, and converts or shapes the signals into easy-to-process signals. The respective output signals of the signal processing units 21 to 25 are encoded by the integrated encoding unit 26 and converted into integrated signals. Then, it is recorded on a removable storage medium such as the flash memory card 27 as a series of signals. The control unit 28 controls an operation of writing the integrated signal from the integrated encoding unit 26 to the flash memory card 27, an operation of reading the integrated signal from the flash memory card 27, and the like.

  FIG. 3 shows an example of the integrated encoding unit 26. The above-described image, sound, and various biological signals each have certain physical characteristics. For example, an image has a property that local correlation is high in the spatial and temporal directions. Since the amount of information is the largest amount of image data, integrated coding is performed so that the amount of information does not increase by embedding other data in the image data. FIG. 3 is a configuration example of an integrated encoding unit that embeds audio data in, for example, image data.

  Image data is supplied to the line memory 31. The image data has a sampling frequency of 13.5 MHz and is quantized to 8 bits per pixel. This image data is 720 pixels in the horizontal direction, 480 lines in the vertical direction, and 30 frames / second in the time direction. The image data for one line stored in the line memory 31 is supplied to the barrel shifter 32 in parallel. The output of the barrel shifter 32 is stored in the encoded data memory 33.

  Audio data is supplied to the barrel shifter 32 as a signal for controlling the shift amount. The audio data is 9-bit data per sample with a sampling frequency equal to the line frequency of 15.7 kHz. Therefore, one sample of audio data is generated for one line of the image. Barrel shift means performing a shift operation in a ring shape.

  FIG. 4A is an example of one-frame image data. FIG. 4A shows an original image in which a central portion in one image is a square dark area as a simple example. Lines L2, L3,..., L478, L479, and L480 exist from the uppermost line L1. There are 720 samples in the lateral direction of each line. However, in the drawing, for the sake of simplicity, the number of lines is considerably smaller than 480.

  The value of the audio data in which one sample is generated per line varies from 0 to 511. Corresponding to the value of the audio data, the data of each line of the image is barrel-shifted to the right in the horizontal direction. The data of the first line L1 or the line positioned at a predetermined interval is not subjected to the barrel shift operation, and the image data of the line not subjected to the barrel shift is used as the reference data at the time of decoding. If the image data of one line has exactly the same value, such as lines L2, L3, L478, L479, and L480, the result of the barrel shift is the same data, so these lines are not suitable for encoding. It is.

  FIG. 4B shows image data as a result of shifting each line in the image of FIG. 4A by audio data. This image data (integrated signal) is stored in the encoded data memory 33. The flash memory card 27 in FIG. 2 corresponds to the encoded data memory 33. As shown in FIG. 4B, encoding is performed so that the continuity in the vertical direction is broken by the audio data while maintaining the continuity in the horizontal direction of the image without increasing the information amount of the image data.

  FIG. 5 shows an example of a decoding apparatus for joint encoding. The encoded data memory 33 stores image data that has been integrally encoded. Data for one line read from the encoded data memory 33 is supplied to the barrel shifter 34. The output of the barrel shifter 34 is supplied to the line memory 35 and the correlation detection unit 36. The line memory 35 stores the data of the first line L1 as an initial value. The image data of the barrel shifter 34 is encoded, while the data stored in the line memory 35 is decoded image data one line before.

  The correlation detector 36 is supplied with the output of the barrel shifter 34 and the output of the line memory 35. When the shift amount of the barrel shifter 34 is changed to 0, 1, 2,..., 511, the correlation between the data for one line of the barrel shifter 34 and the data for one line of the line memory 35, that is, vertical Calculate direction correlation. Then, the shift amount that maximizes the correlation in the vertical direction is detected. Decoded image data is output from the line memory 35. Further, the shift amount of the barrel shifter 34 is supplied to the latch 37. A latch pulse generated when the correlation detection unit 36 detects that the correlation is maximum is supplied to the latch 37, and the shift amount when the correlation is maximum is taken into the latch 37. Audio data is output from the latch 37. Note that the above-described integrated coding is an example, and embedded coding described in Japanese Patent Laid-Open No. 2000-59743 may be used.

  FIG. 6 shows an exemplary configuration of the storage unit 3 of the signal acquisition unit 1. In the storage unit 3, the integrated signal read from the flash memory card 27 is supplied to the encryptor 41 and the integrated decryption unit 42. The integrated signal encrypted by the encryptor 41 is accumulated in the hard disk 44 via the write circuit 43. Further, the accumulated signal is read from the hard disk 44 via the read circuit 45. The hard disk is an example of a storage medium, and a storage medium such as an optical disk, a magnetic tape, or a semiconductor memory may be used.

  The integrated signal read from the flash memory card 27 is decoded by the integrated decoding unit 42. From the integrated decoding unit 42, an image signal, an audio signal, a body temperature signal, a myoelectric potential signal, and a sweat signal are obtained separately. These body sensation signals (audiovisual signals and biological signals) are supplied to the recording trigger detection unit 46. The recording trigger detection unit 46 generates a trigger signal R indicating whether or not it is a sensation signal to be recorded on the hard disk 44, and supplies this trigger signal R to the system control and address generation unit 47. As will be described later, the determination as to whether or not the signal is an effective sensory signal to be recorded on the hard disk 44 is made based on a plurality of sensory signals. Using this trigger signal R, audiovisual signals that the user feels impressive can be selected and recorded on the hard disk 44.

  The system control and address generation unit 47 controls the entire processing of the storage unit 3. Based on the trigger signal R described above, the system control and address generator 47 controls the address from which the integrated signal from the flash memory card 27 is read, and reads the integrated signal determined to be valid. The read integrated signal is encrypted by the encryptor 41 and written to the hard disk 44 via the write circuit 43. The write circuit 43 and the hard disk 44 are controlled by the system control and address generation unit 47. The system control and address generation unit 47 also controls the read operation of the hard disk 44, and a read signal is output via the read circuit 45.

  When the recording trigger detection unit 46 determines an integrated signal to be recorded, control is performed so that the integrated signal is read from the flash memory card 27. Further, a delay element for delaying the integrated signal may be provided between the flash memory card 27 and the encryptor 41 in the recording trigger detection unit 46 for a time necessary to determine the integrated signal to be recorded.

  Reference numeral 48 denotes an attribute data generation unit. When the user operates the terminal (signal acquisition unit 1) and inputs desired attribute data, attribute data related to the integrated signal is generated. For example, event names such as athletic meet and Olympics are used as attribute data. In addition, the date, time, place, etc. experienced are used as attribute data. The attribute data is an experience keyword, and a specific example thereof will be described later. The attribute data is also supplied to the encryptor 41.

  The encryptor 41 receives the integrated signal read from the flash memory card 27 and the attribute data from the attribute data generation unit 48, and encrypts these integrated signal and attribute data using the key signal 49. . Encryption is necessary for privacy protection. As an example, biometric data for personal authentication (such as a fingerprint) recorded separately is used as the key signal 49.

  An example of the recording trigger detection unit 46 in FIG. 6 will be described. FIG. 7 shows a configuration of processing relating to an image signal, and a trigger signal V related to the image signal is generated by the configuration of FIG. The image signal from the integrated decoding unit 42 is supplied to the frame memory 51 and the subtracter 52. The output of the frame memory 51 is supplied to the subtracter 52. The subtractor 52 calculates a difference between the supplied image signals and supplies the calculated difference to the absolute value calculation circuit 53. The absolute value calculation circuit 53 calculates an absolute value for the supplied difference and supplies the calculated difference absolute value to the integration circuit 54. The integration circuit 54 integrates the supplied difference absolute value over a predetermined number of frames, and supplies the integration value to the comparison circuit 55.

  The comparison circuit 55 is supplied with a predetermined threshold value, and a signal indicating the comparison result between the threshold value and the integrated value is supplied to the trigger generation circuit 56. The trigger generation circuit 56 generates a trigger signal V according to the supplied signal. Here, the conditions for generating the trigger signal V include when the object in the image moves, when the CCD moves with the wearer, or when the overall luminance level changes, etc. In this case, the determination can be made based on the inter-frame difference. A configuration may be used in which a motion vector is extracted by a method such as block matching and a trigger signal is generated by performing processing based on the extracted motion vector.

  Next, FIG. 8 shows a configuration for generating a trigger signal T related to the body temperature signal detected by the temperature detector. The body temperature signal from the integrated decoding unit 42 is supplied to the delay element 61 and the subtractor 62. The delay element 61 delays the supplied signal for a predetermined period and supplies it to the subtractor 62. The subtractor 62 calculates a difference between the two supplied signals, and supplies the calculated difference to the absolute value calculation circuit 63. The absolute value calculation circuit 63 calculates an absolute value for the supplied difference, and supplies the calculated difference absolute value to the comparison circuit 64.

  A predetermined threshold value is supplied to the comparison circuit 64, and a signal indicating the comparison result between this threshold value and the output of the absolute value calculation circuit 63 is supplied to the trigger generation circuit 65. The trigger generation circuit 65 generates a trigger signal T according to the supplied signal. With the above configuration, a trigger signal is generated when the body temperature of the wearer changes due to some kind of stimulus being applied to the wearer. The delay period by the delay element 61 may be suitably set to detect a change in body temperature.

  FIG. 9 shows a configuration for generating a trigger signal K related to the myoelectric potential signal detected by the myoelectric potential detection unit. The myoelectric potential signal from the integrated decoding unit 42 is supplied to the absolute value calculation circuit 71. The output of the absolute value calculation circuit 71 is supplied to the peak detection circuit 72. The peak detection circuit 72 detects a peak value from the supplied signal and supplies the detected peak value to the comparison circuit 73. Here, a reset signal is supplied to the peak detection circuit 72 at regular time intervals.

  Each time the reset signal is supplied, the peak value previously detected by the peak detection circuit 72 is reset. In this way, peaks are detected at regular time intervals. The comparison circuit 73 is supplied with a predetermined threshold value, and a signal indicating the comparison result between this threshold value and the output of the peak detection circuit 72 is supplied to the trigger generation circuit 74. The trigger generation circuit 75 generates a trigger signal K according to the supplied signal. In this way, whether or not the trigger signal K can be generated is determined at regular time intervals.

  Although not shown, a trigger signal H corresponding to a sudden increase in the amount of sweat is also generated based on the sweat signal detected by the sweat detection unit.

  With the configuration described above, a trigger signal is generated when the myoelectric potential changes greatly to some extent due to stimulation applied to the wearer, movement of the wearer, or the like. For example, if a myoelectric potential sensor is attached to a facial muscle, a trigger signal is generated based on a change in facial expression. A configuration in which a trigger signal is generated by performing processing based on a difference in myoelectric potential pulse (peak) interval may be used.

  Further, the recording trigger detection unit 46 makes a comprehensive determination based on each trigger signal generated as described above, and generates a signal R indicating the writing effective period so as to accumulate only effective sensation signals. . An example of the configuration for the integrated determination is shown in FIG. Here, a case where the above-described four types of trigger signals V, T, K, and H are supplied as trigger signals is shown. Trigger signals V, T, K, and H are supplied to the AND circuit 81. The AND circuit 81 generates an AND signal based on the trigger signals V, T, K, and H, and supplies the generated AND signal to the waveform shaping circuit 82. The waveform shaping circuit 82 performs waveform shaping processing such as removal of isolated points on the supplied signal. The output of the waveform shaping circuit 82 is supplied to the synchronization circuit 83.

  On the other hand, the control circuit 84 generates a signal related to the synchronization information in accordance with a command from the system control and address generation unit 47 and supplies the generated signal to the synchronization circuit 83. The synchronization circuit 83 performs processing for synchronizing the signal output from the waveform shaping circuit 82 in accordance with system control and an instruction from the address generation unit 47. As a result of such processing, a signal R indicating the write effective period is output from the synchronization circuit 83. This signal R is supplied to the system control and address generator 47. Thereby, the integrated signal of the writing effective period is encrypted and written to the hard disk 44. By the output signal R of the recording detection unit 46 described above, among the audiovisual signals, an impressive signal that causes a change related to the living body is selectively and automatically written to the hard disk 44. Further, the biological signal at that time is also written in the hard disk 44. The biometric signal is used to determine whether or not the user feels impressive, and is stored to allow re-experience at a later date.

  It should be noted that when the trigger signal includes signals other than V, T, K, and H, the trigger signal supplied to the AND circuit 81 in the configuration as described above can be used even when the trigger signal is other than four types. A signal R indicating a signal to be written (a period is specified) can be generated by changing the type and number.

  Next, the rental integrated database 6 will be described with reference to FIG. The encrypted integrated signal (including attribute data) stored in the hard disk 44 of the storage unit 3 of the signal acquisition unit 1 described above is input to the rental integrated database 6 via the communication terminal 4 and the public network 5. In addition, as shown in FIG. 1, data can be input / output to / from the integrated signal presentation device 8 installed in the re-experience center. To access desired data, search for desired data, and download the searched desired data.

The rental integrated database 6 is a database 90 _{1 to} 90 _n that is rented individually.
It has. Those who can use the database are registered users. When the registration is completed, for example, an authentication password (personal ID) is issued. Information that can identify the user, such as fingerprint information, may be used. When the user registers his / her attribute data and integrated signal in his / her database, the personal authentication unit 91 performs authentication (identification). When the authentication is established, the attribute data and the integrated signal encrypted by the writing / reading control unit 92 are registered in the own database.

Further, the company that manages the rental integrated database 6 executes an automatic search and an automatic link program for the public network 5, and based on the attribute data registered in the databases 90 _{1 to} 90 _n , Search for information. Based on the search result, the link destination information is automatically written in the attribute data. In addition, data related to the integrated signal already stored in the database of the rental integrated database 6 is obtained from the public database 7 by linking to the panic database 7 based on the link destination information in the attribute data, and stored in the database. The registered integrated signal is rewritten.

  For such a link to the public database 7, acquisition of information, etc., an encryption decoder 93, an integrated encoding decoding unit 94, an attribute data extraction, editing unit 95, data to be linked and edited. The rental integrated database 6 includes a processing unit 96, an integrated encoding unit 97, and an encryptor 98. The data processing unit 96 also has an interface function with the public database 7.

  The integrated signal read from the personal database and decoded is supplied to the data processing unit 96. Further, the attribute data is extracted by the extraction / editing unit 95, and the extracted attribute data is supplied to the data processing unit 96. The data processing unit 96 executes automatic search and automatic link program for the public network 5 to search for information related to the attribute data. A database of searched information is written in attribute data as a link destination. A process of writing the link destination information as attribute data is one of the editing processes.

  Related information is downloaded to the data processing unit 96 from the public database 7 accessed based on the link destination information. The data processing unit 96 checks whether or not there is a correlation between the decoded integrated signal and the downloaded information, and selects correlated data. The correlated data is recorded in the database via the encoder 97, the encryptor 98, and the write / read controller 92. That is, a part of information in the database is rewritten by related information acquired from the public database 7.

  A data transmission procedure from the storage unit 3 of the signal acquisition unit 1 to the rental integrated database 6 will be described with reference to the flowchart of FIG. The left side of FIG. 12 shows the process in the signal acquisition unit 1, and the process S30 surrounded by the broken line on the right side shows the process in the rental integrated database 6. In step S1, the acquisition unit 1 is connected to the storage unit 3. In step S2, connection to the rental integrated database 6 is made. Thereby, a communication path through the communication terminal 4 is established.

  In step S31, preparations for starting communication are made. The personal ID and password are transmitted from the acquisition unit 1 to the rental integrated database 6. In step S32, personal authentication is performed. In step S33, it is determined whether authentication has been established. When authentication is established, communication is started (step S34), and an integrated signal is read from the storage unit 3 of the acquisition unit 1 (step S4). The read integrated signal is encrypted, and the encrypted integrated signal is transmitted (step S5). The encrypted integrated signal received in step S35 is recorded in the integrated database (step S36).

  The acquisition unit 1 monitors the end of reading of the integrated signal in step S6, and the rental integrated database 6 monitors the end of recording of the integrated signal in the integrated database (step S37). When the reading is finished, the communication is finished (step S38). Whether or not both operations are completed is monitored in step S7. When the end is determined in step S7, the communication is disconnected (step S8). In order to protect the confidentiality of the data, the transmitted integrated signal is deleted from the stored contents of the storage unit 3.

  FIG. 13 is a flowchart of another example of a data transmission procedure from the signal acquisition unit 1 to the rental integrated database 6. Transmission is automatically performed by the timer of the communication terminal 4. First, in step S11, transmission schedule data is read. The transmission schedule data is generated in processing S10 of the transmission scheduler. That is, automatic transmission time is input in step S14, and transmission schedule data is read in step S15.

  In step S12, the current time is read. In step S13, it is determined whether or not the current time has reached the automatic transmission time. When it is determined that the transmission time has come, the storage device 3 is connected (step S1). Subsequent communication processing is the same as the transmission processing described with reference to FIG. 12, and corresponding processing steps are denoted by the same reference numerals and detailed description thereof is omitted.

  FIG. 14 is a flowchart of still another example of a data transmission procedure from the signal acquisition unit 1 to the rental integrated database 6. The process of FIG. 14 automatically transmits an integrated signal to the rental integrated database 6 when the amount of integrated signal stored in the storage unit 3 reaches a predetermined amount. First, in step S21, the integrated signal accumulation amount is read. In step S22, it is determined whether or not the accumulated amount has reached a specified amount.

  If it is determined in step S22 that the accumulated amount has reached the specified amount, the communication terminal is activated (step S23). Then, communication is started (step S24) and connected to the rental integrated database 6 (step S2). The processing on the rental integrated database 6 side is the same as that shown in FIG. 12 or 13 as shown in S30. An ID and a password are transmitted from the activated communication terminal 4, and it is determined in step S24 whether or not preparation for transmission is completed.

  When preparation for transmission of the communication terminal 4 is completed and it is determined in step S25 that preparation for the storage device 3 has been completed, an integrated signal is read (step S4). And it transmits by the communication terminal 4 (step S5). If it is determined in step S26 that the transmission has been completed, the communication is disconnected (step S8).

  If it is determined in step S6 that the reading of the integrated signal has been completed, it is determined in step S7 whether or not the reading operation and the transmission operation have been completed. When it is determined that the process has been completed, the integrated signal is deleted (step S9).

  FIG. 15 shows an example of attribute data included in the integrated signal registered in each database of the rental integrated database 6. In the example of FIG. 15, personal ID 100a, event name 100b, time stamp 100c, location 100d, link destination 1 information 100e, and link destination 2 information 100f are set as attribute data. As attribute data, a spare area is prepared so that link destination information can be added and a new category can be defined later. The link destination information is information necessary for linking to another database in the public database 7.

  Next, a link method will be described based on a specific example of attribute data shown in FIG. In this example, the personal ID 100a is “Taro Patent”, the event name 100b is “Sydney Olympic Opening Ceremony”, the time stamp 100c is “10:00 am on September 15, 2000”, the location 100d is “Olympic Stadium”, The information 100e of the link destination 1 is “ABC broadcast material No. ???”, and the information 100f of the link destination 2 is “NNS broadcast material No. xxx”. This attribute data represents an integrated signal when the user “Taro Taro” experienced the opening ceremony of the Olympics. In addition, the opening-type broadcasting material is stored in, for example, the broadcasting station “ABC” or the broadcasting station “NNS” and is disclosed as public data.

  As described above, the company that manages the rental integrated database 6 searches the public database 7 for relevant information based on the attribute data registered in the database. Based on the search result, the link destination information is written in the attribute data. In the above-described example, the storage location of the data corresponding to the opening ceremony of the Olympics of the broadcasting stations “ABC” and “NNS” is extracted from the public database 7, and this storage location is information 100e indicating the link destination of the attribute data. Automatically write to 100f.

  Then, as described above, it is linked to the public database 7, data related to the integrated signal already accumulated in the database of the rental integrated database 6 is acquired from the public database 7, and the integrated signal registered in the database is rewritten. It is made like. As a result of such processing, the information stored in the database of the rental integrated database 6 adds the information acquired from the public database to the information acquired by the individual user in the acquisition unit 1. As a result, the data registered in the database has a richer content and higher quality. In the above-described example, in addition to the opening ceremony image and sound acquired by the user, the image and sound as a result of coverage by the broadcast station are registered in the database.

  FIG. 17 is a flowchart for explaining the search and automatic link operations performed by the rental integrated database 6. The rental integrated database management company runs an automatic link program according to a contract (or request) with a user. First, in step S41, specific data of the contractor is read from the database, and the personal ID is confirmed. Next, the attribute data is read (step S42), the link condition designated in advance by the user is determined (step S43), and the event name is extracted from the result (step S44). In this case, the link destination may be determined from other information such as a time stamp in the attribute data.

  In step S45, the public database 7 in the public network is searched using the extracted event name as a keyword. The search is started by the search engine (step S46), and the search results are listed (step S47). In step S48, a powerful link destination is extracted from the search result.

  Next, the procedure for making a payment to the link destination is started. In step S49, connection to the link destination is made. In step S50, a payment procedure for the link destination is performed. For example, payment is made by electronic settlement from a personal account registered in the rental integrated database management company to the link destination through the public network 5 (step S51). This payment is the first thing you need, such as an admission fee or deposit, regardless of the amount of data downloaded.

  When the payment is confirmed from the link destination, the link permission is issued (step S52). After confirming this permission, the link destination is disconnected (step S52), and the link destination information is written in the attribute data (step S54). In step S55, it is determined whether or not the processing for the link destination extracted in step S48 has been completed. When all the processing is completed, the program ends.

  When the automatic link destination search and the attribute data editing are completed as described above, the database included in the rental integrated database is automatically edited next. FIG. 18 is a flowchart showing the flow of processing for automatically editing the contents of the integrated database. The rental combined database management company runs an automatic editing program. First, in step S61, the encryption of the integrated signal data is decrypted with the personal key. In step S62, attribute data is read.

  Information on the link destination in the attribute data is extracted (step S63). Based on this information, the rental integrated database 6 is connected to the link destination of the public database 7, for example, the database of the broadcasting station (step S64). Further, a payment procedure is performed for the extracted link destination (step S65). Then, through the public network 5, payment is made by electronic settlement from a personal account registered in the rental integrated database management company to the link destination (step S66). In step S67, the linked data is downloaded. This payment is made according to the amount of data downloaded (pay-as-you-go billing). The end of the download is determined in step S68, and when the download is completed, the connection with the link destination is disconnected in step S69.

  The integrated signal corresponding to the attribute data read in step S62 is decoded, and the integrated signal is separated into image data, audio data, biological signal, and the like (step S70). In step S71, the correlation between the contents of the data downloaded from the link destination, for example, the image data and the decrypted personal image data is detected (step S71). Based on the relationship (correlation of contents) between the personal image data and the downloaded image data, the image and the sound are exchanged or interpolated (added) in step S72. When the automatic editing is completed, each data is collected again by the integrated encoding in step S73, and encrypted again with the personal key in step S74. In step S75, the database is rewritten with this encrypted integrated signal after editing, and the program ends.

  A configuration of an example of the integrated signal presentation device 8 connected to the rental integrated database 6 and installed in the re-experience center is shown in FIG. The presentation device 8 is installed in the re-experience center. When a request is made to re-experience the experience that the user has experienced in the past, the user can go to the re-experience center and re-experience.

  In the integrated signal presentation device 8, a large screen presentation screen 101 and a three-dimensional acoustic device 102 are arranged on the front surface in a special room in which air conditioning such as temperature and humidity can be controlled. In the illustrated example, one screen 101 is shown, but an immersive screen in which screens are arranged on the front and left and right side surfaces may be used. An image is presented on the screen 101 by a reflective or transmissive projector. The sound device 102 may be a device that emits sound through the screen 101 or a device that reproduces a sound field by embedding a large number of speakers in a wall.

  A moving chair 103 is installed in the center of the room. The chair 103 is configured to be able to move in a three-dimensional direction, and is driven by the output of the shaking creation unit 104. The motion creation unit 104 detects a motion vector from the image signal in the integrated signal, and generates a motion signal component based on the detected motion vector. The shaking creation unit 104 includes a storage device such as a hard disk for storing the shaking signal. The moving chair 103 can reproduce, for example, vibration received when walking, running, or riding a car. An example of a method for generating the shaking signal is described in, for example, Japanese Patent Application Laid-Open No. 2000-214755.

  Reference numeral 105 denotes air conditioning equipment that controls the temperature, humidity, and the like of the room. The air conditioning equipment 105 is controlled by the control unit 106. The control unit 106 includes a storage device such as a hard disk for storing control data. The screen 101 and the sound device 102 as the above-described image reproducing device are controlled by the signal processing unit 107. The signal processing unit 107 is provided with a storage device such as a hard disk for storing image data and sound data.

  In order to receive the integrated signal from the rental integrated database 6 and decrypt the received signal, a terminal 108, a personal authentication unit 109, an encryption decoder 110, and an integrated encoding decoding unit 111 are provided. Yes. The integrated signal from the decoding unit 111 is supplied to the signal processing unit 107, and the control signal for each device described above is generated by the signal processing unit 107 based on the integrated signal. The signal processing unit 107, the terminal 108, the personal authentication unit 109, the decoder 110, and the decoding unit 111 are actually composed mainly of a workstation and are installed at one corner of the room.

  FIG. 20 is a flowchart showing a flow of operations related to the integrated signal presentation device 8. When the user desires to re-experience, a visit is made to the re-experience center and predetermined procedures are performed at the window. The predetermined procedure is an application for re-experience using a user-specific card and payment by electronic settlement of the usage fee of the integrated signal presentation device 8. The user-specific card is made of, for example, an IC card, and the user-specific ID, electronic payment account information, and the like are encrypted and recorded using biometric data (such as a fingerprint) as a key. The user presents a card unique to the user at the window, and performs a reservation work for an event that the user wants to re-experience by performing personal authentication, and completes the electronic payment procedure (step S91).

  When it is time for reservation, the user enters the room of the integrated signal presentation device 8 and sits on the moving chair 103 to start the re-experience program. First, in step S92, the personal authentication information is read out, and in step S81, the terminal 108 that is a part of the integrated signal presentation device 8 is connected to the rental integrated database management company through the network.

  Personal authentication is performed in steps S82 and S93 based on the personal authentication information input in advance at the window and the name of the event to be re-experienced. In step S94, it is detected whether or not the authentication is established. If authentication is not established, the process ends at that point. When the authentication is established, the event name is transmitted to the rental integrated database 6 (step S95).

  The rental integrated database 6 starts reading and downloading the corresponding integrated signal data in step S83. In FIG. 20, a process S <b> 80 surrounded by a broken line indicates the process of the rental integrated database 6. In step S <b> 84, the integrated signal is transmitted to the integrated signal presentation device 8. In step S85, the end of transmission is monitored, and when the transmission ends, the connection is disconnected (step S86).

  The integrated signal presentation device 8 accumulates the received integrated signal (step S96). Then, the encryption of the integrated signal is decrypted based on the personal key read in step S98, for example, the biometric signal key such as fingerprint information (step S97). Next, in step S99, the integrated signal is decoded. Thereby, original image data, audio data, various biological signals, and the like are separated.

  Control data for controlling the air-conditioning equipment 105 is generated based on signals from the temperature sensor and the sweat sensor among the biological signals (step S100). In step S101, it is determined whether or not the air conditioning control data generation process has ended. The ambient temperature and humidity at the time of the experience are estimated based on the air conditioning data, and the operation programming including the time variation of the air conditioner is performed.

  A three-dimensional acoustic signal is generated based on the decoded audio signal (step S102). In step S103, it is determined whether or not the acoustic signal generation processing is completed. The voice of conversation, ambient sounds, etc. are reproduced by the audio signal.

  In step S104, a shaking signal is generated based on the decoded image signal. A motion vector of the image signal is detected, and a shaking signal is generated from the detected motion vector. In step S <b> 105, it is determined whether or not the motion signal generation process has been completed. Step S106: The decoded image signal is accumulated. In step S107, it is determined whether or not the accumulation of the image signal is completed.

  When the end is detected in steps S101, 103, 105, and 107, that is, the process waits until all data is obtained. When all the data is ready, the signal processing unit 107 sends an operation start trigger signal to the sway creation unit 104 and the control unit 106 (step S108). In step S109, presentation of each data is started in synchronization with each other by this trigger signal. As a result, a realistic reproduction of events experienced in the past is performed. In addition, a more realistic experience complemented with high-quality data such as broadcasting materials from public databases is also performed. In this way, when the reproduction of the re-experience series data is completed, the program ends.

  The present invention is not limited to the above-described embodiment of the present invention, and various modifications and applications can be made without departing from the gist of the present invention. For example, in the integrated signal presentation device 8, the generation processing of the air conditioning control data, the acoustic signal, the shaking signal, etc. and the storage processing of the image signal are performed in advance after the user makes a reservation, and the user re-experiences with a small waiting time It may be possible. Alternatively, the user may receive attribute data from the rental integrated database, and edit and process the attribute data by himself / herself.

DESCRIPTION OF SYMBOLS 1 ... Signal acquisition part, 2 ... Signal acquisition and encoding part, 3 ... Accumulation part, 6 ... Rental integrated database, 7 ... Public database, 8 ... Integrated signal presentation apparatus

Claims (4)

Audiovisual information signal acquisition means that is attached to the user and acquires an audiovisual information signal consisting of at least one of visual information and auditory information;
A biological information signal acquisition unit that is attached to the user and acquires a biological information signal corresponding to the biological information of the user;
Writes at least selectively storing unit the audiovisual information signal determined to be valid information by a change of the living body information signals, write the attribute data associated with the audiovisual information signal input by the input means uselessly control Control means to
A related information acquisition means for acquiring relevant information part searches the external database as a keyword of the attribute data from the external database,
An information processing system comprising: editing means for editing the stored audiovisual information signal by using related information correlated with the audiovisual information signal among the related information acquired by the related information acquisition means. .   The information processing system according to claim 1, wherein the biological information signal acquisition unit acquires a biological information signal corresponding to body temperature, myoelectric potential, sweating, heartbeat, or blood pressure.   2. The information processing system according to claim 1, wherein the control unit controls to selectively write the audiovisual information signal determined to be valid information by a change in a plurality of types of biological information signals in a storage unit. And further comprising encryption means for encrypting the audiovisual information signal and the biological information signal,
The information processing system according to claim 1, wherein the control unit controls to write the audiovisual information signal and the biological information signal encrypted by the encryption unit in a storage unit.

Images