EP0695109A1 - Vorrichtung zur wiedergabe von video- und audiosignalen - Google Patents

Vorrichtung zur wiedergabe von video- und audiosignalen Download PDF

Info

Publication number
EP0695109A1
EP0695109A1 EP95907878A EP95907878A EP0695109A1 EP 0695109 A1 EP0695109 A1 EP 0695109A1 EP 95907878 A EP95907878 A EP 95907878A EP 95907878 A EP95907878 A EP 95907878A EP 0695109 A1 EP0695109 A1 EP 0695109A1
Authority
EP
European Patent Office
Prior art keywords
listener
signal
video signal
head
audio signal
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
EP95907878A
Other languages
English (en)
French (fr)
Other versions
EP0695109A4 (de
EP0695109B1 (de
Inventor
Kiyofumi Inanaga
Yuji Yamada
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Publication of EP0695109A1 publication Critical patent/EP0695109A1/de
Publication of EP0695109A4 publication Critical patent/EP0695109A4/de
Application granted granted Critical
Publication of EP0695109B1 publication Critical patent/EP0695109B1/de
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/002—Non-adaptive circuits, e.g. manually adjustable or static, for enhancing the sound image or the spatial distribution
    • H04S3/004—For headphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/302—Electronic adaptation of stereophonic sound system to listener position or orientation
    • H04S7/303—Tracking of listener position or orientation
    • H04S7/304—For headphones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2400/00—Details of stereophonic systems covered by H04S but not provided for in its groups
    • H04S2400/01—Multi-channel, i.e. more than two input channels, sound reproduction with two speakers wherein the multi-channel information is substantially preserved
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S2420/00—Techniques used stereophonic systems covered by H04S but not provided for in its groups
    • H04S2420/01—Enhancing the perception of the sound image or of the spatial distribution using head related transfer functions [HRTF's] or equivalents thereof, e.g. interaural time difference [ITD] or interaural level difference [ILD]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S3/00—Systems employing more than two channels, e.g. quadraphonic
    • H04S3/008—Systems employing more than two channels, e.g. quadraphonic in which the audio signals are in digital form, i.e. employing more than two discrete digital channels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/305—Electronic adaptation of stereophonic audio signals to reverberation of the listening space
    • H04S7/306—For headphones

Definitions

  • the present invention relates to a video signal and audio signal reproducing apparatus for reproducing an audio signal through headphones while a listener is watching a picture, for example.
  • the system of reproducing the audio signal through the headphones includes a binaural sound-wave pickup and reproduction system.
  • the binaural sound-wave pickup and reproduction system is the following system. Microphones, so-called dummy-head microphones, are located in left and right auricles of a dummy head which is made to imitate the listener's head. An audio signal from a signal source is picked up by the dummy-head microphones. When the audio signal thus picked up is reproduced and the listener actually listens to the reproduced audio signal with the headphones, the listener can obtain presence with which the listener feels as if he listened to the sounds directly from the signal source.
  • the binaural sound-wave pickup and reproduction system it is possible to improve the picked-up and reproduced sound image in directivity, localization, presence and so on.
  • a signal source as a special source which is picked up by the dummy-head microphones as a sound source signal and different from that use for reproduction with speakers.
  • the relative direction and position of the sound image which the listener perceives are not changed. Therefore, even if the binaural reproduction is carried out by using the headphones, then when the listener changes the direction of the head (face), the sound image is formed inside the listener's head. It is particularly difficult to effect a so-called forward localization, i.e., to localize the sound image in front of the listener. Moreover, in this case, the sound image tends to be elevated above the head and hence becomes unnatural.
  • the following binaural reproduction system using headphones is supposed. Specifically, directivity and localization of a sound image are determined by difference in volume, time, phase and so on between sounds perceived by left and right ears of the listener.
  • the system disclosed in the above publication has a level control circuit and a variable delay circuit in an audio signal line of each of left and right channels and also has a means for detecting the direction of the listener's head.
  • the level control circuit and the variable delay circuit for the audio signal in each of the channels are controlled based on a signal representing the detected direction of the listener's head.
  • characteristics obtained when the differences in volume and time are changed must be determined based on relative positional relationship between a sound source and the listener, a shape of the listener's head, shapes of listener's auricles and so on. Specifically, if a certain characteristic is determined, then the relative positional relationship between the sound source and the listener is fixed so that it is impossible to change a sense of distance and a distance between the sound sources. Further, since listeners have different shapes of heads and auricles, an effect of the method differs depending upon the listeners.
  • the stereophonic reproduction system in the above Japanese patent publication No. 54-19242 requires a memory of a huge capacity for continuously calculating and storing the relationship of the change amounts of the differences in volume and time between the audio signals so that it is very difficult to realize the stereophonic reproduction system.
  • reproduction of the audio signal corresponding to the video signal does not disclose reproduction of the audio signal corresponding to the video signal.
  • an audio reproduction apparatus disclosed in Japanese laid-open patent publication No. 01-112900 filed by the same applicant as the applicant of the present invention, there is provided an apparatus for discretely, not continuously, calculating data of the relationship between the change amounts of the differences in volume and time between audio signals and processing the audio signals.
  • Japanese laid-open patent publication No. 01-112900 in which the audio reproduction apparatus is disclosed presents only an abstract concept of a principle that can be applied to both analog and digital signal processings and lacks a specific description required when the audio reproduction apparatus effects the analog or digital signal processing and is applied to actual products.
  • the means for correcting the characteristics inherent in sound sources used when transfer functions from a virtual sound source to the listener's ears is measured and the characteristics inherent in the headphones used by the listener.
  • the above publication does not disclose reproduction of the audio signal corresponding to the video signal.
  • the present invention is made and a first object thereof is to provide a video signal and audio signal reproducing apparatus which localizes a position of a sound image reproduced from an audio signal such that the sound image corresponds to a picture.
  • the present invention is made and a second object thereof to provide a video signal and audio signal reproducing apparatus which localizes a sound image reproduced from an audio signal such that the sound image corresponds to a virtual sound source.
  • an address of a storage means is designated by an address signal generated by an address signal generating means based on a signal supplied from an angle detecting means and corresponding to an angle and an output signal from a detecting means to thereby read out an impulse response or a control signal stored in the storage means.
  • An audio signal is corrected with respect to a relative movement of a listener and a head movement of the listener with respect to a virtual sound source on a video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from a signal source such that a plurality of reproduced sound images are localized in the direction corresponding to a picture reproduced by a video signal reproducing means. It is possible to reproduce the audio signal corrected by a control means such that the plurality of reproduced sound images are localized at a position corresponding to the picture reproduced by the video signal reproducing means.
  • the detecting means includes an extracting means for extracting positional information output from the signal source together with the video signal and the audio signal and an output signal from the extracting means is supplied to the control means. Therefore, the output signal from the extracting means for extracting the positional information is supplied to the control means.
  • the address of the storage means is designated by the address signal generated by the address signal generating means based on the positional information previously supplied from the signal source together with the video signal and the audio signal to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of a listener and the head movement of the listener with respect to the virtual sound source of the video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source based on the positional information supplied from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the detecting means includes a position detecting means mounted on the listener's head for detecting the relative movement of the listener with respect to the video signal reproducing means and a detection signal from the position detecting means is supplied to the control means. Therefore, the detection signal from the position detecting means for detecting the relative movement of the listener with respect to the video signal reproducing means is supplied to the control means.
  • the address of the storage means is designated by the address signal generated by the address signal generating means based on a signal indicative of a detected relative movement of the listener with respect to the video signal reproducing means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source of the video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source based on the signal indicative of detected relative movement of the listener with respect to the video signal reproducing means such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the position detecting means is provided in an audio signal reproducing means. Therefore, it is easy to detect the relative movement of the listener with respect to the video signal reproducing means.
  • the audio signal from the signal source is corrected based on the signal indicative of the detected relative movement of the listener with respect to the video signal reproducing means such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the position detecting means changes coordinates of an angle information from the angle detecting means based on at least information on a head gyration of the listener with respect to the reference direction and information on movement of the listener's head close to or away from the reference position. Therefore, the coordinates of the angle information from the angle detecting means are changed based on at least the information on the head gyration of the listener with respect to the reference direction and the information on the movement of the listener's head close to or away from the reference position.
  • the audio signal from the signal source is corrected such that a plurality of reproduced sound images are localized in the direction corresponding to the reproduced picture. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the position detecting means adds at least the information on the head gyration of the listener with respect to the reference direction and the information on the movement of the listener's head close to or away from the reference position to the angle information from the angle detecting means to thereby change the angle information from the angle detecting means. Therefore, the information on the head gyration of the listener with respect to the reference direction and the information on the movement of the listener's head close to or away from the reference position are added to the angle information from the angle detecting means to thereby change the coordinates of the angle information from the angle detecting means.
  • the audio signal from the signal source is corrected such that a plurality of reproduced sound images are localized in the direction corresponding to the reproduced picture. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the angle detecting means includes a reset switch and the angle detecting means sets as the reference direction the direction in which the listener turns the head when the reset switch is turned on. Therefore, the angle detecting means sets as the reference direction the direction in which the listener turns the head when the reset switch is turned on.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal supplied from the angle detecting means and corresponding to the angle and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source of the video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the angle detecting means sets the direction as the reference direction. Therefore, when the listener turns the head in the predetermined reference direction, the direction is set as the reference direction for the angle detecting means.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal supplied from the angle detecting means and corresponding to the angle and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source of the video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the audio signal reproducing means includes the reset switch and when the listener wears the audio signal reproducing means, the reset switch is operated and the angle detecting means sets as the reference direction the direction toward a front of a screen of the video signal reproducing means. Therefore, the audio signal reproducing means includes the reset switch and when the listener wears the audio signal reproducing means, the reset switch is operated and the angle detecting means sets as the reference direction the direction toward the front of the screen of the video signal reproducing means.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal supplied from the angle detecting means and corresponding to the angle and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source of the video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the apparatus further includes an input means for employing a signal based on a size of a display unit of the video signal reproducing means as data, a signal from the input means is supplied to the address signal generating means, and the address signal generating means generates an address signal in accordance with the signal corresponding to the angle from the angle detecting means, an output signal from the detecting means and data input from the input means.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal, which corresponds to the signal corresponding to the angle from the angle detecting means, the output signal from the detecting means and the data input from the input means and which is generated by the address signal generating means based on the signal corresponding to the angle and supplied from the angle detecting means and the output signal from the detecting means, to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source of the video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the angle detecting means also detects angles of upward and downward gyrations of the listener with respect to the reference direction and the audio signal reproducing means reproduces the audio signal corrected by the control means based on the impulse response or the control signal read out from the storage means in accordance with the angles in the upward and downward directions detected by the angle detecting means, thereby a plurality of optional reproduced sound images being localized in the direction corresponding to the picture reproduced by the video signal reproducing means.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal supplied from the angle detecting means and corresponding to the angle of the upward and downward gyration of the listener with respect to the reference direction and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source of the video signal reproducing means in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the picture reproduced by the video signal reproducing means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the video signal reproducing means can be mounted on the listener's head, is provided so as to be opposed to both eyes of the listener, and projects the reproduced picture on the position distant from the both eyes of the listener by a predetermined distance. Therefore, the address of the storage means is designated by the address signal generated by the address signal generating means based on the signal corresponding to the angle from the angle detecting means. The impulse response or the control signal stored in the storage means is read out therefrom. The audio signal is corrected by the control means based on the impulse response or the control signal to thereby correct the audio signal with respect to the head movement of the listener in a real-time fashion.
  • the audio signal reproducing means can reproduce the audio signal corrected by the control means such that a plurality of reproduced sound images are localized in the direction corresponding to the reproduced picture projected on the position distant from left and right eyes of the listener by a predetermined distance when the video signal reproducing means reproduces the video signal.
  • the video signal reproducing means includes a head mount body to be mounted on the listener's head and a pair of display units disposed at positions of the head mount body respectively corresponding to listener's both eyes. Therefore, since the video signal reproducing means has a pair of a left display unit and a right display unit disposed at the positions respectively corresponding to the left and right eyes of the listener, it is possible for the left display unit and the right display unit to project the reproduced pictures on the positions distant from the left and right eyes of the listener by a predetermined distance.
  • the video signal reproducing means further includes a pair of rectangular aspheric lenses respectively disposed between the listener's both eyes and the pair of display units. Therefore, since the video signal reproducing means has a pair of a left display unit and a right display unit disposed at the positions corresponding to the left and right eyes of the listener through the left and right rectangular aspheric eyepieces, it is possible to magnify the images projected on the left display unit and the right display unit and to project the reproduced pictures on the positions distant from the left and right eyes of the listener by a predetermined distance in front of the left display unit and the right display unit.
  • the video signal reproducing means includes the head mount body to be mounted on the listener's head and a pair of virtual image display units disposed at the positions of the head mount body respectively corresponding to the listener's both left and right eyes. Therefore, since the video signal reproducing means has a pair of the left virtual image display unit and the right virtual image display unit disposed at the positions of the head mount body respectively corresponding to the listener's eyes, it is possible for the left virtual image display unit and the right virtual image display unit to project the reproduced pictures on the positions distant from the left and right eyes of the listener by a predetermined distance.
  • the address of the storage means is designated by the address signal generated by an address signal generating means based on the signal corresponding to the angle from the angle detecting means and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source position in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction to the virtual sound source. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized in the direction to the virtual sound source.
  • the detecting means includes the extracting means for extracting positional information output from the signal source together with the audio signal and the output signal from the extracting means is supplied to the control means. Therefore, the output signal from the extracting means for extracting the positional information is supplied to the control means.
  • the address of the storage means is designated by the address signal generated by the address signal generating means based on the positional information previously supplied from the signal source together with the video signal and the audio signal to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source position in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source based on the positional information supplied from the signal source such that a plurality of reproduced sound images are localized in the direction to the virtual sound source. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized in the direction to the virtual sound source.
  • the detecting means includes the position detecting means mounted on the listener's head for detecting the relative movement of the listener with respect to the virtual sound source position and the detection signal from the position detecting means is supplied to the control means. Therefore, the detection signal from the position detecting means for detecting the relative movement of the listener with respect to the virtual sound source position is supplied to the control means.
  • the address of the storage means is designated by the address signal generated by the address signal generating means based on the signal indicative of the detected relative movement of the listener with respect to the virtual sound source position to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source position in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source based on the signal indicative of the detected relative movement of the listener with respect to the virtual sound source position such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the position detecting means is provided in the audio signal reproducing means. Therefore, it is easy to detect the relative movement of the listener with respect to the virtual sound source position.
  • the audio signal from the signal source is corrected based on the signal indicative of the detected relative movement of the listener with respect to the virtual sound source position such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the position detecting means changes the coordinates of the angle information from the angle detecting means based on at least the information on the head gyration of the listener with respect to the reference direction and the information on the movement of the listener's head close to or away from the reference position. Therefore, the coordinates of the angle information from the angle detecting means are changed based on at least the information on the head gyration of the listener with respect to the reference direction and the information on the movement of the listener's head close to or away from the reference position.
  • the audio signal from the signal source is corrected such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the position detecting means adds at least the information on the head gyration of the listener with respect to the reference direction and the information on the movement of the listener's head close to or away from the reference position to the angle information from the angle detecting means to thereby change the angle information from the angle detecting means. Therefore, the information on the head gyration of the listener with respect to the reference direction and the information on the movement of the listener's head close to or away from the reference position are added to the angle information from the angle detecting means to thereby change the coordinates of the angle information from the angle detecting means.
  • the audio signal from the signal source is corrected such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the angle detecting means includes a reset switch and the angle detecting means sets as the reference direction the direction in which the listener turns the head when the reset switch is turned on. Therefore, the angle detecting means sets as the reference direction the direction in which the listener turns the head when the reset switch is turned on.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal corresponding to the angle from the angle detecting means and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source position in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the angle detecting means sets that direction as the reference direction. Therefore, when the listener turns the head in the predetermined reference direction, the direction is set as the reference direction by the angle detecting means.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal corresponding to the angle from the angle detecting means and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source position in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the audio signal reproducing means includes the reset switch and when the listener wears the audio signal reproducing means, the reset switch is operated and the angle detecting means sets as the reference direction the direction toward a front of the virtual sound source position. Therefore, the audio signal reproducing means includes the reset switch and when the listener wears the audio signal reproducing means, the reset switch is operated and the angle detecting means sets as the reference direction the direction toward the front of the virtual sound source position.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal corresponding to the angle from the angle detecting means and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source position in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the angle detecting means both detects angles of the upward and downward gyrations of the listener with respect to the reference direction and the audio signal reproducing means reproduces the audio signal corrected by the control means based on the impulse response or the control signal read out from the storage means in accordance with the angles in the upward and downward directions detected by the angle detecting means, thereby a plurality of optional reproduced sound images being localized in the direction corresponding to the virtual sound source position.
  • the address of the storage means is designated, in accordance with the head movement of the listener with respect to the reference direction, by the address signal generated by the address signal generating means based on the signal detecting means corresponding to the angle of the upward and downward gyration of the listener with respect to the reference direction from the angle and the output signal from the detecting means to thereby read out the impulse response or the control signal stored in the storage means.
  • the audio signal is corrected with respect to the relative movement of the listener and the head movement of the listener with respect to the virtual sound source position in a real-time fashion based on the impulse response or the control signal read out from the storage means to thereby correct the audio signal from the signal source such that a plurality of reproduced sound images are localized in the direction corresponding to the virtual sound source position. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the virtual sound source position.
  • the audio signal from the signal source is corrected by selectively reading out correction data from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means.
  • the correction data read out from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means are changed based on the detection signal from the detecting means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the storage means after the impulse response from the virtual sound source position with respect to the reference direction of the listener's head to the both ears corresponding to the head movement of the listener is measured, stores the impulse response, or, after the differences in time and level between the audio signals from the virtual sound source position with respect to the reference direction of the listener's head to the both ears are measured, stores the control signal representing the differences in time and level between the audio signals from the signal source based on measured results. Therefore, the audio signal from the signal source is corrected by selectively reading out the control signal from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means.
  • the control signal read out from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means is changed based on a detection signal from the detecting means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the apparatus further includes the address signal generating means for generating the address signal supplied to the storage means to read out the impulse response or the control signal stored in the storage means. Therefore, the address signal is supplied from the address signal generating means to the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means.
  • the audio signal from the signal source is corrected by selectively reading out the impulse response or the control signal from the storage means.
  • the impulse response or the control signal read out from the storage means based on the information on the angle of the listener supplied from the angle detecting means is changed based on a detection signal from the detecting means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the detecting means includes the extracting means for extracting the positional information output from the signal source together with the video signal and the audio signals and the output signal from the extracting means is supplied to the control means. Therefore, the audio signal from the signal source is corrected by selectively reading out the correction data from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means.
  • the correction data read out from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means are changed based on an output signal from the extracting means for extracting the positional information output together with the video signal and the audio signal from the signal source. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the detecting means includes the position detecting means mounted on the listener's head for detecting the relative movement of the listener with respect to the video signal reproducing means and the detection signal from the position detecting means is supplied to the control means. Therefore, the audio signal from the signal source is corrected by selectively reading out the correction data from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means.
  • the correction data read out from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means are changed based on the detection signal from the position detecting means mounted on the listener's head for detecting the relative movement of the listener with respect to the video signal reproducing means through the detecting means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • the position detecting means is provided in the audio signal reproducing means. Therefore, the relative movement of the listener with respect to the video signal reproducing means is detected with ease.
  • the audio signal from the signal source is corrected by selectively reading out the correction data from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means.
  • the correction data read out from the storage means based on the information on the angle of the listener's gyration supplied from the angle detecting means are changed based on the detection signal from the position detecting means mounted on the listener's head for detecting the relative movement of the listener with respect to the video signal reproducing means through the detecting means. It is possible to reproduce the audio signal corrected by the control means such that the plurality of reproduced sound images are localized at the position corresponding to the picture reproduced by the video signal reproducing means.
  • FIGS. 1 through 11 A video signal and audio signal reproducing apparatus according to an embodiment of the present invention will hereinafter be described in detail with reference to FIGS. 1 through 11.
  • the video signal and audio signal reproducing apparatus of the embodiment of the present invention when a video signal and an audio signal having reproduced sound image position information are reproduced through a headphone while a listener watches a picture, the listener can perceive the equivalent localization, sound field and so on to those perceived when the audio signals are reproduced by speakers located in a predetermined positional relationship in which the speakers should be located when the audio signals are reproduced by the speakers.
  • a plurality of sound images are located in the direction corresponding to the picture on the basis of the reproduced sound image position information.
  • coordinates of the reproduced sound image position information are changed.
  • the video signal and audio signal reproducing apparatus is used in a system of reproducing multichannel video and audio signals, which are obtained by picking up a sound in a stereophonic mode or the like and have the reproduced sound image position information indicating both or either of a microphone position and a sound source position obtained when the sound is picked up, through the headphone while a listener watches the picture.
  • the reproduced sound images in a plurality of channels are localized at the position corresponding to the picture based on the reproduced sound image position information indicating both or either of the microphone position and the sound source position obtained when the sound is picked up.
  • the ordinates of the reproduced sound image position information are changed.
  • FIG. 1 shows an example of the video signal and audio signal reproducing apparatus according to the present invention.
  • the video and audio signals are input from an input terminal 60 to a separating circuit 61.
  • the separating circuit separates the signal into the video signal and the audio signal.
  • the video signal is supplied therefrom to a video signal reproducing apparatus 62.
  • Reference numeral 1 depicts a multichannel digital stereophonic signal source including a picture, such as a digital audio disc (e.g., an optical video disc), a digital satellite broadcasting or the like.
  • Reference numeral 2 depicts an analog stereophonic signal source, such as an analog record, an analog broadcasting or the like. If the audio signal supplied from the separating circuit 61 is a digital audio signal, then the audio signal is supplied to the digital stereophonic signal source 1. If the audio signal supplied from the separating circuit is an analog audio signal, then it is supplied to the analog stereophonic signal source 2. The digital stereophonic signal source 1 and the analog stereophonic signal source 2 separate the supplied audio signals into two-channel digital and analog audio signals L, R, respectively. Alternatively, the digital stereophonic signal source and the analog stereophonic signal source generate the digital and analog audio signals in four to seven channels, respectively.
  • Each of the digital and analog audio signals respectively supplied from the digital stereophonic signal source 1 and the analog stereophonic signal source 2 is the audio signal separated from the video signal having the information on the reproduced sound image position.
  • the video signal has the information on the reproduced sound image position indicating both or either of the microphone position and the sound source position when the sound is picked up.
  • Reference numeral 3 depicts A/D converters which convert analog signals into digital signals.
  • a means for generating the audio signal including the information on the reproduced sound image position indicating both or either of the microphone position and the sound source position obtained when the sound is picked up is a "picking-up and recording system and reproducing system" disclosed in a patent application filed on November 26, 1991 by the same applicant and inventors as those of this application.
  • the means includes a microphone 104 for picking up a sound from a first sound source 100 and a positional information detecting apparatus 103 for indicating both or either of a position of the microphone 104 and a sound source position of the first sound source 100.
  • the means includes a microphone 106 for picking up a sound from a second sound source 101 and a positional information detecting apparatus 105 for indicating both or either of a position of the microphone 106 and a sound source position of the second sound source 101.
  • Audio signals from the microphones and corresponding position signals are multiplexed by multiplexers 109, 110, respectively. Each of the multiplexers multiplexes and records the corresponding audio signal and the position signal on the same channel.
  • a frequency-division multiplex, a time-division multiplex or other multiplex system may be employed.
  • microphones 107, 108 for picking up a sound from another sound source 102 are provided.
  • Outputs from the multiplexers 109, 110 and outputs from the microphones 107, 108 are supplied to a picked-up and recorded signal processing and storing apparatus 112.
  • the picked-up and recorded signal processing and storing apparatus 112 records the audio signal and the position signal in the same audio channel with both of the signals being multiplexed.
  • a camera 111 picks up an image of a scene including the sound sources 100, 101 and 102.
  • a video signal from the camera is also recorded by the picked-up and recorded signal processing and storing apparatus 112.
  • the respective signals stored in the picked-up and recorded signal processing and storing apparatus 112 are supplied through an editing apparatus 113 to an audio and video reproduction apparatus 114 and reproduced by the latter.
  • the position signal is multiplexed together with the corresponding audio signal and both of the signals are recorded in the same channel, it is possible to carry out reproduction corresponding to the position by a reproducing apparatus when the audio signal is reproduced.
  • a real sound image is produced at a sound source position in a reproduced picture.
  • the audio and video reproduction apparatus 114 shown in FIG. 10 corresponds to the video signal and audio signal reproducing apparatus shown in FIG. 1.
  • microphones are independently provided for respective, for example, N sound sources whose sounds are to be picked up, and required positional informations on M microphones (where N > M) of the microphones are output into a channel independent of the channels for the audio signal at the same time.
  • the positional information in this case may be an absolute position information or information on a relative position relative to a predetermined reference position in a sound field.
  • the positional information may be not only an orthogonal coordinates system but also a polar coordinates system.
  • the A/D converters 3 which are as much as the number of the channels of the analog signals are provided.
  • Reference numeral 4 depicts switchers in which both signals inputted as digital signals and signals inputted as analog signals are equally processed as digital signals represented by a constant sampling frequency and a constant number of quantizing bits. While the switchers for two channels are shown here, if the signals are multichannel signals, then the switchers which are as much as the number of channels are provided.
  • a left digital signal L of the digital signal series is supplied to a convolution integrator 5.
  • a set of digitally recorded impulse responses are stored in a memory 6 associated with the convolution integrator 5, the digitally recorded impulse responses being impulse responses from a virtual sound source position in the direction in which a listener 23 turns the head at present with respect to a reference direction of the head to the both ears of the listener and being represented by a constant sampling frequency and a constant number of quantizing bits.
  • the digital signal series are subjected to convolution integral together with the impulse response read out from the memory 6 by the convolution integrator 5 in a real-time fashion.
  • a convolution integrator 7 and a memory 8 supply a crosstalk component of a right digital signal R.
  • the right digital signal R is supplied to a convolution integrator 11.
  • a set of digitally recorded impulse responses are stored in a memory 12 associated with the convolution integrator 11, the digitally recorded impulse responses being impulse responses from the virtual sound source position in the direction in which the listener 23 turns the head at present with respect to the reference direction of the head to both the ears of the listener and being represented by the constant sampling frequency and the constant number of quantizing bits.
  • the digital signal series are subjected to convolution integral together with the impulse response read out from the memory 12 by the convolution integrator 11 in a real-time fashion.
  • a convolution integrator 9 and a memory 10 supply a crosstalk component of a right digital signal L.
  • the convolution integrator 7 and the memory 8 and the convolution integrator 11 and the memory 12 subject the digital signals to the convolution integral together with the impulse responses.
  • the digital signal series subjected by the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12 to the convolution integral together with the impulse responses are supplied to adders 15, 16, respectively.
  • Two-channel digital signals added by the adders 15, 16 are corrected by correcting circuits 17, 18 to remove therefrom characteristics inherent in sound sources and a headphone which are used, and then converted by D/A converters 19, 20 into two-channel analog signals.
  • the two-channel analog signals are amplified by power amplifiers 21, 22 and then supplied to a headphone 24.
  • FIG. 7 shows only a processing of the audio signal.
  • a processing of the video signal in the arrangement is the same as that shown in FIG. 1 and hence will not be described.
  • a pair of digitally recorded impulse responses from the virtual sound source positions in the head direction fixed with respect to the reference direction to the listener's both ears are stored in the memories 6, 8, 10 and 12 associated with the convolution integrators 5, 7, 9 and 11.
  • the digital signal series are subjected to the convolution integral together with the impulse responses in a real-time fashion.
  • the memory 35 stores a control signal representing differences in time and level between sounds obtained at the both ears from the virtual sound source positions with respect to the reference direction of the head to the both ears.
  • a newly detected head movement with respect to the reference direction is converted into a digital address signal representing a magnitude of the head movement including its direction at every constant unit angle or every predetermined angle.
  • the control signal previously stored in the memory 35 is read out therefrom based on the digital address signal.
  • the digital signals in the respective channels subjected to the convolution integral are corrected and changed by control apparatus 50, 51, 52 and 53 in a real-time fashion and results thereof are supplied to the adders 15, 16.
  • FIG. 8 An arrangement shown in FIG. 8 may be employed. Specifically, the digital signal series subjected to the convolution integral together with the impulse responses in a real-time fashion are supplied to the address 15, 16. A newly detected head movement with respect to the reference direction is converted into a digital address signal representing a magnitude of the head movement including its direction at every constant unit angle or every predetermined angle. The control signal previously stored in the memory 35 is read out therefrom based on the digital address signal. The two-channel digital signals supplied from the adders 15, 16 are corrected and changed by control apparatus 54, 56 in a real-time fashion.
  • FIG. 8 shows only the processing of the audio signal. The processing of the video signal in this arrangement is the same as that shown in FIG. 1 and hence will not be described.
  • each of the control apparatus 50, 51, 52, 53, 54 and 56 may be formed by combining a variable delay apparatus and a variable level controller or a level controller for controlling a level in every frequency band, such as a graphic equalizer having a number of divided bands or the like.
  • Information stored in the memory 35 may be impulse response representing difference in time, level and so on between sounds obtained at the both ears from the virtual sound source positions in the direction in which the listener 23 turns the head with respect to the reference direction of the head to the both ears.
  • each of the above-mentioned control apparatus may be formed of an IIR or FIR variable digital filter.
  • the above-mentioned control apparatus give spatial information to the digital signals.
  • the digital signals are corrected by the correcting circuits 17, 18 with respect to the characteristics inherent in the sound sources and the headphone which are used, changed by the correcting circuits 17, 18 in response to the head movement, and then converted by the D/A converters 19, 20 into the analog signals.
  • the analog signals are amplified by the power amplifiers 21, 22 and then supplied to the headphone 24.
  • the correcting circuits 17, 18 for correcting the characteristics inherent in the sound sources and the headphone to be used may process signals in an analog or digital fashion. If the headphones is of wireless type, then the correcting circuits 17, 18 may be provided in a main body of the headphone.
  • the correcting circuits 17, 18 may not necessarily be housed in the main body of the headphone, but may be provided in cords of the headphone, for example, or may be provided in connector units for connecting the apparatus main body and the headphone or a subsequent stage.
  • the correcting circuits 17, 18 may be provided in the control apparatus of the apparatus main body or a subsequent stage.
  • a digital angle detector 28 detects a head movement of the listener 23.
  • FIG. 2 shows a detailed arrangement of the digital angle detector 28.
  • Fig. 2 shows the digital angle detector 28 using horizontal component forces of geomagnetism.
  • a signal indicative of a detected angle is output in the form of a digital signal.
  • a rotary encoder 30 is provided at a center position of the head with an input shaft thereof being vertical and a magnetic needle 29 is provided at the input shaft thereof. Accordingly, the rotary encoder 30 outputs a signal indicative of the head movement of the listener 23 including the direction with reference to the north and south direction indicated by the magnetic needle 29. While the rotary encoder is attached to a head band 27 of the headphone 24, the rotary encoder 30 may be attached to an attachment device provided independently of the head band 27.
  • the output from the rotary encoder 30 of the digital angle detector 28 is supplied to detecting circuits 31, 32.
  • the detecting circuit 31 outputs a direction signal Sd and is set to "0" or "1" when the listener 23 turns the head in the clockwise direction or in the counterclockwise direction.
  • the detecting circuit 32 outputs pulses Pa of the number proportional to an angle changed when the listener 23 changes the head direction, i.e., one pulse Pa each time when the angle of the head of the listener is changed by each 2°.
  • the signal Sd is supplied to a count direction input terminal U/D of an up/down counter 33, and the pulse Pa is supplied to a clock input (count input) terminal CK of the up/down counter 33.
  • a count output of the up/down counter is converted to a digital address signal indicating the direction and magnitude of the head movement of listener 23.
  • the digital address signal is supplied through an address control circuit 34 to the memories 6, 8, 10 and 12 as an address signal.
  • the impulse responses which are previously digitally recorded in the memories 6, 8, 10 and 12, from the virtual sound source positions with respect to the reference direction of the head of the listener 23 to the both ears of the listener 23 are read from corresponding addresses of the tables of the memories 6, 8, 10 and 12.
  • the impulse responses are subjected by the convolutional integrators 5, 7, 9 and 11 to convolution integral together with the digitized audio signals in the respective channels.
  • the digitized audio signals are corrected in a real-time fashion with respect to the direction in which the listener 23 turns the head at present.
  • reference numeral 38 depicts an analog angle detector.
  • FIG. 3 shows an arrangement of the analog angle detector in detail. In the arrangement shown in Fig. 3, an output signal indicative of a detected angle is output as an analog signal.
  • a photosensor 41 composed of a photosensor element, such as a CDS, a photodiode or the like whose resistance value changes in response to light intensity is disposed on the center portion of the head of the listener 23.
  • a light emitter 39 such as a bulb, a light emitting diode or the like, is disposed so as to be opposed to the photosensor 41. The light emitter 39 radiates light of predetermined intensity on the photosensor 41.
  • a movable shutter 40 In the light path of light radiated from the light emitter 39, there is provided a movable shutter 40 whose transmittance for the radiated light is changed depending upon a rotational angle thereof.
  • the movable shutter 40 is rotated together with the magnetic needle 29. Accordingly, when a constant current flows into the photosensor 41, a voltage across both ends of the photosensor of the photosensor 41 is derived as an analog output representing the head movement of the listener 23 including its direction with reference to the north and south direction indicated by the magnetic needle 29.
  • the analog angle detector 38 While the analog angle detector 38 is attached to the head band 27 of the headphone 24 in this embodiment, the analog angle detector may be attached onto an attachment device provided independently of the head band 27.
  • the analog output from the analog angle detector 38 is amplified by an amplifier 42 and then added to an A/D converter 43.
  • a digital output therefrom is supplied through a switcher 44 to the address control circuit 34.
  • the address control circuit 34 generates a digital address signal representing a magnitude of the head movement of the listener 23 with respect to the reference direction including the direction thereof at every constant angle or every predetermined angle.
  • the impulse responses which are previously digitally recorded in the memories 6, 8, 10 and 12, from the virtual sound source positions with respect to the reference direction of the head of the listener 23 to the both ears of the listener 23 are read from corresponding addresses of the tables of the memories 6, 8, 10 and 12.
  • the impulse responses are subjected to convolution integral together with the digitized audio signals in respective channels by the convolution integrators 5, 7, 9 and 11.
  • the digital signals are corrected in a real-time fashion with respect to the direction in which the listener 23 turns the head at present.
  • control signals which are previously digitally recorded in the memory 35, representing differences in time, level and so on between sounds obtained at the ears from the virtual sound source positions with respect to the reference direction of the head of the listener 23 to the both ears of the listener 23 are read from corresponding addresses of the table of the memory 35.
  • the digitized audio signals in respective channels subjected to convolution integral together with the impulse responses by the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12 associated respectively therewith are corrected by the control apparatus 50, 51, 52 and 53 in a real-time fashion with respect to the direction in which the listener 23 turns the head at present.
  • the two-channel digital signals added by the adders 15, 16 are corrected by the control apparatus 54, 56 in a real-time fashion with respect to the direction in which the listener 23 turns the head at present.
  • the control signal is the same as that used in the arrangement shown in FIG. 7.
  • FIG. 4 shows a table data stored in the memories 6, 8, 10 and 12.
  • reference symbol h mn (t) depicts impulse response from a speaker position m to an ear n
  • reference symbol H mn ( ⁇ ) depicts transfer function from the speaker position m to the ear n
  • reference symbol ⁇ depicts an angular frequency of 2 ⁇ f
  • reference symbol f depicts a frequency.
  • FIG. 6 shows an example of control data of the control signals stored in the table in the memory 35.
  • Each of the control apparatus 50 through 54 and 56 may be formed by combining the variable delay apparatus and the variable level controller or the level controller for controlling the level in every frequency band, such as the graphic equalizer having a number of divided bands or the like.
  • Information stored in the memory 35 may be impulse response representing differences in time, level and so on between sounds obtained at the both ears from the virtual sound source positions in the direction in which the listener 23 turns the head with respect to the reference direction of the head to both the ears.
  • Contents stored in the memory 35 have data structure corresponding to the control apparatus 50 through 54 and 56.
  • each of the above-mentioned control apparatus 50 through 54 and 56 may be formed of an IIR or FIR variable digital filter.
  • the speakers may be used as the sound sources used for measuring the control signals representing the difference in time between the sounds obtained at the respective ears and the difference in level therebetween. Positions where sound waves are picked up in the respective ears of the listener 23 may be anywhere from the inlets of the external auditory canals thereof to the ear drums thereof.
  • the positions should be equal to positions used to calculate characteristics of correction for canceling the characteristics inherent in the headphone to be used.
  • each of the digitally recorded impulse responses obtained when an angle ⁇ is changed by a unit angle, e.g., 2° is written in an address of the table of the memory 35.
  • the unit angle is set to be every angle through which the listener 23 can perceive with the left and right ears that he turns the head.
  • a position detector 63 for detecting a relative position such as a distance between a screen of the video signal reproducing apparatus 62 and the listener 23, an angle made by the listener and the screen or the like, is provided on the head band 27 of the headphone 24.
  • the position detector 63 may be three three-dimensionally disposed detection coils for detecting a leakage magnetic flux from the screen of the video signal reproducing apparatus 62, a velocity or acceleration pickup or the like.
  • a method of employing an output of a gyroscope for detecting a three-dimensional position with reference to a reference position a method of employing an electric wave from a satellite (GPS) or the like and a method of measuring distances from a plurality of ultrasonic wave sources may be employed.
  • GPS satellite
  • the position detector 63 supplies a signal indicative of a detected position to a detecting circuit 64. Based on the signal indicative of the detected position, the detecting circuit 64 supplies a control signal used to switch an address of the address control circuit 34 to a switcher 36. A switch signal based on a screen size of the screen of the video signal reproducing apparatus 62 is supplied through an input terminal 65 to the switcher 36.
  • the memory 35 includes three sets of such tables, each of sets having different data values depending upon a relative positional relationship such as a distance between the screen of the video signal reproducing apparatus 62 and the listener 23, an angle made thereby or the like, the screen size of the screen of the video signal reproducing apparatus 62 and so on.
  • An optimum set of the three sets of tables is selected by switching an address of the address control circuit 34 by switching the switcher 36 thereof.
  • reference numeral 37 depicts a center reset switch.
  • values of the up/down counter 33 are reset to "all 0".
  • the center reset switch 37 is turned on, the direction in which the listener 23 turns the head at present is set to be the front direction toward the sound sources.
  • the video signal and audio signal reproducing apparatus are arranged as described above and operates as follows. Specifically, the video signal and the audio signal are input from the input terminal 60.
  • the separating circuit 61 separates the input signals into the video signal and the audio signal.
  • the video signal is supplied therefrom to the video signal reproducing apparatus 62. If the audio signal separated by the separating circuit 61 is the digital audio signal, then it is supplied to the digital stereophonic signal source 1, and if it is the analog audio signal, then it is supplied to the analog signal source 2.
  • the digital audio signals from the multichannel digital stereophonic signal source 1 or the audio signals in respective channels obtained by converting the analog audio signals input to the multichannel analog stereophonic signal source 2 into the digital signals by the A/D converters 3 are selected by the switchers 4.
  • both the digital audio signals and the analog audio signals are the audio signals separated from the video signals, respectively.
  • the audio signals have the information on the reproduced sound image position indicating both or either of the microphone position and the sound source position obtained when the sound is picked up.
  • the digital signal series, together with the impulse responses read out from the memories 6, 8, 10 and 12 are subjected to convolution integral by the convolution integrators 5, 7, 9 and 11 in a real-time fashion, and then supplied to the adders 15, 16.
  • FIGS. 7 and 8 the same processings are carried out.
  • the digitized audio signals in respective channels previously subjected to convolution integral with the impulse responses by the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12 are corrected and changed by the control apparatus 50, 51, 52 and 53 based on the control signals read from the memory 35, and supplied to the adders 15, 16.
  • the two-channel digital signals from the adders 15, 16 are corrected and changed by the control apparatus 54, 56 based on the control signals read from the memory 35.
  • the two-channel digital signals are converted by the D/A converters 19, 20 into the analog signals which are amplified by the power amplifiers 21, 22 and then supplied to the headphones 24.
  • the listener 23 wearing the headphone 24 can listen to sounds reproduced from the audio signals.
  • the movement of the head of the listener 23 with respect to the reference direction at every constant or predetermined angle is detected by the digital angle detector 28 and the analog angle detector 38 and converted by the address control circuit 34 into the digital address signal representing the magnitude of the movement including its direction.
  • the digitally recorded impulse responses or control signals from the virtual sound source positions with respect to the reference direction of the head to both the ears are read out from the memory 35 in response to the digital address signal.
  • the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12 or the control apparatus 50, 51, 52, 53, 54 and 56 correct and change the audio signals based on the impulse responses or the control signals in a real-time fashion.
  • the signals are converted by the convolution integrators 5, 7, 9 and 11, the memories 6, 8, 10 and 12 or the control apparatus 50, 51, 52, 53, 54 and 56 and the address 15, 16 into the two-channel digital signals which have spatial information representing the sound field and are supplied to both the ears.
  • the two-channel digital signals are corrected by the correcting circuits 17, 18 with respect to the characteristics of the headphones and sound sources that are used.
  • the two-channel digital signals are amplified by the power amplifiers 21, 22 and supplied to the headphone 24.
  • the position detector 63 for detecting the relative relationship such as the distance between the screen of the video signal reproducing apparatus 62 and the listener 23, the angle made thereby or the like supplies the signal indicative of the detected position to the detecting circuit 64.
  • the detecting circuit 64 Based on the signal indicative of the detected position from the position detector 63, the detecting circuit 64 supplies the control signal used to switch the address of the address control circuit 34 to the switcher 36.
  • the switch signal based on the screen size of the screen of the video signal reproducing apparatus 62 is supplied through the input terminal 65 to the switcher 36.
  • the memory 35 includes three sets of the above tables, each of sets having the different data value depending upon the relative positional relationship such as the distance between the screen of the video signal reproducing apparatus 62 and the listener 23, the angle made thereby, the screen size of the screen of the video signal reproducing apparatus 62 and so on.
  • the optimum set of the three sets of tables is selected by switching the address of the address control circuit 34 by switching the switcher 36 thereof.
  • the address of the address control circuit 34 is switched in accordance with the relative positional relationship such as the distance between the screen of the video signal reproducing apparatus 62 and the listener 23, the angle made thereby or the like, and the optimum table is selected. It is possible that the position of the reproduced sound image corresponds to both or either of the positions, on the screen of the video signal reproducing apparatus 62, for example, of the microphone and the sound source used when the sound is picked up.
  • the address of the address control circuit 34 is switched in accordance with the screen size of the screen of the video signal reproducing apparatus 62, and the optimum table is selected. It is possible that, even when the screen size of the video signal reproducing apparatus 62, for example, is switched, the position of the reproduced sound image corresponds to both or either of the positions, on the screen having the switched screen size, for example, of the microphone and the sound source used when the sound is picked up.
  • the switcher 36 of the address control circuit 34 is switched based on the positional information from the position detector 63 and a relative address used to read out the table of the memory is changed to thereby move a position of a sound field
  • a plurality of tables are previously prepared in the memory, the impulse response or the control signal is read out therefrom when the positional information is changed by the positional information from the position detector 63, and by changing the audio signals in time and level, the listener 23 perceives as if the position of the sound source was changed.
  • FIGS. 1, 7 and 8 show only arrangements used when the single listener 23 listens to the reproduced sounds, when the plurality of listeners 23 listen to the reproduced sound, stages succeeding the convolution integrators 5, 7, 9 and 11 shown in FIG. 7 are branched off by terminals or stages succeeding the address 15, 16 shown in FIG. 8 are branched off by terminals.
  • the signals are processed in response to the gyration of the head of each listener after corrected and converted by the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12 into the digital signals having the spatial information. Therefore, it is unnecessary to prepare the expensive A/D converters 3 and convolution integrators 5, 7, 9 and 11 which are as many as the number of the listeners.
  • the digital angle detector 28 or the analog angle detector 38 when the listener 23 turns the head, the digital angle detector 28 or the analog angle detector 38 generates the digital signal or the analog signal in response to the direction of the movement of the head.
  • the signal has a value in response to the direction of the head of the listener 23.
  • the value is supplied therefrom through the address control circuit 34 as the address signal to the memory 35.
  • the digitally recorded impulse responses corresponding to the direction of the head of the listener 23, from the virtual sound positions with respect to the reference direction of the head to both the ears among the data corresponding to those stored in the table shown in FIG. 4 or the control signals representing the difference in time between the sounds obtained at both the ears and the difference in level therebetween among the data shown in FIG. 6.
  • the read data are supplied to the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12 or the control apparatus 50, 51, 52, 53, 54 and 56.
  • the output therefrom is amplified by the amplifier 42 and converted by the A/D converter 33 into the digital signal in response to the direction of the head of the listener 23.
  • the digital signal is supplied as the address signal through the address control circuit 34 to the memory 35.
  • the digitally recorded impulse responses corresponding to the direction of the head of the listener 23, from the virtual sound positions with respect to the reference direction of the head to both the ears or the control signals shown in FIG. 6 representing the difference in time between the sounds obtained at the ears and the difference in level therebetween.
  • the read data are supplied to the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12 or the control apparatus 50, 51, 52, 53, 54 and 56.
  • the correcting circuits 17, 18 have one, combination or all of the correction characteristics used to correct the characteristics inherent in the sound sources, the sound field and the headphones to be used. Accordingly, since the correcting circuits can carry out the digital signal processings including the above correction at once, they can carry out the signal processing in a real-time fashion.
  • the audio signals L, R to be supplied to the headphone 24 are corrected based on the digitally recorded impulse responses from the virtual sound source positions corresponding to the head direction of the listener 23 with respect to the reference direction of the head to both the ears or the control signals representing the difference in time between the sounds obtained at both the ears and the difference in level therebetween, it is possible to obtain the sound field which allows the listener to feel as if a plurality of speakers were located at the virtual sound source positions and the audio signals were reproduced thereby.
  • control signals which are digitally recorded in the table of the memory 35 and represent the difference in time between the sounds obtained at both the ears and the difference in level therebetween are read out therefrom. Since the data of the control signals are purely electronically supplied to the control apparatus 50, 51, 52 and 53 in order that the control apparatus correct the digital signals previously convoluted by the convolution integrators 5, 7, 9 and 11 and the memories 6, 8, 10 and 12, the characteristics of the audio signals can be changed without delay after the listener 23 turns the head. Therefore, the listener is prevented from feeling unnatural.
  • reverberation signals generated by reverberation circuits 13, 14 are supplied to the headphone 24 so that such a spacial impression as is obtained in a listening room and a concert hall is added to the audio signals. Therefore, it is possible for the listener to perceive an excellent stereophonic sound field.
  • the signals may be transmitted thereto in a wireless fashion by providing a modulator and a transmitter at a stage succeeding the convolutional integrators 5, 7, 9 and 11 shown in FIG. 7 and a receiver and a demodulator on the headphone 24 side and receiving a transmitted signal by the receiver and the modulator or by providing a modulator and a transmitter at a stage succeeding the adders 15, 16 shown in FIG. 8 and a receiver and a modulator and receiving a transmitted signal by the receiver and the modulator.
  • change amounts of the digitally recorded control signals representing the difference in time between the sounds obtained at both the ears and the difference in level therebetween obtained when the angle ⁇ is changed are set to be larger or smaller than a standard value by setting a table, then amounts of positional changes of the sound images with respect to the head direction of the listener 23 are different from each other. Therefore, it is possible to change perception of distance from the listener 23 to the sound image and to set the perception in accordance with a screen size.
  • the reverberation signals generated by the reverberation circuits 13, 14 are added to the reproduced sounds and allows the listener to listen to the reproduced sounds as if the sounds were sounds reflected by a wall of a hall or a reverberation sounds, it is possible to obtain the presence which allows the listener to feel as if he listened to the music in a famous concert hall.
  • FIG. 11 shows positions of the reproduced sound images. Directivity of the sound image is controlled by the information indicative of the microphone position and the information indicative of the sound source position supplied from the positional information detecting apparatus 103 and 105 shown in FIG. 10. Sounds corresponding to the sound sources are controlled in their directions so as to be oriented from the sound sources corresponding to positions of objects 117, 118 and 119 reproduced on a screen 116 as a display unit of a TV monitor and located at the positions of the objects 117, 118 and 119.
  • real sound image positions are determined at positions from which beams of positions corresponding to the sound source positions on the screen 116 of the TV monitor 115 are oriented. It is possible to form the real sound image at the TV monitor 115 as if the sounds were emanated from the positions of the sound sources in the image displayed on the screen 116 of the TV monitor 115.
  • a positional information changer 93 is provided on an inner side of the headphone 24.
  • the switch of the positional information changer is turned on.
  • the listener 23 rotates with respect to the reference direction or moves close to or away from the reference position, it is possible to change the coordinates of the information indicative of the microphone positions and the information indicative of the sound source positions and supplied from the positional information detecting apparatus 103, 105 shown in FIG. 10 in accordance with the listener's movement.
  • the real sound image positions are determined at positions from which beams of positions corresponding to the changed sound source positions on the screen 116 of the TV monitor 115 are oriented. It is possible to form the real sound image at the TV monitor 115 as if the sound was emanated from the positions of the changed sound sources in the image displayed on the screen 116 of the TV monitor 115.
  • the positional information changer 93 shown in FIGS. 9 and 11, provided on the inner side of the headphone 24 corresponds to the position detector 63 and the detecting circuit 64 shown in FIGS. 1, 7 and 8.
  • the TV monitors 92, 115 shown in FIGS. 9 and 11 correspond to the video signal reproducing apparatus 62 shown in FIGS. 1, 8 and 9.
  • the reference direction and the reference position may correspond to the TV monitor 92 or may be changed arbitrarily.
  • the positional information changer 93 may be a simple input apparatus, e.g., a personal computer, for example.
  • the positional information changer 93 may change a specific part of the coordinates of the information indicative of the microphone position and the information indicative of the sound source positions supplied from the positional information detecting apparatus 103, 105 shown in FIG. 10 to add changed positional information to original positional information.
  • the addresses of the memories 6, 8, 10 and 12 or the memory 35 are designated by the address signal from the address control circuit 34 as the address signal converting means.
  • the impulse responses or the control signals stored in the memories 6, 8, 10 and 12 or the memory 35 as the storage means are read out therefrom.
  • the audio signals are corrected by the convolutional integrators 5, 7, 9 and 11 or the control apparatus 50, 51, 52, 53, 54 and 56 as the control means.
  • the audio signals are corrected in a real-time fashion with respect to the head movement of the one or plurality of listeners 23.
  • the audio signals corrected by the convolutional integrators 5, 7, 9 and 11 or the control apparatus 50, 51, 52, 53, 54 and 56 as the control means are reproduced by the headphone 24 such that a plurality of reproduced sound images are localized at the positions corresponding to the image reproduced by the video signal reproducing apparatus 62 and the TV monitors 92 and 115 as the video reproducing means based on the reproduced sound image position information indicative of both or either of the positions of the microphones 104, 106, 107 and 108 used upon the pickup of a sound and the positions of the sound sources 100, 101 and 102 used upon the pickup of the sound.
  • the positional information detector 93 as the reproduced sound image position information changes the coordinates of the reproduced sound image position information in response to the head movement of the listener 23. Thus, it is possible to localize the reproduced sound images in a bidirectional fashion.
  • the positional information changer 93 as the reproduced sound image position information changing means changes the coordinates of the reproduced sound image position information based on at least the information indicative of the gyration with respect to the reference direction and the information indicative of the listener's movement close to or away from the reference position, it is possible to change the coordinates of the reproduced sound image when the listener is moved.
  • the positional information changer 93 as the reproduced sound image position information changing means changes the coordinates of the reproduced sound image position information by adding thereto at least the information indicative of the gyration with respect to the reference direction and the information indicative of the listener's movement close to or away from the reference position, it is possible to change a part of the coordinates of the reproduced sound image when the listener is moved.
  • a plurality of reproduced sound images may be localized at the positions of the objects 117, 118 and 119 reproduced on the screen of the TV monitor with corresponding to the respective positions.
  • real sound image positions are localized at positions from which beams of positions corresponding to the sound source positions on the screen 116 of the TV monitor 115 are oriented. It is possible to form the real sound image at the TV monitor 115 as if the sound was emanated from the positions of the sound sources in the image displayed on the screen 116 of the TV monitor 115.
  • the audio signals corrected by the convolutional integrators 5, 7, 9 and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 are reproduced by the headphone 24 based on the reproduced sound image position information indicative of both or either of the positions of the microphones 104, 106, 107 and 108 and the positions of a plurality of the sound sources 100, 101 and 102 such that a plurality of reproduced sound images are localized at the positions of a plurality of objects 117, 118 and 119 of the image reproduced by the TV monitors 92 and 115 with corresponding to respective positions, it is possible to form the real sound images which allow the listener to feel as if the sounds were emanated from the positions of the plurality of objects 117, 118 and 119 of the reproduced image.
  • the image reproduced from the video signal may include images of the sound sources 100, 101 and 102, and by controlling the directivity of the sound images based on the information indicative of the microphone positions and the information indicative of the sound source positions and supplied from the positional information detecting apparatus 103 and 105 and orienting the corresponding sounds from the positions of the objects 117, 118 and 119 reproduced on the screen 116 as the display unit of the TV monitor 115, a plurality of reproduced sound images may be localized at the positions of the objects 117, 118 and 119 reproduced on the screen of the TV monitor with corresponding to the positions.
  • real sound image positions are defined at positions from which beams of positions corresponding to the sound source positions on the screen 116 of the TV monitor 115 are oriented. It is possible to form the real sound image at the TV monitor 115 as if the sound was emanated from the positions of the sound sources in the image displayed on the screen 116 of the TV monitor 115.
  • the image reproduced from the video signal includes the images of the sound sources 100, 101 and 102 and the audio signals corrected by the convolutional integrators 5, 7, 9 and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 are reproduced by the headphone 24 based on the reproduced sound image position information indicative of the positions of a plurality of the sound sources 100, 101 and 102 such that the reproduced sound images are localized at the positions of a plurality of objects 117, 118 and 119 of the image reproduced by the TV monitors 92 and 115 with corresponding to respective positions, it is possible to form the real sound images which allow the listener to feel as if the sounds are emanated from the positions of the plurality of objects 117, 118 and 119 of the reproduced image.
  • the positions of the microphones 104, 106, 107 and 108 may be changed in response to a scene of the image reproduced from the video signal, and by controlling the directivity of the sound images based on the information indicative of the microphone positions supplied from the positional information detecting apparatus 103 and 105 and orienting the corresponding sounds from the respective positions of the objects 117, 118 and 119 reproduced on the screen 116 as the display unit of the TV monitor 115, a plurality of reproduced sound images may be localized at the positions of the objects 117, 118 and 119 with corresponding to the positions.
  • real sound image positions are defined at positions from which beams of positions corresponding to the sound source positions on the screen 116 of the TV monitor 115 are oriented. It is possible to form the real sound image at the TV monitor 115 as if the sound was emanated from the positions of the sound sources in the image displayed on the screen 116 of the TV monitor 115.
  • the positions of the microphones 104, 106, 107 and 108 may be changed in response to a scene of the image reproduced from the video signal and the audio signals corrected by the convolutional integrators 5, 7, 9 and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 are reproduced by the headphone 24 based on the reproduced sound image position information indicative of the positions of the microphones 104, 106, 107 and 108 upon picking up sounds such that the reproduced sound images are localized in the directions to the positions of the microphones 104, 106, 107 and 108 of the image reproduced by the TV monitors 92 and 115, it is possible to form the real sound images which allow the listener to feel as if the sounds were emanated from the directions of the positions of the microphones 104, 106, 107 and 108 of the reproduced image.
  • a reset switch 90 is provided at the headphone 24.
  • the reference position of the head gyration is set.
  • a reset switch 91 may be provided on an inner side of the headphone 24 to reset the reference position during a predetermined time when the listener wears the headphone 24.
  • the digital angle detector 28 and the analog angle detector 38 have the reset switches 90 and the direction in which the listener 23 turns the head when the reset switch 90 is turned on is set as the reference direction, it is possible to set an optional direction as a front direction by operation of the reset switch 90.
  • the headphone 24 since the headphone 24 includes the reset switch 91 and the digital angle detector 28 and the analog angle detector 38 set the direction toward the front of the screen of the TV monitor 92 as the reference direction when the listener 23 wears the headphone 24, it is possible to always set the direction toward the picture screen as the front direction whenever the listener wears the headphone 24.
  • the TV monitors 92 and 115 may be a movie screen, for example.
  • the screen may have a plurality of the screen sizes such as a cinema scope size, a vista size or the like and the reproduced sound image may be localized in accordance with the screen size based on the reproduced sound image position information indicative of both or either of the microphone positions and the sound source positions obtained when the sounds are picked up.
  • the positional information changer 93 may be provided with the input terminal 65 shown in FIGS. 1, 7 and 8 to input a switching signal in accordance with the screen size to the input terminal 65.
  • the screen has a plurality of the screen sizes and the audio signals corrected by the convolutional integrators 5, 7, 9 and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 are reproduced by the headphone 24 such that a plurality of reproduced sound images are localized in the direction corresponding to the image reproduced on the screen with corresponding to the screen size by changing kinds of tables of the memory 35 or the memories 6, 8, 10 and 12 or the change amounts read therefrom based on the reproduced sound image position information indicative of both or either of thee microphone positions and the sound source positions obtained when the sounds are picked up, it is possible to localize the plurality of reproduced sound images in the direction corresponding to the image even if the screen size of the screen is changed.
  • the reproduced sound images may be localized so as to correspond to the images on the screens based on the reproduced sound image position information indicative of both or either of the microphone positions and the sound source positions obtained when the sounds are picked up.
  • the arrangement according to the above embodiment is suitable for use in localizing the reproduced sound images in response to the movement of the image in the movement direction of the image based on the reproduced sound image position information indicative of both or either of the microphone positions and the sound source positions obtained when the sounds are picked up. It is also possible to localize the positions of the reproduced sound images in a bidirectional fashion.
  • impulse sound sources and dummy-head microphones of necessary channel number are disposed at predetermined positions in a suitable room such that a preferable reproduced sound field should be obtained when the sound is reproduced by the headphone 24.
  • the speakers may be used as sound sources used to measure the impulses.
  • Positions where sound waves are picked up in each of ears of the dummy head may be anywhere from the inlets of the external auditory canal thereof to the eardrum thereof. However, the positions should be equal to positions used to obtain the correction characteristics for canceling the characteristics inherent in the headphones to be used.
  • the control signals can be measured by radiating impulse sounds from the speakers in the respective channels and picking up the radiated impulse sounds with microphones provided in the ears of the dummy head at every constant angle ⁇ . Accordingly, since one set of impulse responses is obtained per channel at a certain angle ⁇ 1, if the signal sources has five channels, then five sets of control signals, i.e., ten control signals can be obtained per angle. Accordingly, the control signals representing the difference in time between the sounds obtained at the left and right ears and the difference in level therebetween are obtained from the impulse responses.
  • the correction characteristics for canceling the characteristics inherent in the sound sources, the sound field, the headphones which are used, and so on are calculated in such a manner that the same dummy-head microphones as those used to obtain impulse responses of a sound field are used, the headphone to be used are mounted on the dummy head, and impulse responses having inverted characteristics of impulse responses between the microphones in the respective ears of the dummy head are calculated from inputs from the headphone.
  • the correction characteristics may be directly calculated by using adaptive processings such as an LMS algorithm or the like so as to become approximate to a desired value as a correction value for canceling the characteristics inherent in the headphone.
  • Specific correction of characteristics inherent in the headphones can be realized by either subjecting the digital audio signals to the convolution integral with the impulse responses representing the calculated correction characteristics in view of a processing in a time domain or filtering the analog signal obtained by the D/A conversion by an analog filter having inverted characteristics in view of an analog signal processing at any time from a time when the audio signals are input to a time when the audio signals are supplied to the headphone.
  • control data can be obtained similarly to a case where the memory has plural sets of tables.
  • the data stored in the table may be limited to a range of a general direction of the head of the listener 23.
  • the angle may be set to be the angle through which the listener can perceive that he turns the head.
  • speakers disposed near the respective ears of the listener 23 may be substituted for the headphone 24.
  • the input audio signals may be digitally recorded signals or signals recorded in an analog fashion both of which are picked up in a multichannel stereophonic mode or the like.
  • the angle detection means for detecting the movement of the head of the listener 23 may output a digital signal or an analog signal.
  • the characteristics of audio signals supplied to the headphone 24 are changed in synchronism with the movement of the head of the listener 23, the characteristics are changed not continuously in response to the movement of the head of the listener 23 but by reading data from the tables of the memory 35 at either of every constant unit angle and every predetermined angle which are necessary and sufficient for human beings to recognize in accordance with human auditory characteristics. Therefore, the same effect as that achieved when the characteristics of the audio signals are continuously changed can be achieved only by calculation with resect to necessary and sufficient changes in the movement of the head of the listener 23. Accordingly, the storage capacity of the memory 35 can be saved and high-speed calculations more than required becomes unnecessary in view of a processing speed of calculations.
  • the characteristics of the audio signals are changed without delay after the listener 23 turns the head, the listener is prevented from feeling such unnaturalness as he feels when using a conventional system.
  • the change amounts of the control signals representing the difference in time between the sounds obtained at the respective ears and the difference in level therebetween obtained when the angle ⁇ is changed are set to be larger or smaller than the standard value depending upon the tables, amounts of positional changes of the sound images with respect to the head direction of the listener 23 are different from each other. Therefore, it is possible to change perception of distance from the listener 23 to the sound image and to set the perception in accordance with the screen size.
  • the suitable reverberation signals generated by the reverberation circuits 13, 14 are added to the reproduced sounds if necessary, it is possible to obtain the presence which allows the listener to feel as if he listened to the music in a famous concert hall.
  • the audio signals are corrected in response to the respective gyrations of the heads of a plurality of listeners 23 based on the control signals representing the difference in time between the sounds obtained at the respective ears and the difference in level therebetween, the audio signals can be reproduced by a plurality of headphones 24 simultaneously and it is unnecessary to prepare the expensive A/D converters 3 and the expensive convolution integrators 5, 7, 9 and 11 which are as many as the number of the listeners 23. Therefore, the apparatus can be arranged with considerably inexpensive costs.
  • the vibratory gyroscope may be used as the head gyration detector.
  • a head gyration detector it is possible for a head gyration detector to be small and light, to have low consumed power and long lifetime, and further to be easy to handle and inexpensive.
  • the vibratory gyroscope since the vibratory gyroscope does not utilize an inertial force but is operated by a Coriolis force, it is unnecessary to dispose the vibratory gyroscope in the vicinity of a center of the gyration of the head of the listener 23 and hence the vibratory gyroscope may be attached to any portion of the gyration detection unit. Therefore, it is possible to simplify its arrangement and fabrication.
  • FIGS. 12 to 14 A video signal and audio signal reproducing apparatus according to another embodiment of the present invention will hereinafter be described with reference to FIGS. 12 to 14.
  • the video signal and audio signal reproducing apparatus of the embodiment of the present invention when a video signal and an audio signal having reproduced sound image position information indicative of both or either of the positions of the microphones used upon the sound pickup and the positions of the sound sources used upon the sound pickup are reproduced through a headphone while a listener watches a picture, the listener can perceive the equivalent localization, sound field and so on to those perceived when the audio signals are reproduced by speakers located in a predetermined positional relationship in which the speakers should be located when the audio signals are reproduced by the headphones.
  • a plurality of reproduced sound images are located in the direction corresponding to the picture on the basis of the reproduced sound image position information indicative of the position of the microphone and the position of the sound source or the position of either one thereof.
  • the video signal and audio signal reproducing apparatus is used in a system of reproducing multichannel video and audio signals, which are obtained by picking up a sound in a stereophonic fashion or the like and have the reproduced sound image position information indicating both or either of the microphone position and the sound source position obtained when the sound is picked up, through the headphone while the listener watches a picture.
  • the reproduced sound images in a plurality of channels are localized at a position corresponding to the picture based on the reproduced sound image position information indicating both or either of the microphone position and the sound source position obtained when the sound is picked up.
  • FIG. 12 shows an example of the video signal and audio signal reproducing apparatus according to the present invention.
  • the arrangements shown in FIGS. 12 to 14 correspond to those shown in FIGS. 1, 7 and 8, respectively. Only different points of the arrangements shown in FIGS. 12 to 14 from those shown in FIGS. 1, 7 and 8 will be described and the common arrangements and operations need not be described.
  • the video signal is supplied to the video signal reproducing apparatus 62 and to a positional information extracting circuit 66.
  • the positional information extracting circuit 66 is a circuit for previously extracting the reproduced sound image position information supplied together with the video signal.
  • the reproduced sound image position information indicates both or either of the positions, on the screen of the video signal reproducing apparatus 62, of the microphone and the sound source used when the sound is picked up.
  • the positional information from the positional information extracting apparatus 66 is supplied to the switcher 36 for switching the address of the address control circuit 34.
  • the switching signal generated in accordance with the screen size of the screen of the video signal reproducing apparatus 62 is also supplied to the switcher 36 through the input terminal 65.
  • the memory 35 includes three sets of the above tables, for example, each of sets having different data value depending upon the relative positional relationship such as the distance between the screen of the predetermined video signal reproducing apparatus 62 and the listener 23, the angle made thereby or the like, the screen size of the screen of the video signal reproducing apparatus 62 and so on.
  • An optimum set of the three sets of tables is selected by switching the address of the address control circuit 34 by switching the switcher 36 thereof.
  • the video signal and audio signal reproducing apparatus are arranged as described above and operates as follows. Specifically, the video signal and the audio signal are input from the input terminal 60.
  • the separating circuit 61 separates the input signal into the video signal and the audio signal.
  • the video signal is supplied therefrom to the video signal reproducing apparatus 62. If the audio signal separated by the separating circuit 61 is the digital audio signal, then it is supplied to the digital stereophonic signal source 1, and if it is the analog audio signal, then it is supplied to the analog stereophonic signal source 2.
  • both the digital audio signals and the analog audio signals are the audio signals separated from the video signals having the reproduced sound image position information indicative of both or either of the microphone position and the sound source position obtained when the sound is picked up.
  • the relative reproduced sound image position information indicative of the distance between the screen of the predetermined video signal reproducing apparatus 62 and the listener 23, the angle made thereby and so on is supplied from the positional information extracting circuit 66 to the address control circuit 34.
  • the switching signal generated in accordance with the screen size of the screen of the video signal reproducing apparatus 62 is also supplied through the input terminal 65 to the switcher 36.
  • the address of the address control circuit 34 is switched by switching the switcher 36.
  • the memory 35 includes three sets of the above tables, for example, in advance each of sets having different data values depending upon the relative positional relationship such as the distance between the screen of the predetermined video signal reproducing apparatus 62 and the listener 23, the angle made thereby or the like, the screen size of the screen of the video signal reproducing apparatus 62 and so on.
  • An optimum set of the three sets of the tables is selected by switching the address of the address control circuit 34 by switching the switcher 36 thereof such that the reproduced sound image position is changed in response to the change of the reproduced sound image position information or the screen size.
  • the address of the address control circuit 34 is switched in accordance with the relative relationship such as the distance between the screen of the predetermined video signal reproducing apparatus 62 and the listener 23, the angle made thereby and so on and an optimum table is selected, it is possible to set the reproduced sound image position in accordance with both or either of the positions, on the screen of the video signal reproducing apparatus 62, of the microphones and the sound sources used when the sounds are picked up.
  • the address of the address control circuit 34 is switched in accordance with the screen size of the screen of the video signal reproducing apparatus 62 and an optimum table is selected, even when the screen size of the video signal reproducing apparatus 62 is switched, it is possible to set the reproduced sound image position in accordance with both or either of the positions, on the screen having the switched screen size, of the microphones and the sound sources used when the sounds are picked up.
  • the audio signals L, R supplied to the headphone 24 are corrected based on the digitally recorded impulse response from the virtual sound source positions with respect to the reference direction of the head corresponding to the head direction of the listener 23 to the both ears or the control signal representing the difference in time between the sounds obtained at the both ears and the difference in level therebetween, it is possible to obtain, by localizing the reproduced sound image in accordance with the picture, the perception of the sound field which allows the listener to feel as if the audio signals were reproduced by a plurality of speakers located at the virtual sound source positions.
  • a video signal and audio signal reproducing apparatus according to another embodiment of the present invention will hereinafter be described in detail with reference to FIGS. 15 to 26.
  • the listener when an audio signal is reproduced through the headphone while the listener watches the picture, the listener can perceive the equivalent localization, sound field and so on to those perceived when the audio signals are reproduced by speakers located in a predetermined positional relationship in which the speakers should be located when the audio signals are reproduced by the speakers.
  • a plurality of reproduced sound images are located in the direction corresponding to the reproduced pictures projected on positions distant from left and right eyes of the listener 23 by a predetermined distance.
  • the video signal and audio signal reproducing apparatus is used in a system of reproducing multichannel video and audio signals, which are obtained by picking up a sound in a stereophonic fashion or the like, through the headphone while the listener watches the picture.
  • digitized audio signals in respective channels to be recorded or transmitted for localizing the respective sound images in the predetermined relationship e.g., at right, left and center positions in front of the listener and other positions
  • the reproduced sound images in a plurality of channels are localized in the direction corresponding to the reproduced pictures projected to positions distant from the left and right eyes of the listener 23 by a predetermined distance.
  • FIG. 15 shows an example of the video signal and audio signal reproducing apparatus according to the present invention. Arrangements shown in FIGS. 15 to 17 respectively correspond to those shown in FIGS. 12, 13 and 14. Only different points of the arrangements shown in FIGS. 15 to 17 from those shown in FIGS. 12, 13 and 14 will be described and the common arrangements and operation need not be described.
  • the video signal and the audio signal are input from the input terminal 60.
  • the separating circuit 61 separates the input signal into the video signal and the audio signal.
  • the video signal is supplied therefrom to a video signal reproducing circuit 67.
  • the video signal reproducing circuit 67 subjects the video signal to a predetermined pre-processing for reproduction and then supplies the processed video signal to a virtual image display 193. If the audio signal separated by the separating circuit 61 is the digital audio signal, then it is supplied to the digital stereophonic signal source 1, and if it is the analog audio signal, then it is supplied to the analog stereophonic signal source 2.
  • the video signal is supplied to the positional information extracting circuit 66.
  • the positional information extracting circuit 66 is a circuit for previously extracting the reproduced sound image position information supplied together with the video signal.
  • the reproduced sound image position information indicates both or either of the positions, on the screen of the virtual image display 193, of the microphone and the sound source used when the sound is picked up.
  • the video signal After being subjected to the predetermined pre-processing for reproduction by the video signal reproducing circuit 67, the video signal is supplied to the virtual image display 193 which the listener 23 wears.
  • the video signal and audio signal reproducing apparatus is arranged as described above and is operated as follows.
  • the video signal and the audio signal are input from the input terminal 60.
  • the separating circuit 61 separates the video signal and the audio signal.
  • the video signal is supplied therefrom to the video signal reproducing circuit 67. If the audio signal separated by the separating circuit 61 is the digital audio signal, then it is supplied to the digital stereophonic signal source 1, and if it is the analog audio signal, then it is supplied to the analog stereophonic signal source 2.
  • the memory 35 includes three sets of the above tables, for example, each of sets having different data value depending upon the relative positional relationship such as the distance between the screen of the virtual image display 193 and the listener 23, the angle made thereby or the like, the screen size of the screen of the virtual image display 193 and so on.
  • An optimum set of the three sets of the tables is selected by switching the address of the address control circuit 34 by switching the switcher 36.
  • the address of the address control circuit 34 is switched in accordance with the relative relationship such as the distance between the screen of the virtual image display 193 and the listener 23, the angle made thereby and so on and an optimum table is selected, it is possible to set the reproduced sound image position in accordance with both or either of the positions, on the screen of the virtual image display 193, of the microphones and the sound sources used when the sounds are picked up.
  • the address of the address control circuit 34 is switched in accordance with the screen size of the screen of the virtual image display 193 and an optimum table is selected, even when the screen size of the virtual image display 193 is switched, it is possible to set the reproduced sound image position in accordance with both or either of the positions, on the screen having the switched screen size, of the microphones and the sound sources used when the sounds are picked up.
  • the audio signals L, R supplied to the headphone 24 are corrected based on the digitally recorded impulse response from the virtual sound source positions with respect to the reference direction of the head corresponding to the head direction of the listener 23 to the both ears or the control signal representing the difference in time between the sounds obtained at the both ears and the difference in level therebetween, it is possible to obtain the perception of the sound field which allows the listener to feel as if the audio signals were reproduced by a plurality of speakers located at the virtual sound source positions.
  • the virtual image display 193 is supplied with information on the head movement of the listener 23 with respect to the reference direction from the digital angle detector 28 or the analog angle detector 38. Accordingly, when the listener 23 turns the head, the listener 23 may watch that the image projected at the position distant from the left and right eyes of the listener by a predetermined distance is continuously changed based on the information as if the listener was watching the image with changing the angle of his head.
  • the change amounts of the digitally recorded control signals representing the difference in time between the sounds obtained at the respective ears from the virtual sound source positions with respect to the reference direction of the head of the listener 23 to the both ears and obtained when the angle ⁇ is changed and the difference in level therebetween are set to be larger or smaller than the standard value depending upon the tables, then the amounts of positional changes of the sound images with respect to the head direction of the listener 23 are different from each other. Therefore, it is possible to change perception of distance from the listener 23 to the sound image.
  • the change amounts it is possible to change the reproduced images projected on the positions distant from the left and right eyes of the listener 23 by a predetermined distance in accordance with the screen size of the virtual image display used in the video signal and audio signal reproducing apparatus.
  • FIG. 18 shows a principle of an operation of the virtual image display of the video signal and audio signal reproducing apparatus according to another embodiment of the present invention.
  • the virtual image display 193 has a liquid crystal display apparatus (hereinafter referred to as "LCD") 184 disposed in front of a right eye 180 of the listener 23 through a lens 182 and an LCD 185 disposed in front of a left eye 181 through a lens 183. Reproduced images displayed on the LCDs 184 and 185 are magnified by the lenses 182 and 183. Thus, the video signal is reproduced by projecting a virtual image 187 in front of the LCD 184 and the LCD 185.
  • LCD liquid crystal display apparatus
  • the image informations projected on the LCDs 184 and 185 are converted by the lenses 182 and 183 as eyepieces into the virtual images.
  • the virtual images are input as independent informations through the right eye 180 and the left eye 181 into a brain. In the brain, the different informations are superposed on each other to form one image.
  • the apparatus is adjusted with an angle 186 of convergence of eyes as an angle at which the visual axes are inclined inward.
  • LCDs originally developed by the applicant of this invention are used as the LCDs 184 and 185, being 0.7-inch 103 thousand pixels LCD for reproducing a highly fine definite image. Since the two LCDs are used as the LCDs 184 and 185 for the right eye 180 and the left eye 182, respectively, it is possible to reproduce an image on a large screen as the virtual image 187. It is possible to employ a 3D monitor for projecting a stereoscopic image. It is possible to realize a 2D image with the same image as that projected on the LCDs 184 and 184 and a 3D image with an image with parallax.
  • small-sized high-magnification rectangular aspheric plastic lenses which are developed by the applicant of this application independently are employed as the lenses 182 and 183.
  • plastic lenses it is possible to reproduce a sharp image on an entire screen of the LCDs by suppressing distortion produced when the images projected on the LCDs 184 and 185 are magnified.
  • FIG. 19 shows an appearance of the virtual image display of the video signal and audio signal reproducing apparatus according to another embodiment of the present invention.
  • the virtual image display 193 has a scope 194 which has the lenses 182 and 183 and the LCDs 184 and 185 and which is disposed so as to cover the left and right eyes of the listener 23 and is mounted on the head of the listener 23 by using an arm 196.
  • a forehead pad 195 supports a forehead of the listener 23 and attachment of the virtual image display can be adjusted by an adjuster 197 so as to be tighter or looser.
  • the virtual image display 193 is formed independently of the headphone 24 in the above embodiment, the virtual display 193 and the headphone 24 are integrally formed with the arm 196 of the virtual image display and the head band 27 of the headphone being fixed to each other.
  • the virtual image display 193 as the video reproducing means has the LCD 185 as a left liquid crystal display unit and the LCD 184 as a right liquid crystal display unit respectively disposed at the positions corresponding to the left and right eyes of the listener 23, it is possible to project the reproduced image onto the positions distant from the left and right eyes of the listener 23 by a predetermined distance by the LCD 185 as the left liquid crystal display unit and the LCD 184 as the right liquid crystal display unit.
  • the virtual image display 193 as the video reproducing means has the LCD 185 as a left liquid crystal display unit and the LCD 184 as a right liquid crystal display unit respectively disposed through the left and right rectangular aspheric eyepieces 182 and 183 at the positions corresponding to the left and right eyes of the listener 23, it is possible to project the reproduced images onto the positions distant from the left and right eyes of the listener 23 by a predetermined distance in front of the LCD 185 as the left liquid crystal display unit and the LCD 184 as the right liquid crystal display unit by magnifying the images projected on the LCD 185 as the left liquid crystal display unit and the LCD 184 as the right liquid crystal display unit.
  • FIGS. 20 to 26 show simulated arrangements of the speakers of the video signal and audio signal reproducing apparatus according to another embodiment of the present invention.
  • the direction to a virtual image position 192 on which the picture reproduced from the video signal corresponding to the audio signal is projected is set as the front direction.
  • the simulated arrangements of the speakers are made as follows. Initially, the sound image is localized as if the speakers were disposed in a forward region A from a straight line passing through the left and right ears 23L and 23R of the listener 23. Subsequently, the sound image is localized as if the speakers were disposed in a region B on the straight line passing through the left and right ears 23L and 23R of the listener 23. Further, the sound image is localized as if the speakers were disposed in a backward region C from the straight line passing through the left and right ears 23L and 23R of the listener 23.
  • the convolutional integrators 5, 7, 9 and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 as the control means are reproduced by the headphone 24 as an audio reproducing means such that a plurality of reproduced sound images are localized in front of and behind the listener 23 in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance when the video signal is reproduced by the virtual image display 193 as the video signal reproducing means, it is possible to localize the plurality of reproduced sound images in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance.
  • the reproduced sound image in front of the listener if the picture at the virtual image position 192 is in front of the listener, behind the listener if the picture is behind the listener, on the left side of the listener if the picture is on the left side of the listener, and on the right side of the listener if the picture is on the right side of the listener. If a picture is moved, then the reproduced sound image is moved in accordance with the movement of the picture and then localized at the predetermined position.
  • the virtual image display 193 is supplied with information on the head movement of the listener 23 with respect to the reference direction from the digital angle detector 28 or the analog angle detector 38. Accordingly, when the listener 23 turns the head, the listener 23 may watch that the picture projected on the virtual image position 192 distant from the left and right eyes of the listener by a predetermined distance is continuously changed based on the information as if the listener was watching the picture with changing the angle of his head.
  • the reference position of the head gyration is set by pressing a reset switch 190 provided in the headphone 24 by the listener 23.
  • the reset may be effected when the listener 23 wears the headphone 24 having a reset switch 191 on its inner side.
  • the direction may be set as the reference direction.
  • the digital angle detector 28 or the analog angle detector 38 since the digital angle detector 28 or the analog angle detector 38 has the reset switch 190 and the direction in which the listener 23 turns the head when the reset switch 190 is turned on is set as the reference direction, it is possible to set an optional direction as the front direction by operating the reset switch 190.
  • the digital angle detector 28 or the analog angle detector 38 sets the direction as the reference direction, it is possible to automatically set the predetermined direction as the reference direction.
  • the headphone 24 since the headphone 24 has the reset switch 191 and the digital angle detector 28 or the analog angle detector 38 sets the direction toward the front of the screen at the virtual image position 192 as the reference direction when the listener 23 wears the headphone 24, it is possible to constantly set the direction toward the picture screen as the front direction when the listener wears the headphone 24. While the headphone 24 is provided with the reset switches 190 and 191 in the above embodiment, the virtual image display 193 may be provided therewith.
  • the specific simulated arrangement of the speakers are made as shown in FIGS. 21 to 26.
  • the simulated arrangement of the speaker for one-channel monophonic reproduction is made as shown in FIG. 21. Specifically, a virtual image position 211 on which the picture is projected is disposed in front of seats 210 where the listener 23 stays. At this time, the audio signal is reproduced such that the reproduced sound image is localized as if a center speaker C was located at the center of the virtual image position 211 in front of the seats 210.
  • the simulated arrangement of the speakers for two-channel stereophonic reproduction is made as shown in FIG. 22. Specifically, a virtual image position 221 on which the picture is projected is disposed in front of seats 220 where the listener 23 stays. At this time, the audio signal is reproduced such that the reproduced sound image is localized as if a left speaker L and a right speaker R were respectively located on the left and right sides of the virtual image position 221 in front of the seats 220.
  • the simulated arrangement of the speakers for three-channel reproduction is made as shown in FIG. 23.
  • a virtual image position 231 on which the picture is projected is disposed in front of seats 230 where the listener 23 stays.
  • the audio signal is reproduced such that the reproduced sound image is localized as if a center speaker C was located at the center of the virtual image position 231 in front of the seats 230, a left speaker L and a right speaker R were respectively located on the left and right sides of the virtual image position 231, and a sub woofer speaker W was located in the vicinity of the center speaker C.
  • the simulated arrangement of the speakers for four-channel reproduction is made as shown in FIG. 24.
  • a virtual image position 241 on which the picture is projected is disposed in front of seats 240 where the listener 23 stays.
  • the audio signal is reproduced such that the reproduced sound image is localized as if a center speaker C was located at the center of the virtual image position 241 in front of the seats 240, a left speaker L and a right speaker R were respectively located on the left and right sides of the virtual image position 241, surround speakers S were located at the left and right rear position of the seats 240 and at rear left and right sides of the seats, and a sub woofer speaker W was located in the vicinity of the center speaker C.
  • the simulated arrangement of the speakers for five-channel reproduction is made as shown in FIG. 25.
  • a virtual image position 251 on which the picture is projected is disposed in front of seats 250 where the listener 23 stays.
  • the audio signal is reproduced such that the reproduced sound image is localized as if a center speaker C was located at the center of the virtual image position 251 in front of the seats 250, a left speaker L and a right speaker R were respectively located on the left and right sides of the virtual image position 251, left surround speakers S L were located at the left rear position and rear left side of the seats 250, right surround speakers S R were located at the right rear position and rear right side of the seats 250, and a sub woofer speaker W was located in the vicinity of the center speaker C.
  • the convolutional integrators 5, 7, 9 and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 as the control means are reproduced by the headphone 24 as the audio reproducing means such that the five-channel reproduced sound images are localized at the center front, left and right front, and right and left rear of the listener 23 in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance when the virtual image display 193 as the video reproducing means reproduces the video signal, it is possible to localize the five-channel reproduced sound images at the center front, left and right front and right and left rear of the listener 23 in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance.
  • the sub woofer speaker for a channel for reproduction of only a low-frequency sound as a sub woofer channel may be provided in the vicinity of the center speaker,
  • the simulated arrangement of the speakers for reproduction in front five channels and rear two channels is made as shown in FIG. 26.
  • a virtual image position 261 on which the picture is projected is disposed in front of seats 260 where the listener 23 stays.
  • the audio signal is reproduced such that the reproduced sound image is localized as if a center speaker C was located at the center of the virtual image position 261 in front of the seats 260, a left speaker L and a right speaker R were respectively located on the left and right sides of the virtual image position 261, a left extra speaker L E was located between the center speaker C and the left speaker L, a right extra speaker R E was located between the center speaker C and the right speaker R, left surround speakers S L were located at the left rear side of the seats 260, and right surround speakers S R were located at the right rear side of the seats 260.
  • a sub woofer speaker W for a channel for reproduction of only a low-frequency reproduced sound may be located in the vicinity of the center speaker C.
  • the convolutional integrators 5, 7, 9 and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 as the control means are reproduced by the headphone 24 as the audio reproducing means such that the seven-channel reproduced sound images are localized at the center front, left and right front of, at the left and right center front of, and at the right and left rear of the listener 23 in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance when the virtual image display 193 as the video reproducing means reproduces the video signal, it is possible to localize the seven-channel reproduced sound images at the center front, left and right front of, at the left and right center front of, and at the right and left rear of the listener 23 in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance.
  • the virtual image positions 192, 211, 221, 231, 241, 251 and 261 may have a plurality of screen size such as a cinema scope size, a vista size or the like by adjusting the lenses 182 and 183 as an optical system, for example, and the reproduced sound image may be localized in accordance with the screen size thereof.
  • the switching signal generated in accordance with the screen size is input to the input terminal 65.
  • the virtual image display 193 is supplied with information on the head movement of the listener 23 with respect to the reference direction from the digital angle detector 28 or the analog angle detector 38. Accordingly, when the listener 23 turns the head, the listener 23 may watch that the image projected on each of the virtual image positions 211, 221, 231, 241, 251 and 261 shown in FIGS. 21 to 26 distant from the left and right eyes of the listener by a predetermined distance is continuously changed based on the information as if the listener was watching the image with changing the angle of his head.
  • each of the virtual image positions 192, 211, 221, 231, 241, 251 and 261 has a plurality of screen sizes and the audio signals corrected by the memories and convolutional integrators 5, 6, 7, 8, 9, 10, 11 and 12 and the control apparatus 50, 51, 52, 53, 54 and 56 are reproduced by the headphone 24 such that a plurality of reproduced sound images are localized in accordance with the screen size in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance by the virtual image display 193, it is possible to localize the plurality of reproduced sound images in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance even if the screen sizes of the virtual image positions 192, 211, 221, 231, 241, 251 and 261 are changed.
  • the reproduced sound images may be respectively localized so as to correspond to the pictures on the screens at the virtual image positions 192, 211, 221, 231, 241, 251 and 261.
  • the reproduced sound images may be respectively localized so as to correspond to the pictures on the screens at the virtual image positions 192, 211, 221, 231, 241, 251 and 261.
  • the above arrangement is most suitable for use in localizing the reproduced sound images corresponding the movement of the pictures in the direction in which the picture is moved.
  • the virtual image displays 193 as the video reproduction means are disposed at least in front of and behind of the listener 23 and on the left and right sides of the listener so as to cover the listener 23 and the audio signals corrected by the memories 6, 8, 10 and 12, the convolutional integrators 5, 7, 9, and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 as the control means are reproduced by the headphone 24 as the audio reproducing means such that a plurality of optional reproduced sound images are localized in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance when the virtual image display 193 as the video reproduction means reproduces the video signal, it is possible to localize the plurality of reproduced sound images in front of and behind of the listener and on the left and right sides of the listener in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance.
  • the virtual image displays 193 as the video reproduction means are disposed at least in front of and behind of the listener 23, on the left and right sides of the listener and above and below the listener so as to cover the listener 23 and the headphone 24 as the audio reproducing means reproduces the audio signals corrected by the memories 6, 8, 10 and 12, the convolutional integrators 5, 7, 9, and 11 and the control apparatus 50, 51, 52, 53, 54 and 56 as the control means such that a plurality of optional reproduced sound images are localized in the direction corresponding to the reproduced picture projected on the position distant from the left and right eyes of the listener 23 by a predetermined distance when the virtual image display 193 as the video reproduction means reproduces the video signal, it is possible to localize the plurality of reproduced sound images in front of and behind of the listener, on the left and right sides of
  • the video signal and audio signal reproducing apparatus is most suitable for use in localizing the reproduced sound images corresponding the movement of the pictures in the direction in which the picture is moved, based on the reproduced sound image position information indicative of both or either of the positions of the microphones and the sound sources used when the sounds are picked up.
  • the video signal and audio signal reproducing apparatus can also be used to localize the reproduced sound images in a bidirectional fashion.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Signal Processing (AREA)
  • Stereophonic System (AREA)
EP95907878A 1994-02-14 1995-02-14 Vorrichtung zur wiedergabe von video- und audiosignalen Expired - Lifetime EP0695109B1 (de)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
JP17601/94 1994-02-14
JP1760294 1994-02-14
JP17602/94 1994-02-14
JP1760194 1994-02-14
JP3497594 1994-03-04
JP34975/94 1994-03-04
JP3725494 1994-03-08
JP37254/94 1994-03-08
PCT/JP1995/000197 WO1995022235A1 (fr) 1994-02-14 1995-02-14 Systeme de reproduction visuelle et sonore

Publications (3)

Publication Number Publication Date
EP0695109A1 true EP0695109A1 (de) 1996-01-31
EP0695109A4 EP0695109A4 (de) 2004-12-22
EP0695109B1 EP0695109B1 (de) 2011-07-27

Family

ID=27456807

Family Applications (1)

Application Number Title Priority Date Filing Date
EP95907878A Expired - Lifetime EP0695109B1 (de) 1994-02-14 1995-02-14 Vorrichtung zur wiedergabe von video- und audiosignalen

Country Status (4)

Country Link
US (1) US5796843A (de)
EP (1) EP0695109B1 (de)
JP (1) JP3687099B2 (de)
WO (1) WO1995022235A1 (de)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2351425A (en) * 1999-01-20 2000-12-27 Canon Kk Video conferencing apparatus
EP1675432A1 (de) * 2004-12-27 2006-06-28 Siemens Aktiengesellschaft Mobiles Kommunikationsendgerät zum Erzeugen einer ortsfesten virtuellen Schallquelle

Families Citing this family (52)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH1051888A (ja) * 1996-05-28 1998-02-20 Sony Corp スピーカ装置および音声再生システム
JPH114966A (ja) * 1996-10-01 1999-01-12 Sony Computer Entateimento:Kk ゲーム機用操作装置及びゲーム装置
KR0185021B1 (ko) * 1996-11-20 1999-04-15 한국전기통신공사 다채널 음향시스템의 자동 조절장치 및 그 방법
US7085387B1 (en) * 1996-11-20 2006-08-01 Metcalf Randall B Sound system and method for capturing and reproducing sounds originating from a plurality of sound sources
JPH10211358A (ja) * 1997-01-28 1998-08-11 Sega Enterp Ltd ゲーム装置
JPH11220797A (ja) * 1998-02-03 1999-08-10 Sony Corp ヘッドホン装置
US6239348B1 (en) * 1999-09-10 2001-05-29 Randall B. Metcalf Sound system and method for creating a sound event based on a modeled sound field
JP4737804B2 (ja) * 2000-07-25 2011-08-03 ソニー株式会社 音声信号処理装置及び信号処理装置
JP4679699B2 (ja) * 2000-08-01 2011-04-27 ソニー株式会社 音声信号処理方法及び音声信号処理装置
JP4304845B2 (ja) * 2000-08-03 2009-07-29 ソニー株式会社 音声信号処理方法及び音声信号処理装置
EP1194006A3 (de) * 2000-09-26 2007-04-25 Matsushita Electric Industrial Co., Ltd. Signalverarbeitungsgerät und Aufnahmemedium
JP2002171460A (ja) * 2000-11-30 2002-06-14 Sony Corp 再生装置
US7783054B2 (en) * 2000-12-22 2010-08-24 Harman Becker Automotive Systems Gmbh System for auralizing a loudspeaker in a monitoring room for any type of input signals
JP2002199500A (ja) * 2000-12-25 2002-07-12 Sony Corp 仮想音像定位処理装置、仮想音像定位処理方法および記録媒体
AUPR333001A0 (en) * 2001-02-23 2001-03-22 Lake Technology Limited Sonic terrain and audio communicator
WO2004032351A1 (en) * 2002-09-30 2004-04-15 Electro Products Inc System and method for integral transference of acoustical events
US20070009120A1 (en) * 2002-10-18 2007-01-11 Algazi V R Dynamic binaural sound capture and reproduction in focused or frontal applications
US7333622B2 (en) * 2002-10-18 2008-02-19 The Regents Of The University Of California Dynamic binaural sound capture and reproduction
US20040091120A1 (en) * 2002-11-12 2004-05-13 Kantor Kenneth L. Method and apparatus for improving corrective audio equalization
JP2006086921A (ja) * 2004-09-17 2006-03-30 Sony Corp オーディオ信号の再生方法およびその再生装置
US7636448B2 (en) * 2004-10-28 2009-12-22 Verax Technologies, Inc. System and method for generating sound events
CA2598575A1 (en) * 2005-02-22 2006-08-31 Verax Technologies Inc. System and method for formatting multimode sound content and metadata
US7612793B2 (en) * 2005-09-07 2009-11-03 Polycom, Inc. Spatially correlated audio in multipoint videoconferencing
JP5067595B2 (ja) * 2005-10-17 2012-11-07 ソニー株式会社 画像表示装置および方法、並びにプログラム
TW200717402A (en) * 2005-10-31 2007-05-01 Asustek Comp Inc Monitor with prompting sound effects
EP2005793A2 (de) * 2006-04-04 2008-12-24 Aalborg Universitet Binauraltechnologieverfahren mit positionsverfolgung
GB2437399B (en) * 2006-04-19 2008-07-16 Big Bean Audio Ltd Processing audio input signals
US7792674B2 (en) * 2007-03-30 2010-09-07 Smith Micro Software, Inc. System and method for providing virtual spatial sound with an audio visual player
KR100947027B1 (ko) * 2007-12-28 2010-03-11 한국과학기술원 가상음장을 이용한 다자간 동시 통화 방법 및 그 기록매체
JP4849121B2 (ja) * 2008-12-16 2012-01-11 ソニー株式会社 情報処理システムおよび情報処理方法
US20100223552A1 (en) * 2009-03-02 2010-09-02 Metcalf Randall B Playback Device For Generating Sound Events
JP2011101229A (ja) * 2009-11-06 2011-05-19 Sony Corp 表示制御装置、表示制御方法、プログラム、出力装置、および送信装置
US10158958B2 (en) * 2010-03-23 2018-12-18 Dolby Laboratories Licensing Corporation Techniques for localized perceptual audio
CN116471533A (zh) * 2010-03-23 2023-07-21 杜比实验室特许公司 音频再现方法和声音再现系统
JP5672741B2 (ja) * 2010-03-31 2015-02-18 ソニー株式会社 信号処理装置および方法、並びにプログラム
KR101652401B1 (ko) * 2010-09-07 2016-08-31 삼성전자주식회사 입체 영상 디스플레이 장치 및 입체 영상 표시 방법
TWI453451B (zh) * 2011-06-15 2014-09-21 Dolby Lab Licensing Corp 擷取與播放源於多音源的聲音之方法
US8183997B1 (en) * 2011-11-14 2012-05-22 Google Inc. Displaying sound indications on a wearable computing system
US10448161B2 (en) 2012-04-02 2019-10-15 Qualcomm Incorporated Systems, methods, apparatus, and computer-readable media for gestural manipulation of a sound field
CN117253498A (zh) 2013-04-05 2023-12-19 杜比国际公司 音频信号的解码方法和解码器、介质以及编码方法
KR101799294B1 (ko) * 2013-05-10 2017-11-20 삼성전자주식회사 디스플레이 장치 및 이의 제어 방법
US9813837B2 (en) 2013-11-14 2017-11-07 Dolby Laboratories Licensing Corporation Screen-relative rendering of audio and encoding and decoding of audio for such rendering
CN109996166B (zh) * 2014-01-16 2021-03-23 索尼公司 声音处理装置和方法、以及程序
US9226090B1 (en) * 2014-06-23 2015-12-29 Glen A. Norris Sound localization for an electronic call
US10085107B2 (en) * 2015-03-04 2018-09-25 Sharp Kabushiki Kaisha Sound signal reproduction device, sound signal reproduction method, program, and recording medium
CN105260016B (zh) * 2015-09-24 2018-06-01 联想(北京)有限公司 一种信息处理方法及电子设备
CN105407443B (zh) * 2015-10-29 2018-02-13 小米科技有限责任公司 录音方法及装置
EP3322200A1 (de) * 2016-11-10 2018-05-16 Nokia Technologies OY Audiowiedergabe in echtzeit
CN110572760B (zh) * 2019-09-05 2021-04-02 Oppo广东移动通信有限公司 电子设备及其控制方法
WO2021117576A1 (ja) * 2019-12-13 2021-06-17 ソニーグループ株式会社 信号処理装置、信号処理方法およびプログラム
CN114422935B (zh) * 2022-03-16 2022-09-23 荣耀终端有限公司 音频处理方法、终端及计算机可读存储介质
US20240259760A1 (en) * 2023-01-13 2024-08-01 Sonos, Inc. Room-Informed Binaural Rendering

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4952024A (en) * 1986-08-29 1990-08-28 Gale Thomas S Three-dimensional sight and sound reproduction apparatus for individual use
EP0438281A2 (de) * 1990-01-19 1991-07-24 Sony Corporation Schallsignalwiedergabegerät
EP0479605A2 (de) * 1990-10-05 1992-04-08 Texas Instruments Incorporated Verfahren und Gerät zur Erzeugung einer tragbaren optischen Anzeige

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPS5944198A (ja) * 1982-09-06 1984-03-12 Matsushita Electric Ind Co Ltd ヘツドホン装置
US4817149A (en) * 1987-01-22 1989-03-28 American Natural Sound Company Three-dimensional auditory display apparatus and method utilizing enhanced bionic emulation of human binaural sound localization
JP2751166B2 (ja) * 1987-10-26 1998-05-18 ソニー株式会社 ヘッドホン装置
US5495534A (en) * 1990-01-19 1996-02-27 Sony Corporation Audio signal reproducing apparatus
JP2605450B2 (ja) * 1990-05-08 1997-04-30 ヤマハ株式会社 音像定位装置
US5469511A (en) * 1990-10-05 1995-11-21 Texas Instruments Incorporated Method and apparatus for presentation of on-line directional sound
JPH04192066A (ja) * 1990-11-27 1992-07-10 Matsushita Electric Works Ltd 商品疑似体験ショウルームシステム
JP3003322B2 (ja) * 1991-09-30 2000-01-24 ソニー株式会社 映像表示装置
JPH05115099A (ja) * 1991-10-22 1993-05-07 Nippon Telegr & Teleph Corp <Ntt> 頭外定位ヘツドホン受聴装置
JPH05168097A (ja) * 1991-12-16 1993-07-02 Nippon Telegr & Teleph Corp <Ntt> 頭外音像定位ステレオ受聴器受聴方法
JP2741817B2 (ja) * 1992-03-06 1998-04-22 日本電信電話株式会社 頭外定位ヘッドホン受聴装置
US5495576A (en) * 1993-01-11 1996-02-27 Ritchey; Kurtis J. Panoramic image based virtual reality/telepresence audio-visual system and method
US5526183A (en) * 1993-11-29 1996-06-11 Hughes Electronics Helmet visor display employing reflective, refractive and diffractive optical elements

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4952024A (en) * 1986-08-29 1990-08-28 Gale Thomas S Three-dimensional sight and sound reproduction apparatus for individual use
EP0438281A2 (de) * 1990-01-19 1991-07-24 Sony Corporation Schallsignalwiedergabegerät
EP0479605A2 (de) * 1990-10-05 1992-04-08 Texas Instruments Incorporated Verfahren und Gerät zur Erzeugung einer tragbaren optischen Anzeige

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of WO9522235A1 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB2351425A (en) * 1999-01-20 2000-12-27 Canon Kk Video conferencing apparatus
EP1675432A1 (de) * 2004-12-27 2006-06-28 Siemens Aktiengesellschaft Mobiles Kommunikationsendgerät zum Erzeugen einer ortsfesten virtuellen Schallquelle
WO2006069825A1 (de) * 2004-12-27 2006-07-06 Siemens Aktiengesellschaft Mobiles kommunikationsendgerät zum erzeugen einer ortsfesten virtuellen schallquelle

Also Published As

Publication number Publication date
EP0695109A4 (de) 2004-12-22
US5796843A (en) 1998-08-18
EP0695109B1 (de) 2011-07-27
JP3687099B2 (ja) 2005-08-24
WO1995022235A1 (fr) 1995-08-17

Similar Documents

Publication Publication Date Title
EP0695109B1 (de) Vorrichtung zur wiedergabe von video- und audiosignalen
JP3796776B2 (ja) 映像音声再生装置
US5696831A (en) Audio reproducing apparatus corresponding to picture
EP0674467A1 (de) Audiowiedergabeeinrichtung
KR100878457B1 (ko) 음상정위 장치
US5272757A (en) Multi-dimensional reproduction system
JP4735993B2 (ja) 音声処理装置、音像定位位置調整方法、映像処理装置及び映像処理方法
US7333622B2 (en) Dynamic binaural sound capture and reproduction
US5761314A (en) Audio reproducing apparatus and headphone
JP3577798B2 (ja) ヘッドホン装置
US5526429A (en) Headphone apparatus having means for detecting gyration of user&#39;s head
JP4744695B2 (ja) 仮想音源装置
JP5754595B2 (ja) トランスオーラルシステム
US20130243201A1 (en) Efficient control of sound field rotation in binaural spatial sound
JP2671329B2 (ja) オーディオ再生装置
US20230403528A1 (en) A method and system for real-time implementation of time-varying head-related transfer functions
EP0549836B1 (de) Multidimensionales Tonwiedergabesystem
JPH08140200A (ja) 立体音像制御装置
JP3573797B2 (ja) ステレオ音声再生装置
JP6274244B2 (ja) 収音再生装置、収音再生プログラム、収音装置及び再生装置
JPH06282285A (ja) 立体音声再生装置
JP2024152931A (ja) 音響処理装置、音響処理方法、及び音響処理プログラム
Pellieux et al. STEREOPHONIC PRESENTATION OF INFORMATION: SPACE SIMULATION UNDER HEADPHONES

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 19950907

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): DE FR GB

A4 Supplementary search report drawn up and despatched

Effective date: 20041110

RIC1 Information provided on ipc code assigned before grant

Ipc: 7H 04S 3/00 B

Ipc: 7H 04S 1/00 A

17Q First examination report despatched

Effective date: 20070709

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 69536180

Country of ref document: DE

Effective date: 20110922

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20120502

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 69536180

Country of ref document: DE

Effective date: 20120502

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20140219

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20140219

Year of fee payment: 20

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20140218

Year of fee payment: 20

REG Reference to a national code

Ref country code: DE

Ref legal event code: R071

Ref document number: 69536180

Country of ref document: DE

REG Reference to a national code

Ref country code: GB

Ref legal event code: PE20

Expiry date: 20150213

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF EXPIRATION OF PROTECTION

Effective date: 20150213