JP2008236397A - Acoustic control system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an acoustic control system capable of performing exact acoustic control in accordance with the position of a person by accurately detecting the position and movement of the person listening to sound. <P>SOLUTION: When a sound reproduction start instruction is inputted, a sound reproducing device 20 starts to reproduce sound, and an acoustic control processing part 18 starts acoustic control of the sound reproducing device 20. When sound reproduction is started, a camera 12 photographs a person in a vehicle. A face detection processing part 14 detects a face area of the person from an image photographed by the camera 12. A face position operating part 16 operates the position of the face of the person detected by the face detection processing part 14. An acoustic control processing part 18 adjusts sound outputting timing and performs acoustic control (for example, equalization) on the basis of the position of the face operated by the face position operating part 16. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an acoustic adjustment system, and more particularly to an acoustic adjustment system that adjusts the sound of a sound reproducing device including a plurality of speakers.

Japanese Patent Application Laid-Open No. 2004-151867 describes an automatic on-vehicle acoustic device that detects time differences among detection signals from a plurality of microphones that detect a person's voice in a vehicle, calculates the position of the person, and performs acoustic adjustment according to the position of the person. An adjustment device is disclosed.
JP-A-11-268593

  In Patent Document 1, the position of a person is calculated from sounds detected by a plurality of microphones. However, when noise from the outside of the vehicle, sound being reproduced in the vehicle, or the like is detected by the microphone, the position of the person is accurately determined. There was a problem that it was not detected. In addition, there is a problem that the position of a person cannot be detected unless a voice is produced when the body is moved during or upon boarding.

  The present invention has been made in view of such circumstances, and it is possible to accurately detect the position and movement of a listener who listens to sound, and to perform accurate acoustic adjustment according to the position of the listener. The purpose is to provide a system.

  In order to solve the above problems, an acoustic adjustment system according to the first aspect of the present invention includes an audio output unit that reproduces and outputs audio, an imaging unit that captures an image of a listener of the audio output by the audio output unit, Face detecting means for detecting a listener's face from an image taken by the photographing means, face position calculating means for calculating the position of the listener's face detected by the face detecting means, and position of the listener's face And a sound adjusting means for adjusting the sound of the sound output from the sound output means.

  According to the first aspect of the present invention, the position of the listener's face can be accurately detected by photographing the image of the listener and performing the face detection process, so that it is accurate according to the position of the listener's face. Acoustic adjustment can be performed.

  Invention 2 of the present application further comprises ear detection means for detecting an ear of the listener from the face image detected by the face detection means in the sound adjustment system of Invention 1 of the present application, and the face position calculation means includes the listening position The position of the user's ear is calculated, and the sound adjustment means adjusts the sound of the sound output from the sound output means according to the position of the listener's ear.

  According to the present invention 2, accurate acoustic adjustment can be performed according to the position of the listener's face and ear.

  Invention 3 of the present application further comprises first detection means for detecting whether or not the listener is facing the front of the seating seat in the sound adjustment system of Invention 1 or 2, wherein the sound adjustment means comprises: When it is detected that the listener has turned to the back or side of the seat, the volume of the sound output from the sound output means is reduced or the sound reproduction is stopped.

  Invention 4 of the present application is the acoustic adjustment system according to Inventions 1 to 3 of the present application, in which, when the listener is seated separately in the front and rear seats, whether or not the listener in the back seat has entered the front seat side. A second detecting means for detecting the sound; and the sound adjusting means detects the sound output from the sound output means when it is detected that the listener in the rear seat has entered the front seat side. The volume is lowered or the reproduction of the sound is stopped.

  Invention 5 of the present application further comprises third detection means for detecting whether or not the listener detected by the face detection means has closed eyes for a predetermined time or more in the sound adjustment system of Inventions 1 to 4 of the present invention, The adjusting means lowers the volume of the sound output from the sound output means or stops the sound reproduction when it is detected that the listener has closed his eyes for a predetermined time or more. .

  Invention 6 of the present application is the sound adjustment system of Invention 5 of the present application, wherein the sound adjustment system is installed in an automobile, and when it is detected that the listener has closed eyes for a predetermined time or more, It further comprises fourth detection means for detecting whether or not the engine is running, and the acoustic adjustment means detects that the listener has closed his eyes for a predetermined time or more and the car engine is running. In this case, the volume of the sound output from the sound output means is increased.

  According to the present inventions 3 to 6, accurate acoustic adjustment can be performed according to the state of the listener (for example, when the listener in the vehicle is talking or sleeping).

  According to the present invention, it is possible to accurately detect the position of the listener's face and ear by photographing the image of the listener and performing face detection processing. And accurate acoustic adjustment.

  Hereinafter, preferred embodiments of an acoustic adjustment system according to the present invention will be described with reference to the accompanying drawings.

  FIG. 1 is a block diagram showing the main configuration of an acoustic adjustment system according to an embodiment of the present invention. The acoustic adjustment system 10 of the present embodiment includes a camera 12, a face detection processing unit 14, a face position calculation unit 16, an acoustic adjustment processing unit 18, an audio reproduction device 20, a speaker system 22, an operation unit 24, and a state detection unit 26. ing. The acoustic adjustment system 10 according to the present embodiment performs acoustic adjustment of a car audio system installed in a car. The camera 12 and the speaker system 22 are arranged in a boarding space inside the automobile.

  The audio reproducing device 20 receives, for example, a music player supplied to a speaker 22 that reads an audio signal from a recording medium such as a cassette tape, a compact disc (registered trademark), a mini disc (registered trademark), or a radio wave and receives an audio signal A radio receiver that converts the signal into the speaker 22 and supplies it to the speaker 22.

  The speaker system 22 includes a plurality of speakers arranged at predetermined positions in the vehicle, and converts the audio signal supplied from the audio reproduction device 20 into sound and outputs the sound.

  The operation unit 24 is a means for performing operation inputs such as an audio playback start instruction, a playback end instruction, a radio reception frequency adjustment, a volume adjustment, and the like to the audio playback device 20.

  When a sound reproduction start instruction is input, sound reproduction by the sound reproduction device 20 is started and acoustic adjustment of the sound reproduction device 20 is started by the sound adjustment processing unit 18.

  The camera 12 captures an image of a person (listener) in the car 100 when the sound reproduction is started. The face detection processing unit 14 detects the listener's face area from the image taken by the camera 12. The face position calculation unit 16 calculates the position of the listener's face detected by the face detection processing unit 14. The sound adjustment processing unit 18 performs adjustment of sound output timing and sound adjustment (for example, equalization) based on the face position obtained by the face position calculation unit 16.

  The detection of the face and the position of the face is repeated at predetermined time intervals while the sound reproduction by the sound reproduction device 20 continues. The state detection unit 26 stores the face state detected by the face detection processing unit 14 and the face position detected by the face position calculation unit 16. Then, the state detection unit 26 detects the listener's state based on the latest face state and face position and the previously stored face state and face position, and transmits them to the sound adjustment processing unit 18. The sound adjustment processing unit 18 performs sound adjustment according to the state of the listener. For example, when a listener seated in the rear seat 30 starts in front (for example, when it is detected that the face area of the listener in the rear seat 30 has increased), or the front seat 28 or the rear seat When it is detected that the listener seated at 30 is facing the back or side of the automobile 100, the sound adjustment processing unit 18 decreases the volume or stops the reproduction of the sound. The sound adjustment processing unit 18 lowers the volume or stops the sound reproduction when any of the detected listeners closes their eyes for a predetermined time or longer (that is, when the listener is asleep). To do. In addition, the state detection unit 26 detects whether the engine of the automobile 100 is running when the listener at the driver's seat closes his eyes for a predetermined time or longer (that is, when the driver is sleeping). Then, the sound adjustment processing unit 18 increases the sound volume when the listener at the driver's seat has closed his eyes for a predetermined time or more and the engine of the automobile 100 is running. Thereby, appropriate acoustic adjustment can be performed according to the state of the listener in the vehicle.

  Next, a method for controlling sound based on the position of the listener's face will be specifically described. FIG. 2 is a plan view of the automobile showing the first embodiment of the camera arrangement in the vehicle. As shown in FIG. 2, a plurality of speakers 220R to 226R, 220L to 226L, front seats 28R and 28L, and a rear seat 30 are arranged in the automobile 100.

  As shown in FIG. 2, cameras 120 </ b> R and 120 </ b> L are arranged in the front portion of the automobile 100. In addition, cameras 122R and 122L are arranged at the rear of the front seats 28R and 28L, respectively. The cameras 120R and 120L photograph the listener seated in the front seats 28R and 28L, and the cameras 122R and 122L photograph the listener seated in the rear seat 30. The face detection processing unit 14 detects the face area of the listener seated on the front seats 28R and 28L from images taken by the cameras 120R and 120L. Further, the face detection processing unit 14 detects the position of the ear from the detected face area.

  Here, as a face detection processing method, for example, there is a method in which a pixel having a color close to a color specified in advance as a skin color is extracted from an original image, and the extracted region is detected as a face region. In this face detection processing method, for example, in a color space for distinguishing skin color from other colors, a skin color range on the color space is determined from pre-sampled skin color information, and a range in which the color of each pixel is determined. This is done by determining whether or not it has entered. In addition, as a method of ear detection processing, for example, a region (eye region) including two black colors arranged on the left and right is detected from the skin color regions detected by the face detection processing, and thus the detected eye Some detect the vicinity of the edge of the skin color area as an ear area with reference to the position of the area.

  The face position calculation unit 16 performs triangulation on the position of the ear region in the images photographed by the cameras 120R and 120L, and determines the positions of the left and right ears of the listener sitting on the front seats 28R and 28L. Coordinates (X, Y, Z) are calculated.

  Similarly, the face detection processing unit 14 detects the ear region of the listener seated on the rear seat 30 from images taken by the cameras 122R and 122L. The face position calculation unit 16 performs triangulation on the area of the listener's ear in the images taken by the cameras 122R and 122L, and coordinates of the positions of the left and right ears of the listener seated in the rear seat 30 ( X, Y, Z) is calculated.

  The sound adjustment processing unit 18 simultaneously outputs the sound output from the left speaker 220L,... 226L to the listener's left ear and the sound output from the right speaker 220R,. The sound output timing and the volume are adjusted so as to reach the target. Thereby, the listener in a vehicle can listen to the same sound as being at an equal distance from each speaker.

  Note that it may be possible to select which listener among the detected listeners to optimize the sound. In this case, for example, an operating means for selecting which seat of the front seat (driver seat, front passenger seat) and rear seat to optimize the sound may be provided. In addition, display means for displaying the image of the listener photographed by the camera 12 and operation means for selecting which listener in the image to optimize the sound may be provided.

  FIG. 3 is a plan view of an automobile showing a second embodiment of the camera arrangement in the car. In the example shown in FIG. 3, the camera 124 is arranged at the front part (for example, a room mirror) of the automobile 100, and the camera 126 is arranged near the center of the canopy of the automobile 100 (for example, near the room light). ing. The camera 124 photographs the listener who is seated in the front seats 28R and 28L and the rear seat 30. The camera 126 is a camera capable of wide-angle photographing including a wide-angle lens or a fish-eye lens, and photographs the listener's head in the vehicle from above.

  The face detection processing unit 14 detects the face area and the ear area of the listener seated in the front seats 28R and 28L and the rear seat 30 of the listener from the images taken by the camera 124. The face and ear detection methods are the same as described above.

  The face position calculation unit 16 detects the listener's head from the image captured by the camera 126 and calculates the coordinates (X, Y). Then, the face position calculation unit 16 uses the coordinates (X, Y) of the listener's head and the position of the ear of the listener in the vehicle (from the position of the ear region determined by the face detection processing unit 14). X, Y, Z) is calculated. As a result, it is possible to adjust the sound in accordance with the position of the listener's ear in the vehicle.

  FIG. 4 is a plan view of an automobile showing a third embodiment of the arrangement of cameras in the car. In the example illustrated in FIG. 4, the camera 130 is disposed in the front part (for example, a room mirror) of the automobile 100. Cameras 132R, 134R, 132L, and 134L are arranged near the left and right canopies of the automobile 100. The camera 130 photographs a listener who is seated in the front seats 28R and 28L and the rear seat 30. The cameras 132R and 134R capture a listener who is seated on the right side of the front seat 28R and the rear seat 30, respectively. The cameras 132L and 134L photograph the listener seated on the left side of the front seat 28L and the rear seat 30, respectively.

  The face detection processing unit 14 detects the face area and the ear area of the listener seated in the front seats 28R and 28L and the rear seat 30 of the listener from the images taken by the camera 130. The face and ear detection methods are the same as described above.

  The face position calculation unit 16 detects the listener's head from images taken by the cameras 132R, 134R, 132L, and 134L, and calculates the coordinates (X, Y, Z). Then, the face position calculation unit 16 calculates the position of the listener's ear in the vehicle from the coordinates (X, Y, Z) of the listener's head and the position of the ear region obtained by the face detection processing unit 14. Coordinates (X, Y, Z) are calculated. As a result, it is possible to adjust the sound in accordance with the position of the listener's ear in the vehicle.

  In the example shown in FIG. 4, two pairs of cameras for detecting the position of the listener's head are arranged on the left and right, but one pair may be used. The cameras 126, 130R and 130L, 132R and 132L for detecting the position of the listener's head may be, for example, an infrared camera. Further, a distance measuring sensor or an infrared sensor may be used instead of the camera for detecting the position of the listener's head.

  Further, the camera 130 may be configured as a camera (for example, a stereo camera) having a function of measuring the distance to the face, without a camera for detecting the position of the listener's head.

  FIG. 5 is a flowchart showing the flow of the sound adjustment process. First, when sound reproduction is started (step S10), photographing of a listener inside the vehicle is started by the camera 12 arranged in the vehicle, and detection processing of a face region and an ear region is performed on the captured image. (Step S12). Next, the coordinates (X, Y, Z) of the position of the ear region are calculated and recorded (step S14). Then, an acoustic adjustment process is performed according to the position of the ear (step S16).

  And the process of said step S12 to step S16 is repeated for every predetermined time. In step S <b> 16, for example, when a listener seated in the rear seat 30 starts in front (for example, when it is detected that the face area has increased), or seats in the front seat 28 or the rear seat 30. When it is detected that the listener who is listening is facing the back or side of the automobile 100, the sound volume is reduced or the sound reproduction is stopped. Also, if any of the detected listeners has closed their eyes for a predetermined time or longer (that is, if the listener is asleep), the volume is reduced or the sound reproduction is stopped. Further, the volume is increased when the listener of the driver's seat closes his eyes for a predetermined time or longer with the engine of the automobile running (that is, when the driver is sleeping). Thereby, an accurate acoustic adjustment can be performed according to a case where a listener in the vehicle is talking or asleep.

  According to the present embodiment, the position of the listener's face and ear can be accurately detected by photographing the listener's image and performing face detection processing. Accordingly, accurate acoustic adjustment can be performed.

  In the above description, the example in which the sound adjustment system of the present invention is applied to the sound adjustment of the car audio system has been described. However, for example, the sound adjustment system can also be applied to the sound adjustment of the speaker system of a home theater.

The block diagram which shows the main structures of the acoustic adjustment system which concerns on one Embodiment of this invention. The top view of the motor vehicle which shows 1st Embodiment of arrangement | positioning of the camera in a vehicle The top view of the motor vehicle which shows 2nd Embodiment of arrangement | positioning of the camera in a vehicle The top view of the motor vehicle which shows 3rd Embodiment of arrangement | positioning of the camera in a vehicle Flow chart showing the flow of acoustic adjustment processing

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Acoustic adjustment system, 12 ... Camera, 14 ... Face detection process part, 16 ... Face position calculation part, 18 ... Sound adjustment process part, 20 ... Sound reproduction apparatus, 22 ... Speaker system, 24 ... Operation part, 26 ... State Detection unit

Claims (6)

Audio output means for reproducing and outputting audio;
Photographing means for photographing an image of a listener of the sound output by the sound output means;
Face detection means for detecting the face of the listener from the image photographed by the photographing means;
Face position calculating means for calculating the position of the listener's face detected by the face detecting means;
Sound adjusting means for adjusting the sound of the sound output from the sound output means according to the position of the listener's face;
An acoustic adjustment system comprising: Ear detection means for detecting the listener's ear from the face image detected by the face detection means;
The face position calculating means calculates the position of the listener's ear,
The acoustic adjustment system according to claim 1, wherein the acoustic adjustment unit performs acoustic adjustment of a sound output from the sound output unit according to a position of an ear of the listener. First detecting means for detecting whether the listener is facing the front of the seated seat,
The sound adjusting means may reduce the volume of the sound output from the sound output means or stop the sound reproduction when it is detected that the listener is facing the back or side of the seat. The acoustic adjustment system according to claim 1 or 2, characterized in that When the listener is seated separately in the front and rear seats, further comprising second detection means for detecting whether the listener in the back seat has entered the front seat side,
The sound adjusting means lowers the volume of the sound output from the sound output means or stops the sound reproduction when it is detected that the listener in the back seat has entered the front seat side. The acoustic adjustment system according to any one of claims 1 to 3, wherein Further comprising third detection means for detecting whether or not the listener detected by the face detection means has closed their eyes for a predetermined time or more;
The sound adjustment means lowers the volume of the sound output from the sound output means or stops the sound reproduction when it is detected that the listener has closed his eyes for a predetermined time or more. The acoustic adjustment system according to any one of claims 1 to 4. The acoustic adjustment system is installed in an automobile,
When it is detected that the listener has closed his eyes for a predetermined time or more, further comprising fourth detection means for detecting whether the engine of the automobile is running,
The sound adjustment means increases the volume of the sound output from the sound output means when it is detected that the listener has closed his eyes for a predetermined time or more and the automobile engine is running. The acoustic adjustment system according to claim 5.

Images