JP2011166407A - Acoustic setting device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an acoustic setting device for setting an audio output condition concerning a listening sound output state in response to change in the correlation of a still object with respect to speakers, while detecting the still object arranged in a listening space. <P>SOLUTION: The acoustic system 100 sets the audio output condition related to the output state toward the listening space SP with a plurality of speakers 101 arranged therein concerning the listening sound to be listened by a listener, which is output from the speakers 101. The system includes: an audio output unit for driving the speakers 101, on the basis of an audio signal; a listening space environment detector for detecting the still object existing in the listening space SP; and an audio output condition setter for allowing the audio output condition setting to be the setting corresponding to the change when the listening space environment detector detects the change in the correlation of the in-space existence object with respect to the speakers 101. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a sound setting device and a sound setting method for setting an audio output condition for a listening sound to be heard by a listener output from a speaker such as an audio device to a listening space.

  In recent years, various technologies related to an acoustic system that arranges a plurality of speakers and expresses a realistic sound field have been proposed. In such an acoustic system, an acoustic setting device that realizes a sound field by setting audio output conditions related to listening sound such as speaker position, sound filter processing, sound pressure and output timing according to the listening position of the specified listener It has.

  There is also known a technique in which a camera is installed in an acoustic system and a skin color component is detected and an audio output condition is set according to the position when a face enters a captured frame (see, for example, Patent Document 1). ).

However, in the above-described conventional sound setting device, first, it is necessary to set an audio output condition regarding the output state of the listening sound with respect to the listening space according to the environment of the listening space before use. For example, in the 5.1ch surround system, it is necessary for the listener himself / herself to measure the distances of a plurality of speakers such as the front, rear, and subwoofer with respect to the listener position and to set the audio output condition.
In recent years, instead of measuring by a listener, a sound collecting device such as a headset is installed in advance at the listener position to check the listening space environment in the listening space. In that case, because it must be in an environment that is not affected by the listener or obstacles, it will be troublesome for the listener, and it is necessary to check each sound range at the time of sound collection, There is a problem that setting time is required.

Above all, when a change occurs in the environment in the set listening space, the listener needs to set the position of each speaker again according to the change. For example, when a change in the listening space environment occurs such that the chair placed in the listening space is replaced with a different one and the installation position is changed, the listener's position and the degree of sound reflection change, It is necessary to reset the audio output conditions regarding the output state of the listening position of each speaker and other listening sound to the listening space.
In addition, since the technique described in Patent Document 1 sets the audio output condition according to the skin color component only when a face enters, it can cope with the change in the size and position of the stationary object as described above. In addition, when the skin color component (face) could not be detected even though there was a listener, it was not possible to set an appropriate audio output condition of the listening sound for the listening space.

  An object of the present invention is to solve the above-described problems, and an audio relating to an output state of a listening sound in accordance with a change in a relative relationship between a stationary object and a speaker while detecting a stationary object arranged in the listening space. An object of the present invention is to provide a sound setting device and a sound setting method capable of setting output conditions.

  In order to achieve the above object, the sound setting device of the present invention provides an output state of a listening sound to be heard by a listener output from the speaker toward a listening space in which a plurality of speakers are installed. An audio setting device for setting audio output conditions related to audio output means for driving the plurality of speakers based on audio signals, listening space environment detecting means for detecting a stationary object existing in the listening space, and The listening space environment detecting means comprises audio output condition setting means for setting an audio output condition corresponding to the change when detecting that the relative relation of the stationary object to the speaker has changed. And

  In the acoustic setting device of the present invention, when the relative relationship of the stationary object to the speaker changes while detecting the stationary object arranged in the listening space, the audio output condition setting relating to the output state of the listening sound to the listening space is set to the relative relationship. Therefore, the optimum audio output condition can be set even when the relative relationship between the stationary object and the speaker changes in the listening space.

FIG. 1 is a perspective view illustrating an external appearance of an acoustic system including an acoustic setting device according to a first embodiment of the present invention as viewed from an oblique front. FIG. 2 is a plan view illustrating an appearance of an acoustic system including the acoustic setting device according to the first embodiment. FIG. 3 is a block diagram illustrating an outline of an acoustic system including the acoustic setting device according to the first embodiment. FIG. 4 is a block diagram illustrating an outline of a camera unit incorporated in an acoustic system including the acoustic setting device according to the first embodiment. FIG. 5 is a diagram showing subdivided areas in the RGB image data used in the first embodiment. FIG. 6 is a flowchart showing the flow of the audio output condition setting process in the sound system including the sound setting device of the first embodiment. FIG. 7 is a perspective view showing an example of the listening space SP for explaining the operation of the first embodiment. FIG. 8 is a perspective view showing a state in which the sound setting device according to the first embodiment is operated corresponding to the listening space SP shown in FIG. FIG. 9 is a flowchart showing details of the AF process executed in step S6 of FIG. FIG. 10 is a perspective view showing an example of environmental variation of the listening space SP for explaining the operation of the first embodiment. 11A and 11B are diagrams for explaining the operation of the first embodiment. FIG. 11A shows image data of the listening space SP shown in FIG. 7, and FIG. 11B shows a grouping area obtained by AF processing of the image data. Yes. 12A and 12B are diagrams for explaining the operation of the first embodiment. FIG. 12A shows image data of the listening space SP shown in FIG. 10, and FIG. 12B shows a grouping area obtained by performing AF processing on the image data. Yes. FIG. 13 is a diagram illustrating a state in which the sound setting device according to the first embodiment rotates the speakers 101 and 101 horizontally to set the angles θ1 and θ2 according to the listening space environment of the listening space SP illustrated in FIG. is there. FIG. 14 is a diagram illustrating a state in which the sound setting device according to the first embodiment rotates the speakers 101 and 101 horizontally to set the angles θ1 and θ2 according to the listening space environment of the listening space SP illustrated in FIG. is there. FIG. 15 is a flowchart illustrating the flow of an audio output condition setting process in an acoustic system including the acoustic setting device according to the second embodiment. FIG. 16 is a perspective view showing an example of the listening space SP for explaining the operation of the second embodiment. FIG. 17 is a perspective view showing an example of environmental variation of the listening space SP for explaining the operation of the second embodiment. FIG. 18 is a diagram for explaining the operation of the second embodiment, where (a) shows image data of the listening space SP shown in FIG. 16, (b) shows a grouping area obtained by AF processing of the image data, (C) shows the face detection range in the grouping area. FIG. 19 is a diagram for explaining the operation of the second embodiment. (A) shows the image data of the listening space SP shown in FIG. 17, (b) shows the grouping area obtained by AF processing of the image data, (C) shows the face detection range in the grouping area. FIG. 20 is a diagram illustrating a state in which the sound setting device according to the second embodiment rotates the speakers 101 and 101 horizontally to set the angles θ1 and θ2 according to the listening space environment of the listening space SP illustrated in FIG. is there. FIG. 21 is a diagram illustrating a state in which the speakers 101 and 101 are rotated in the horizontal direction according to the listening space environment of the listening space SP illustrated in FIG. . FIG. 22 is a flowchart illustrating the first half of the flow of the audio output condition setting process in the sound system including the sound setting device according to the third embodiment. FIG. 23 is a flowchart illustrating the second half of the flow of the audio output condition setting process in the sound system including the sound setting device according to the third embodiment. FIG. 24 is a perspective view showing an example of the listening space SP for explaining the operation of the third embodiment. FIG. 25 is an explanatory diagram of an example in which the sound setting device according to the third embodiment is operated in accordance with the listening space environment of the listening space SP shown in FIG. 24, and (a) is the image data of the listening space SP shown in FIG. 24. (B) shows a grouping area obtained by AF processing of the image data, and (c) shows a face detection range in the grouping area. FIG. 26 is a diagram illustrating a state in which the sound setting device according to the third embodiment rotates the speakers 101 and 101 horizontally to set the angles θ1 and θ2 according to the listening space environment of the listening space SP illustrated in FIG. is there. FIG. 27 is a flowchart illustrating the first half of the flow of the audio output condition setting process in the sound system including the sound setting device according to the fourth embodiment. FIG. 28 is a flowchart illustrating the second half of the flow of the audio output condition setting process in the sound system including the sound setting device according to the fourth embodiment. FIG. 29 is an explanatory diagram of the line-of-sight detection process of the sound setting device according to the fourth embodiment. FIG. 29A illustrates an example of a template used for the line-of-sight detection process, and FIG. 29B illustrates an example of a matching process using the template. Yes. FIG. 30 is an explanatory diagram of an example in which the sound setting device according to the fourth embodiment operates in accordance with the listening space environment of the listening space SP, where (a) shows image data of the listening space SP, and (b) shows the image. A grouping area obtained by AF processing of data is shown, (c) shows a face detection range before the eye gaze detection process, and (d) shows a face detection range after the eye gaze detection process. FIG. 31 is an explanatory diagram of an operation example of the sound setting device according to the fourth embodiment. The speaker 101 corresponds to the listening space environment when the right listener MA sleeps from the listening space SP shown in FIG. , 101 is rotated horizontally and is set to angles θ1 and θ2.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described with reference to the accompanying drawings.

[Constitution]
First, based on FIGS. 1-4, the structure of the acoustic setting apparatus of Example 1 is demonstrated. 1 is a perspective view showing an external appearance of the acoustic system 100 including the acoustic setting device according to the first embodiment of the invention, and FIG. 2 is an acoustic system 100 including the acoustic setting device according to the first embodiment. FIG. 3 is a block diagram showing the outline of the acoustic system 100, and FIG. 4 is a block diagram showing the outline of the camera unit 102 incorporated in the acoustic system 100. As shown in FIG.

  The acoustic system 100 includes a pair of speakers 101, 101, a camera unit 102, a drive stage 103 that rotates the speaker 101 about a vertical axis, and a power switch 104.

As shown in FIG. 3, the acoustic system 100 includes an external device I / F block 111, a D / A conversion block 112, a signal processing block 113, a drive block 114, a camera unit 102, and an acoustic output block (audio output means) 30. I have. The camera unit 102 includes a camera control block 21 and a lens unit block 22, and the sound output unit 30 includes an amplifier block 31.
The external device I / F block 111 has an audio input terminal for inputting an audio signal from an external device such as a television or a DVD player. The audio signal input to the external device I / F block 111 is subjected to signal processing to be converted into digital output data by the signal processing block 113. The signal processing block 113 is configured by a DSP (Digital Signal Processor), and performs processing such as noise reduction and filter processing on input data.

  The digital output data converted by the signal processing block 113 is converted into analog data by the D / A conversion block 112, and the analog data is input to the sound output unit 30. The sound output unit 30 can perform amplification processing on the analog data by the amplifier block 31.

  The drive block 114 can rotate the drive stage 103 below each speaker 101 by a predetermined angle. The drive block 114 and the drive stage 103 correspond to means for changing the direction of the speakers 101 and 101.

Next, the configuration of the camera unit 102 will be described with reference to FIG.
The camera unit 102 is an imaging unit that captures an image of the listening space SP (see FIG. 7) and obtains image data. The camera unit 102 includes a camera control block 21 and a lens unit block 22, which will be described later in the first embodiment. It functions as a listening space environment detection means. As shown in FIG. 7, the listening space SP refers to a space where the sound system 100 is arranged and audio output is made by the speakers 101, 101. In FIG. 7, the room where the chair CH1 is placed. Is shown as an example.

  Returning to FIG. 4, the camera control block 21 performs control to execute processing of an image signal obtained by the lens unit block 22, auto focus (hereinafter referred to as AF) processing to be described later, and the like. Do.

The camera control block 21 includes a camera processor 7 (hereinafter simply referred to as “processor 7”).
The processor 7 includes a CCD1 signal processing block 72, a CCD2 signal processing block 76, a CPU block 73, a local SRAM 77, a memory controller block 71, and an I2C block 75, which are connected to each other via a bus line. An SDRAM 9 for storing YUV image data is disposed outside the processor 7 and is connected to the processor 7 by a bus line. A ROM 8 storing a control program is arranged outside the processor 7 and is connected to the processor 7 by a bus line.

  The lens unit block 22 includes a lens barrel unit 5. The lens barrel unit 5 includes a zoom optical system 51 having a zoom lens 51a and a focus optical system 52 having a focus lens 52a. The zoom optical system 51 and the focus optical system 52 are driven by a zoom motor 51b and a focus motor 52b, respectively. The operation of each of the motors 51b and 52b is controlled by a motor driver 55 controlled by the CPU block 73 of the processor 7. As the zoom motor 51b, a DC motor is used, and when the power switch 104 is turned on, the focal length is controlled to move to a position corresponding to 35mm in terms of 35mm film. Further, a stepping motor is used as the focus motor 52b, and the driving range of the focus lens 52a is set to a focal length of 1 m in the front direction of the acoustic system 100 in consideration of the listening space environment when the acoustic system 100 is actually installed. It is made to move in the range of ~ 5m. An AF process to be described later is executed based on the amount of extension (pulse amount) of the range by the focus lens 52a.

  The lens barrel unit 5 includes a photographic lens that connects a subject image to the CCD 10 that is an image sensor. The CCD 10 converts the subject image into an image signal and inputs it to the F / E-IC 6. The F / E-IC 6 has a CDS circuit 61, an ADC circuit 62, and an A / D converter 63, performs predetermined processing on each image signal, converts it to a digital signal, and inputs it to the CCD 1 signal processing block 72 of the processor 7. To do. These signal processing operations are controlled via the timing generator 64 by a VD signal (vertical drive signal) / HD signal (horizontal drive signal) output from the CCD1 signal processing block 72 of the processor 7.

Next, the AF operation of the camera unit 102 will be described.
First, light incident on the CCD 10 through both lenses 51a and 52a is converted into an electric signal and sent to the CDS circuit 61 and the A / D converter 63 as R, G, and B of analog signals. Each signal converted into a digital signal by the A / D converter 63 is converted into a YUV signal by a YUV converter (not shown) in the SDRAM 9 and written into the frame memory by the memory controller block 71. This YUV signal is output for each VD signal so that it can be updated each time. The YUV signal is read by the memory controller block 71, and an AF evaluation value indicating the degree of focus of the image data and an AF evaluation value indicating the exposure state of the data are calculated.

  The AF evaluation value data is read out as feature data to the CPU block 73 and used for AF processing. When the AF evaluation value (the integral value (average value) of the high frequency component of the Fourier coefficient on the image plane) is in focus, the high frequency component is the highest because the edge portion of the subject is clear. By utilizing this, during the focus detection operation by AF, the AF evaluation value at each focus lens position is acquired, and the point (peak position) at which the maximum is obtained is detected. Also, taking into account the fact that there are multiple points that are maximal, if there are multiple points, determine the magnitude of the evaluation value of the peak position, the degree of decrease and increase of the evaluation values around it, and determine the most reliable point. AF is executed as the in-focus position.

  In the first embodiment, the AF evaluation value is calculated in each of a plurality of subdivided areas in the digital RGB signal. FIG. 5 shows subdivided areas in the RGB image data used in the first embodiment. In the first embodiment, an area divided into eight in both the horizontal direction and the vertical direction is used. Each area has a coordinate, horizontal 1 at the upper left, vertical 1, horizontal 8 at the lower right, and vertical 8. Note that the number of area divisions in this case is not limited to this and may be further subdivided.

  The flow of the audio output condition setting process in the acoustic system 100 according to the first embodiment will be described with reference to the flowchart of FIG.

  First, in step S1, it is confirmed whether or not the power switch 104 of the sound system 100 is turned on. If it is not turned on, the process is terminated, and if it is turned on, the process proceeds to step S2.

  In step S2 that proceeds when the power source is On, it is confirmed whether or not acoustic data is received by the external device I / F block 111. If not, the process ends. If it is received, step S3 is performed. Proceed to

  In step S3, it is confirmed whether or not the drive stage 103 of the acoustic system 100 is already driven. If not, the process proceeds to step S6. If already driven, the process proceeds to step S4.

  In step S4, the listening space SP is imaged by the camera unit 102, and after performing a change confirmation process for checking whether or not there has been an environmental change in the listening space SP, the process proceeds to step S5. In addition, the environmental fluctuation | variation in Example 1 means the change of the relative relationship with respect to the speakers 101 and 101 of the object which exists in the space containing the stationary object arrange | positioned in listening space SP. Further, the stationary object refers to an object placed in the listening space SP and stationary, and the chair CH1 in the listening space SP shown as an example in FIG. In addition, although there are various methods for confirming environmental changes, in the first embodiment, the difference between the image data output in the current frame in the camera unit 102 and the image data of the previous frame is used. Check using image variation. This calculation of image fluctuation will be described later.

  In step S5, it is confirmed whether or not there is a screen change. If there is an image change, the process proceeds to step S6. If there is no image change, the process proceeds to step S8. As described above, in steps S4 and S5, the relative displacement of the object existing in the space with respect to the camera unit 102 is detected in order to detect whether or not the relative relationship of the stationary object with respect to the speakers 101 and 101 has changed. In the acoustic system 100, the part that performs this processing corresponds to the listening space environment detection means.

In step S6, AF processing is executed, and the process proceeds to next step S7. The AF process is a process for measuring the distance between the camera unit 102 and a stationary object existing in the listening space SP using the AF function of the camera unit 102. The part that performs this processing in the camera unit 102 corresponds to the distance detecting means, and details thereof will be described later.
In step S7, a process for driving the drive stage 103 by the drive block 114 is executed based on the result of the AF process, and then the process proceeds to step S8. In the first embodiment, the process of driving the drive stage 103 is a process of rotating the drive stage 103 around a vertical axis (not shown). As a result, the horizontal direction of each speaker 101, 101 changes, and the audio output condition setting of the listening space SP changes. The parameter for setting the audio output condition is an angle. As shown in FIG. 8, the angles θ1 and θ2 with respect to the direction in front of the two speakers 101 and 101 and along the optical axis O of the camera unit 102 are determined. . The calculation of the angles θ1 and θ2 is performed by either the CPU block 73 or the signal processing block 113.
In step S <b> 8, the sound output unit 30 performs sound output processing for reproducing the audio signal from the external device obtained by the external device I / F block 111 using the speakers 101 and 101, and then the processing ends. Details of the sound output processing in step S8 will be described later.

  As described above, the audio output condition is set by changing the direction of the speakers 101 and 101 in response to the change of the stationary object by the processing of steps S7 and S8. In the acoustic system 100, the part that performs these processes corresponds to the audio environment setting means.

Next, a description will be given of the variation confirmation processing in step S4.
In the first embodiment, the image fluctuation confirmation used for the fluctuation confirmation processing stores the image data continuously acquired at the timing synchronized with the VD signal in the buffer memory of the SDRAM 9, and the next acquired image data, Comparison is made with the integration result calculated from the luminance difference. For example, first, the previous image data is stored in the buffer memory, the difference between the stored image data and the image data acquired at the latest timing is calculated, and then acquired at the latest timing after this calculation. The written image data is overwritten on the previous image data and stored in the buffer memory. Then, the difference calculation is repeated using the overwritten latest image data and the image data acquired at the next timing.

  The difference calculation processing performed using the image data acquired at the latest timing and the image data acquired at the previous timing stored in the SDRAM 9 is performed between adjacent pixels of each pixel constituting each image data. The luminance difference is integrated in each of the horizontal direction and the vertical direction, and is compared with the integration result acquired at the previous timing, and the difference is obtained from the sum of the horizontal difference result and the vertical difference result. This is a process for calculating the confirmation evaluation value Q. The fluctuation confirmation evaluation value Q is calculated for each VD signal generation timing.

式（１）において、Ｄ（ｉ，ｊ）はＡＦ処理エリア内の画素の座標を示す。ＨｓｔａｒｔはＡＦ処理エリアの水平開始位置であって、ｍはＡＦ処理エリアの水平範囲である。
また、垂直方向の隣接する画素間での輝度差分の積算結果をＶ（ｈ）とすると、その演算式は下記の式（２）で表される。

式（２）において、Ｄ（ｉ，ｊ）はＡＦ処理エリア内の画素の座標を示す。ＶｓｔａｒｔはＡＦ処理エリアの垂直開始位置であって、ｎはＡＦ処理エリアの垂直範囲である。
上記式（１）と式（２）によって算出されるＨ（ｖ）とＶ（ｈ）と、その一つ前のタイミングで算出された積分結果Ｈ’(ｖ)とＶ’(ｈ)を用いて、その差分の総和をＱ（ｔ）とすると、その演算式は下記の式（３）で表される。

式（３）において、ｔはＶＤタイミングカウントである。 An arithmetic expression used for the difference calculation process in the variation confirmation process will be described.
Assuming that the integration result of the luminance difference between adjacent pixels in the horizontal direction at the latest timing is H (v), the calculation formula is expressed by the following formula (1).

In Expression (1), D (i, j) indicates the coordinates of the pixels in the AF processing area. Hstart is the horizontal start position of the AF processing area, and m is the horizontal range of the AF processing area.
Further, if the integration result of the luminance difference between adjacent pixels in the vertical direction is V (h), the calculation formula is expressed by the following formula (2).

In Expression (2), D (i, j) indicates the coordinates of the pixels in the AF processing area. Vstart is the vertical start position of the AF processing area, and n is the vertical range of the AF processing area.
Using H (v) and V (h) calculated by the above formulas (1) and (2), and integration results H ′ (v) and V ′ (h) calculated at the timing immediately before. When the sum of the differences is Q (t), the calculation formula is expressed by the following formula (3).

In equation (3), t is the VD timing count.

  This evaluation value Q (t) is calculated for each image update timing (Vd timing), and in step S5, it is determined that there has been an environmental change if the difference is equal to or greater than a preset determination threshold. Note that the determination threshold value is preferably set to a value that allows a certain level of change in light and dark because a change in image data due to light and dark is also assumed. Further, in the first embodiment, as described above, the variation of the stationary object is detected by the inter-frame difference of the image data. However, the present invention is not limited to this. If there is no problem, other methods for calculating the optical flow from the image difference can be used.

Next, details of the AF processing executed in step S6 will be described based on the flowchart of FIG. In the AF process, the distance measurement area is a range from a position of 1 m to a position of 5 m in the optical axis direction from the camera unit 102, and the focus of the focus lens 52a is arranged at a distance of 1 m at the start of the AF process. Keep it.
First, in step S61, a wait process is performed until the falling edge of the VD signal is detected. When the falling edge of the VD signal is detected, the process proceeds to step S62.
In step S62, the focus motor 52b is driven according to the predetermined number of pulses to start the movement of the focus lens 52a toward the focal length of 5 m, and then the process proceeds to the next step S63.

  In step S63, a video signal after moving the focus lens 52a is acquired, an AF evaluation value is calculated from image data based on this video signal, and the process proceeds to step S64. In step S64, it is determined whether or not the position of the focus lens 52a has been moved to an end position (a position corresponding to a focal length of 5 m in the first embodiment). If the end position has been reached, the process proceeds to step S65. If not, the process returns to step S62, and focus drive and AF evaluation value acquisition are repeated.

  In step S65, a peak position detection process is executed, and the process proceeds to step S66. In this peak position detection process, first, the peak position is detected by the integrated value of the high-frequency component of each pixel in each subdivided area of FIG. Then, in each area, the positions where the peak positions are considered to be the same distance are grouped, and the end area number including the range is acquired.

Here, explanation is added about grouping.
In the description of the grouping, a case where the chair CH1 in the listening space SP is replaced will be described as an example. That is, in the listening space SP in which the acoustic system 100 is installed, the chair CH1 for a plurality of people as shown in FIG. 7 is replaced with a chair CH2 for one person as shown in FIG. As described above, the difference in grouping when a stationary object fluctuates will be described.

11 and 12 show image data and grouping results when AF processing is performed on the listening space SP shown in FIGS. 7 and 10, respectively. That is, when the listening space SP is in the state shown in FIG. 7, the image data shown in FIG. 11 (a) is obtained and grouped in the hatched area in FIG. 11 (b). In this example, the grouping area is an area of 1 to 7 in the horizontal direction and 6 to 8 in the vertical direction. On the other hand, when the listening space SP is in the state shown in FIG. 10, the image data shown in FIG. 12 (a) is obtained and grouped in the hatched area in FIG. 12 (b). In this example, the area is 2 to 5 in the horizontal direction and 6 to 8 in the vertical direction. In this way, even if an environmental change in which a stationary object fluctuates occurs in the listening space SP, this change can be detected by performing grouping, and the presence position of the existing object in the listening space SP can be estimated. .
In the final step S66, a process of moving the focus of the focus lens 52a to the peak position is performed, and the AF process is terminated.

Next, the stage drive process in step S7 described in the flowchart of FIG. 6 will be described. As described above, the parameter for driving the stage is an angle. However, as shown in FIG. 8, the two angles θ1 and θ2 are calculated from the grouping area by the AF process as shown in FIG. The detection value is used. In the first embodiment, since the speakers 101 and 101 are rotated only in the horizontal direction, the speakers 101 and 101 are rotated so that the front surfaces of the speakers 101 and 101 face the both ends of the grouping area in the horizontal direction. The audio output states of the speakers 101 and 101 are made uniform. Therefore, the horizontal area positions corresponding to both ends of the grouping area are calculated from the angle of view with a focal length of 35 mm and the listening distance d, and the differences L1 and L2 between the angle of view edge and the horizontal area position (see FIG. 12B). ), And the angles θ1 and θ2 are calculated based on the following formulas (4) and (5) based on atan (arc tangent) with the differences L1 and L2 listening distance d.
θ1 = atan (L1 / d) (4)
θ2 = atan (L2 / d) (5)
Next, the sound output process in step S8 described in the flowchart of FIG. 6 will be described.
In the sound output processing, the digitally converted audio signal is subjected to amplification processing or the like and output to the speakers 101 and 101. At this time, if it is necessary to change the output timing depending on the listening position, a process of shifting the output timings of the speakers 101 and 101 may be performed by performing a filter process. In that case, the listening distance d from the stationary object (for example, chairs CH1 and CH2) estimated by the grouping acquired in the AF process to each of the speakers 101 and 101 is calculated and filtered.

  Next, the operation of the first embodiment is the variation in which the chair CH1 shown in FIG. 7 as a stationary object is replaced with the chair CH2 shown in FIG. 10 in the listening space SP in which the acoustic system 100 is arranged, that is, the speaker 101. , 101, the case where a change occurs in the relative relationship of the stationary object will be described as an example.

  In the sound system 100, when the power switch 104 is turned on, the audio output condition setting process is continuously executed. Here, since the drive stage 103 is not driven at the time of the first process of the audio output condition setting process, the AF process is executed based on the flow of steps S1, S2, S3, and S6. At this time, when a chair CH1 for two or three persons is placed at the position shown in FIG. 7 in the listening space SP, as shown in FIG. A grouping in the range of 1 to 7 in the direction and 6 to 8 in the vertical direction is formed. Then, by the stage drive process in step S7, angles θ1 and θ2 (see FIG. 13) for directing the front surfaces of the speakers 101 and 101 to both ends in the horizontal direction of the grouping area are obtained. Further, based on both angles θ1 and θ2, The drive stage 103 is rotated. Along with this, both the speakers 101, 101 rotate integrally with the drive stage 103 to be in a state where the front faces slightly outward as shown in FIG. 13, and thereafter the sound output processing is executed.

  After that, when the chair CH1 is left as it is, it is not determined in step S5 that the screen has changed, and the sound output process is executed while the orientation of the speakers 101 and 101 is maintained.

  When the acoustic system 100 operates in this way, the sound field by the acoustic system 100 is set to an optimal audio output condition that is rich in the presence for the listener MA (see FIG. 16) hung on the chair CH1.

  On the other hand, in the listening space SP, when it is replaced with the chair CH2 for one person shown in FIG. 10 and is arranged as shown in the figure, an environmental change occurs, it is determined in step S5 that there is a screen change, and again. The AF process and the stage drive process are executed, and the audio output condition is set again.

  At this time, the range of the horizontal direction 2-5 and the vertical direction 6-8 is grouped by the peak position detection process of step S65, as shown in FIG.12 (b). Then, the angles θ1 and θ2 shown in FIG. 14 corresponding to both ends in the horizontal direction of the grouping area are obtained by the stage driving process in step S7. In this example, the angle θ1 does not change, but the angle θ2 changes greatly, and accordingly, the drive stage 103 arranged on the right side of the apparatus is rotated and arranged on the right side of the apparatus as shown in FIG. The speaker 101 is rotated.

  Therefore, the sound field by the acoustic system 100 is changed to a setting of audio output conditions that form an optimal sound field that is rich in the presence for the listener MA hung on the chair CH2.

As described above, in the first embodiment, the presence of stationary objects arranged in the listening space SP is detected, audio output conditions (acoustic parameters) are set according to these stationary objects, and the speaker in the listening space SP is set. When a change (image fluctuation) occurs in the relative relationship between the still objects 101 and 101, the audio output condition can be changed in accordance with the change of the still object, thereby enabling optimal audio output.
In this case, it is not necessary for the listener MA to measure the distance from the speaker 101, and the trouble of the listener MA can be saved. In addition, the setting of the audio output condition can be changed according to the change in the relative relationship of the other stationary objects such as the chairs CH1 and CH2 in the listening space SP to the speakers 101 and 101, and the listener MA can set the audio output condition. For this reason, it is not necessary to move such a stationary object, and it is possible to perform settings without taking time and effort.

  Above all, even if the relative relationship of the stationary object changes in the listening space SP, the listener MA automatically changes the audio output condition setting without resetting it, which further increases the labor of the listener MA. Can be omitted.

(Other examples)
In the following, acoustic ringing devices of other embodiments will be described.
In the description of the other embodiments, the same components as those in the first embodiment are denoted by the same reference numerals and the description thereof is omitted. Regarding the action, the action different from that of the first embodiment will be described, and the description of the same action as that of the first embodiment will be omitted.

<Example 2>
The sound setting device according to the second embodiment is different from the first embodiment in the content of processing in the sound system 100, and the listener detection unit that detects the listener MA is used as the listening space environment detection unit that detects the listening space environment of the listening space SP. It is an example to which is added.

  The flow of processing in the acoustic system 100 of the second embodiment will be described based on the flowchart of FIG. In this flowchart, steps S1 to S6 are the same as those in the first embodiment, and therefore, the same step symbols as those in the first embodiment are denoted by the same reference numerals, and the description thereof is omitted.

When the AF process in step S6 ends, the process proceeds to step S21, a face detection determination process is executed, and the process proceeds to step S22.
In the face detection determination process in step S21, it is determined whether or not the listener MA exists based on whether or not a face exists in the listening space SP. If the listener MA exists, the face is determined. Grouping is performed again for existing positions. Regarding the face detection method, the methods listed in the following a to c that have already become known in the digital still camera technology are known, and in the second embodiment, any one of the following methods is used. Shall.
a) Journal of Television Society Vol. 49, no. 6, pp. No. 787-797 (1995) “Proposal of a modified HSV color system effective for face area extraction” is a method of extracting a face area by converting a color image into a mosaic image and paying attention to a skin color area.
b) Journal of the Institute of Electronics, Information and Communication Engineers, Vol. 74-D-II, no. 11, pp. 1625-1627 (1991) “A method for extracting a face region from a still gray scene image”, the geometrical structure of each part constituting the head of a front human figure such as hair, eyes and mouth. Of extracting the head region of a frontal person using various shape features.
c) As shown in “Video phone face area detection and its effect” in Image Lab 1991-11 (1991), in the case of a moving image, the contour edge of a human image generated by a subtle movement of a person between frames is detected. A method of extracting a frontal person image using it.

  In step S22, the presence / absence of the listener MA is determined based on the result of the face detection determination process in step S21. If the listener MA exists, the process proceeds to step S23. If the listener MA is absent, the process proceeds to step S7. move on.

In step S23, the face detection range setting process for setting the range is performed again by the grouping process at the face detection position, and then the process proceeds to step S7. As described above, the part that performs the processes of steps S21 to S23 added in the second embodiment corresponds to the listener detection unit.
In addition, since the process after step S7 is the same as that of Example 1, description is abbreviate | omitted.

Next, the operation of the second embodiment will be described.
In describing this action, as a change in the environment in the listening space SP, the listener MA lay down on the chair CH1 as shown in FIG. 17 from the state in which the listener MA sat on the chair CH1 as shown in FIG. A case where the state is changed will be described as an example. That is, in Example 2, the relative relationship of the chair MA1 as a stationary object with respect to the speakers 101 and 101 is not changed and is in a constant state, but the relative relationship of the listener MA is changed.

  It is assumed that the operation of the sound setting device according to the second embodiment is started in the listening space environment of the listening space SP illustrated in FIG. In this case, first, processing is executed in the order of steps S1 to S3, S6, S21, S22, S23, S7, and S8, and the drive stage 103 is driven according to the listening space environment of the listening space SP to After the orientation is set, a sound output is made.

In this case, the image data shown in FIG. 18A is obtained in the AF process in step S6, and the grouping result shown in FIG. 18B is obtained. As shown in FIG. Grouped in the range of 4 to 8 in the vertical range.
Furthermore, in the second embodiment, an area in which a face is detected is set as shown by hatching in FIG. 18C by the face detection range setting process in step S23. In the illustrated example, the face detection range is The horizontal range is set to 4 to 5, and the vertical range is set to 4 to 6.

In the second embodiment, in the stage driving process in step S7, as in the first embodiment, the difference between L1 and L2 between the horizontal end and the view angle end of the face detection range and the atan (arc tangent) between the listening distance d is used. The angles θ1 and θ2 are calculated, and the drive stage 103 is driven. As a result, as shown in FIG. 20, both speakers 101, 101 face the front center direction of the acoustic system 100 so as to face both ends of the face detection range.
Therefore, in the acoustic system 100, a sound field full of a sense of reality that is optimal for the listener MA seated near the center in the width direction of the chair CH1 can be obtained.

  Next, the case where the posture of the listener MA changes from the state of being seated on the chair CH1 as shown in FIG. 16 to the state of lying on the chair CH1 as shown in FIG.

  In this case, such a change in the relative relationship of the listener MA with respect to the speakers 101, 101 appears in the screen fluctuation, and AF processing, face detection range setting processing, etc. are executed again based on the flow of steps S5, S6, S21 to S23. Is done.

  At this time, in the AF process, the image data shown in FIG. 19A is obtained, and further, a grouping area as shown in FIG. 19B is obtained. In this case, the horizontal range of the grouping area by AF processing is the same as the example shown in FIG.

  Therefore, in the second embodiment, the face detection range shown in FIG. 19C is set by the face detection range setting process in step S23. In this case, the face detection range is 2 to 4 in the horizontal range and the vertical range. It is determined that the range is 5 to 8, and the range is different from the state shown in FIG.

Therefore, in the stage driving process in step S7, the angles θ1 and θ2 are changed based on the face detection range, as shown in FIG. 21, and the directions of the speakers 101 and 101 are determined based on the rotation of the driving stages 103 and 103. Is changed.
Also in this case, both speakers 101, 101 face both ends of the face detection range, and a sound field full of a sense of reality that is optimal for the listener MA lying on the chair CH1 is obtained.

  As described above, in the second embodiment, in addition to the change in the relative relationship between the stationary objects such as the chairs CH1 and CH2 with respect to the speakers 101 and 101, the audio output condition can be set according to the change in the relative relationship of the listener MA. It is possible to provide the listener MA with a more realistic sound field.

  Moreover, in the second embodiment, not only face detection, but also AF processing is performed on an object existing in a space including a stationary object existing in the listening space SP as in the first embodiment. For this reason, even when face detection cannot be performed due to hiding the feature points, for example, parts such as eyes and mouth, at the time of face detection, AF processing for a stationary object (a chair in the example shown in FIG. 16) is performed as in the first embodiment. It is possible to set the audio output condition based on the result. Therefore, even when face detection is not performed, it is possible to set an audio output condition suitable for the listening space SP.

<Example 3>
Next, Example 3 will be described.
The third embodiment is an example in which the number of listeners MA is determined in the listening space SP, and the change in the number of listeners MA can be dealt with.

The flowcharts of FIGS. 22 and 23 show the processing flow of the third embodiment. In the third embodiment, only a part of the processing flow in the acoustic system 100 is different from the second embodiment, and the same processing steps as those in the second embodiment are denoted by the same step codes as those in the first and second embodiments. A description thereof will be omitted, and only differences will be described.
The difference from the first and second embodiments is that the face detection number search process is performed in step S31 until the process proceeds to step S23 in which it is determined in step S22 that the listener MA is present and the face detection range setting process is performed. Is a point.

  This face detection number search process detects the number of listeners MA in the listening space SP, and the number of faces that can be detected by the face is affected by the calculation capability of the CPU block 73 in the camera unit 102, etc. At least about four people are considered necessary.

  Next, face detection range setting processing for performing grouping processing on the face detection range is performed again in a state where the number of people is determined in the face detection number search processing in step S31. This face detection range setting process itself is the same as step S23 in the second embodiment, but as shown in FIG. 24, when there are two listeners MA, the process is as follows.

  That is, in the AF process in step S6, grouping is performed based on the image data shown in FIG. 25A, and the grouping result shown in FIG. 25B is obtained. In this case, the grouping area is 1 to 7 in the horizontal range and 4 to 8 in the vertical range. On the other hand, the face detection range set in the face detection range setting process in step S23 is set to a range including all of a plurality of faces (in this example, two faces). In FIG. As indicated by diagonal lines, a horizontal range of 3 to 7 and a vertical range of 4 to 6 are set.

  Therefore, in the stage driving process in step S7, as in the first embodiment, the angles θ1 and θ2 are calculated by the atan (arc tangent) between the differences L1 and L2 between the both ends of the face detection range and the view angle end and the listening distance d. Then, the drive stage 103 is driven.

Next, an operation example of the third embodiment will be described.
First, when the listening space environment of the listening space SP is in a state where one listener MA is sitting on the chair CH1, as shown in FIG. 16, as in the description of the second embodiment, FIG. The face detection range shown is set, and the speakers 101, 101 are tilted as shown in FIG. Therefore, an optimal audio output condition setting is formed for one listener MA sitting on the chair CH1.

  As shown in FIG. 24, when an environmental change in which two listeners MA sit on the chair CH1 occurs from this listening space environment, it is determined in S5 that there is a screen change, and AF processing (S6), face detection determination Processing (S21), face detection number search processing (S31), and face detection range setting processing (S23) are executed.

  By these processes, a face detection range including two faces is set as shown in FIG. 25C. In the stage driving process as described above, the horizontal edge and the angle of view of the face detection range are set. The angles θ1 and θ2 are calculated by the atan (arc tangent) of the differences L1 and L2 and the listening distance d, and the drive stage 103 is driven. Therefore, as shown in FIG. 20, the speakers 101 and 101 are changed in direction from a state in which they are inclined toward the center, as shown in FIG. Therefore, it is possible to form an even and optimal sound field.

  As described above, in the third embodiment, when a plurality of listeners MA exist in the listening space SP, the range is such that all of the plurality of listeners MA whose faces are detected are included. By outputting audio, an optimal sound field can be provided for all of the plurality of listeners MA.

  Similarly to the second embodiment, even when the face cannot be detected, the audio output condition can be set based on the AF processing result for the stationary object, and the audio output condition suitable for the listening space SP can be set.

<Example 4>
The sound setting device according to the fourth embodiment is an example in which the optimal sound setting can be performed when the listener MA sleeps.

The flowcharts of FIGS. 27 and 28 show the processing flow of the fourth embodiment. In the fourth embodiment, only part of the processing flow in the acoustic system 100 is different from that in the third embodiment. The same processing steps as those in the third embodiment are denoted by the same step codes as those in the first to third embodiments. A description thereof will be omitted, and only differences will be described.
The difference from the third embodiment is that the line-of-sight detection process is performed according to the number of detected persons between the face detection number search process in step S31 and the face detection range setting process in step S23.

  That is, in step S41, which is performed after the number of persons is determined in the face detection number search process in step S31, it is determined whether or not the number of detected persons is two or more. Proceeding to the setting process, if the number of detected persons is two or more, the process proceeds to step S42.

  In step S42, a line-of-sight detection process is performed on each face image whose face has been detected. This line-of-sight detection process is a process for confirming whether the listener MA is open (wakes up) or closed (sleeps). In this case, the eye image template TP shown in FIG. 29A is used, and the eye image template TP is matched with the face image while being enlarged or reduced, the image data G is searched, and the eye is detected, that is, the eye is opened. Detect whether or not. Note that FIG. 29B shows a matching state of the eye image template TP. Here, when the eye opening is not detected in the face image of the listener MA whose face has been detected, a process of excluding the face from the listener MA as being not detected is performed.

  Therefore, when setting the face detection range including all of the plurality of listeners MA, when the eye opening of the listener MA adjacent to the end and the neighbor in the horizontal direction is not detected and excluded from the listener MA, These listeners MA are not included in the face detection range including a plurality of face images.

  Therefore, in the fourth embodiment, as shown in FIG. 24, there are two listeners MA and MA in the listening space SP, and both the listeners MA and MA are open as shown in the figure. As in the third embodiment, a face detection range including the faces of the two listeners MA and MA is set as shown by the hatched area in FIG. In this case, the drive stages 103 and 103 are rotated by angles θ1 and θ2 shown in FIG. 26, and both speakers 101 and 101 are tilted in the center direction so as to face both ends in the horizontal direction of the face detection range.

  On the other hand, as shown in the image data of FIG. 30A, the case where the listener MA on the right side of the front of the apparatus is closed will be described. In this case, the grouping by AF processing is 1 to 7 in the horizontal range and 4 to 8 in the vertical range, as in the example shown in FIG. 25B, and the face detection number search processing in step S31 is executed. The face detection range at the time is an area of 3 to 7 in the horizontal range and 4 to 6 in the vertical range, as in the example shown in FIG.

  Further, the line-of-sight detection process in step S42 is executed, and the right listener MM is excluded from the face detection range because it is closed, so the face set by the face detection range setting process in step S23 The detection range is an area of 3 to 4 in the horizontal range including only the face of the listener MA on the left side, and 4 to 6 in the vertical range, as indicated by hatching in FIG.

  Therefore, in the stage driving process in step S7, as in the first embodiment, as shown in FIG. 31, the atan (arc) between the differences L1 and L2 between the horizontal ends of the face detection range and the view angle ends and the listening distance d is obtained. The angles θ1 and θ2 are calculated by (tangent). As a result, the drive stage 103 is driven, and the speaker 101 on the left side of the apparatus is greatly inclined.

  Therefore, in Example 4, when there are a plurality of listeners MA, it is confirmed whether or not the eyes are closed, and an audio output that forms an optimal sound field only for the listeners MA who are thought to be awake with their eyes open. Conditions can be set.

  In addition, after the audio output condition is set, when the listener MA who has detected the face sleeps and closes his eyes, the face detection range setting process and the line-of-sight detection process are performed by detecting the image fluctuation, and the sleeper sleeps. The listener MA is excluded and the face detection range is set, and the audio output conditions (angles θ1 and θ2) are reset according to this setting, and an optimal sound field is formed for the listener MA who is awake. The

  Similarly to the second embodiment, even when the face cannot be detected, the audio output condition can be set based on the AF processing result for the stationary object, and the audio output condition suitable for the listening space SP can be set.

  The embodiments of the present invention have been described using the examples. However, the present invention is not limited to these examples, and various modifications and substitutions may be made without departing from the scope of the present invention. Can do.

For example, in the embodiment, the drive stages 103 and 103 are rotated in the horizontal direction, but may be rotated in the vertical direction.
In the embodiment, an example of adopting so-called hill-climbing AF that scans the peak value in the AF processing has been shown. However, a method that produces a distance result, for example, a stereo parallax image using two-stage lenses. Thus, if the distance can be calculated, the distance to the object existing in the space existing in the listening space SP can be captured at a higher speed.
Moreover, although the example which used the two speakers 101 and 101 was shown in the Example, the number is not restricted to this, You may use 3 or more speakers, such as 5.1ch.

  In the embodiments, chairs CH1 and CH2 are shown as stationary objects, but the stationary objects are not limited to these chairs.

  Further, in the embodiment, the example in which the audio output condition is set automatically at the start of the execution of the audio output condition setting process is shown, but at least when the relative relationship of the object existing in the space changes, As long as the audio output condition is set according to this change, the initial setting may be automatically set based on other factors or manually set.

  In the embodiment, the audio output condition is set to rotate the speaker. However, the present invention is not limited to this. If the audio output condition related to the listening sound is set, the speaker position and the audio filter are set. You may make it perform the setting regarding listening sound, such as a process, a sound pressure, and an output timing.

  In the embodiment, the distance detecting means in the listening space environment detecting means is the one using the autofocus function of the imaging means. However, if the distance from the stationary object can be measured, the reflection of sound waves and light is possible. Other means such as one that measures distance using time may be used.

  In the embodiment, when detecting a stationary object, all the objects captured as image data and the distances measured by the AF processing are processed as stationary objects. For example, in the embodiment, it is determined whether there is an image variation. By using this means, processing for setting a still object that does not change between preset image data may be added.

  The present invention can be used for an audio device and a television or a projector equipped with the audio device.

30 Sound output unit (audio output means)
100 sound system (audio output condition setting means)
101 speaker 102 camera unit (listening space environment detection means)
CH1 Chair (stationary object)
CH2 Chair (stationary object)
SP listening space

WO2006057131

Claims (12)

A sound setting device for setting an audio output condition relating to an output state of a listening sound to be heard by a listener output from the speaker toward a listening space where a plurality of speakers are installed.
Audio output means for driving the plurality of speakers based on an audio signal;
A listening space environment detecting means for detecting a stationary object existing in the listening space;
The listening space environment detecting means comprises audio output condition setting means for setting an audio output condition corresponding to the change when detecting that the relative relation of the stationary object to the speaker has changed. A characteristic sound setting device.   2. The listening space environment detection unit includes an imaging unit that obtains image data of the listening space, and detects a change in the relative relationship based on a difference between the image data that fluctuates in time. The sound setting device described in 1.   The listening space environment detecting means divides an entire image data area from the image data into image areas for obtaining image data of the listening space, and relative to the speaker and the stationary object in each area. The sound setting device according to claim 1, further comprising a distance detection unit that acquires distance information.   The sound setting device according to claim 3, wherein the distance detection unit detects the relative distance using an autofocus mechanism of the imaging unit. The listening space environment detecting means has listener detection means for detecting the listener existing in the listening space by detecting the characteristics of the listener's face using the image data,
The audio output condition setting means corresponds to the change in the audio output condition setting when the listening space environment detecting means detects a change in a relative relationship with respect to the speaker in either the stationary object or the listener. The sound setting device according to any one of claims 2 to 4, wherein the setting is performed.   The listener detection unit detects, when a plurality of listeners are detected, detection of an eye opening of a face image of the listener, and excludes a listener of a face image of an unopened eye from the listener. Item 6. The acoustic setting device according to Item 5.   The sound setting device according to any one of claims 1 to 6, wherein the audio output condition setting means includes means for changing the direction of the speaker with respect to the listening space.   The listening space environment detecting means includes imaging means for obtaining image data of the listening space, and detects the in-focus position in each area by equally dividing the image data into a plurality of areas vertically and horizontally. Are grouped, the coordinate positions of both ends of the grouped area in the horizontal direction are specified, and the direction of the speaker is set based on the horizontal coordinate position and the distance to the stationary object. The sound setting device according to claim 7. A sound setting method for setting an audio output condition related to an output state of a listening sound to be heard by a listener output from the speaker toward a listening space where a plurality of speakers are installed.
An audio output step of driving the plurality of speakers based on an audio signal;
A listening space environment detection step of detecting a stationary object present in the listening space;
An audio output condition setting step for setting an audio output condition corresponding to the change when detecting that the relative relation of the stationary object to the speaker has changed in the listening space environment detection step; A characteristic sound setting method.   The listening space environment detection step includes an imaging step of obtaining image data of the listening space, and divides the entire image data area from the image data into subdivided areas, and relative to the speaker and the stationary object in each area The acoustic setting method according to claim 9, further comprising a distance detection step of acquiring distance information. The listening space environment detection step includes a listener detection step of detecting the listener existing in the listening space by detecting the feature of the listener's face using the image data,
The audio output condition setting step includes setting an audio output condition corresponding to the change when detecting that the relative relationship with respect to the speaker has changed in either the stationary object or the listener. The sound setting method according to claim 10.   In the listener detection step, when a plurality of listeners are detected, opening of the listener's face image is detected, and listeners of the unopened face image are excluded from the listener. Item 12. The sound setting method according to Item 11.

Images