JP2016181913A - Image data generating device and image data generating method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image pickup device capable of easily understanding a situation of sound being acquired by a user.SOLUTION: Image data of a display image is generated, where the display image is obtained by changing a sound acquisition level display image indicating sound acquisition level of a video signal according to a preset amplification degree of the sound signal.SELECTED DRAWING: Figure 15

Description

The present invention accurately picks up sound generated at any of a plurality of positions within the angle of view,
The present invention relates to an imaging apparatus that can easily understand the state of collected sound.

In general, an imaging device called a video camera captures a subject and picks up sound from the subject using a microphone. A video camera equipped with a video / audio recording / playback function records and plays back video information of a moving image obtained by shooting a subject and audio information collected during shooting.

A microphone mounted on a video camera is generally a monaural microphone or a stereo microphone. When picking up sound with a monaural microphone,
Regardless of the position where the sound is generated, it is recorded as a one-channel sound signal. When collecting sound with a stereo microphone, left and right sounds are recorded as 2-channel audio signals.

JP 2000-298933 A JP 2006-287544 A

Akio Ando, “Highly Realistic Acoustic Technology and Its Theory”, IEICE Fundamental Review Vol.3 No.4, p.33-46

In a video camera equipped with a monaural microphone or a stereo microphone, it is difficult to accurately collect sound generated at each of a plurality of positions within the angle of view.

For example, it is assumed that a person who is the main subject to be photographed is located near the center of the angle of view, and an animal sounds at the left or right end of the angle of view. Under such circumstances, it is difficult to accurately collect and record the sounds generated by animals. Therefore, it is required that sound generated at any of a plurality of positions within the angle of view can be accurately picked up.

In addition, when picking up sound generated at any of multiple positions within the angle of view, the user can easily pick up the state of the sound being picked up, such as whether or not the sound can be picked up accurately. It is required to be able to understand.

In order to meet such a demand, the present invention can accurately pick up sound generated at any of a plurality of positions within the angle of view, and the user can easily determine the state of sound being picked up. An object is to provide an imaging device that can be understood.

In order to solve the above-described problems of the related art, the present invention provides an image of a display image obtained by changing a sound collection level display image indicating a sound collection level of a sound signal in accordance with a set amplification level of the sound signal. An image data generation device (26) for generating data and an image data generation method are provided.
Further, in order to solve the above-described problems of the conventional technology, the present invention generates image data for generating image data of a speech recognition level display image indicating a speech recognition level indicating a recognition level of a human voice included in a speech signal. An apparatus (26) and an image data generation method are provided.

According to the imaging apparatus of the present invention, it is possible to accurately pick up sound generated at any of a plurality of positions within the angle of view, and the user can easily understand the state of the sound being picked up. Can do.

It is a block diagram which shows the imaging device of one Embodiment. It is a figure which shows the external shape of the imaging device of one Embodiment, and the example of the to-be-photographed object which an imaging device image | photographs. It is a figure which shows the specific structural example of the array microphone 10 in FIG. It is a figure for demonstrating the principle of the arithmetic processing in the audio | voice signal calculating part 12 (12 ') in FIG. It is a block diagram which shows the specific structural example of the audio | voice signal calculating part 12 in FIG. It is a block diagram which shows the specific structural example of the audio | voice signal calculating part 12 'in FIG. It is a figure which shows the 1st example of the sound collection level display image displayed by superimposing on an image in the imaging device of one Embodiment. It is a figure which shows the 2nd example of the sound collection level display image displayed on an image in the imaging device of one Embodiment. It is a figure which shows the 3rd example of the sound collection level display image displayed on an image in the imaging device of one Embodiment. It is a figure which shows the 4th example of the sound collection level display image displayed on an image in the imaging device of one Embodiment. It is a figure which shows various sound collection level display images. FIG. 9 is a diagram illustrating an example of a sound collection level display image when an amplification degree of an audio signal is adjusted in the second example of the sound collection level display image illustrated in FIG. 8. It is a figure which shows the example of the sound collection level display image at the time of adjusting the amplification degree of an audio | voice signal in the 3rd example of the sound collection level display image shown in FIG. It is a figure which shows the example of the sound collection level display image at the time of adjusting the amplification degree of an audio | voice signal in the 4th example of the sound collection level display image shown in FIG. It is a figure which shows the other example of the sound collection level display image at the time of adjusting the amplification degree of an audio | voice signal. It is a flowchart which shows operation | movement in the case of displaying a sound-collection level display image superimposed on an image in the imaging device of one Embodiment. It is a flowchart which shows the recording operation | movement in the imaging device of one Embodiment. It is a figure which shows the state which connected the imaging device of one Embodiment to the television receiver. It is a figure which shows the image which superimposed the image of the additional information for selecting the angle direction of a horizontal direction, and the identification image for classifying the angle direction of a horizontal direction. It is a figure which shows the image which superimposed the image of the additional information for selecting the angle direction of a horizontal direction, and the identification image for classifying the angle direction of a horizontal direction and a perpendicular direction. It is a flowchart which shows the reproduction | regeneration operation | movement in the imaging device of one Embodiment.

Hereinafter, an imaging apparatus according to an embodiment will be described with reference to the accompanying drawings. The imaging apparatus according to an embodiment is a video camera having a video / audio recording / playback function as an example.

In FIG. 1, a video camera 110 according to an embodiment includes an array microphone 10 and a stereo microphone 11. A monaural microphone may be used instead of the stereo microphone 11, or the stereo microphone 11 may be omitted and only the array microphone 10 may be used. However, it is preferable to provide the stereo microphone 11.

FIG. 2 shows an example of the external shape of the video camera 110 and an example of a subject 200 taken by the video camera 110. The array microphone 10 and the stereo microphone 11 are arranged on the upper surface of the housing of the video camera 110 as shown in FIG. The arrangement shown in FIG. 2 is merely an example, the position of the array microphone 10 and the stereo microphone 11 on the housing,
The mutual positional relationship is arbitrary.

As shown in FIG. 3A, the array microphone 10 includes a plurality of microphone elements ME.
Are arranged in a line in the horizontal direction. The distance between the centers of two microphone elements ME adjacent in the horizontal direction is d. The array microphone 10 in which the microphone elements ME shown in FIG.

As shown in FIG. 3B, the array microphone 10 includes a plurality of microphone elements ME.
The horizontal array 10b1, 10b2, 10b3 may be arranged in the vertical direction.
The distance between the centers of two microphone elements ME adjacent in the vertical direction is d. FIG. 3 (b)
An array microphone 10 in which a plurality of horizontal rows of microphone elements ME shown in FIG. Here, the distance between the centers of two microphone elements ME adjacent in the vertical direction may not be d.

The array microphone 10 shown in FIGS. 1 and 2 is the array microphone 10A shown in FIG. 3A or the array microphone 10B shown in FIG. (A), (b of FIG.
), The number of microphone elements ME is six in the horizontal direction, but the number of microphone elements ME in the horizontal direction is not limited to six. The array microphones 10B are arranged in three rows in the vertical direction, but the number of arrangement in the vertical direction is not limited to three. The array microphone 10
Is not limited to the arrangement in a straight line or a plane.

The array microphone 10A has a normal direction to the arrangement direction of the microphone elements ME of 0.
Assuming degrees, it is possible to pick up sounds having directivity in directions of up to 90 degrees on the left and right. The array microphone 10B can pick up sound having directivity in directions of up to 90 degrees on the left and right and up to 90 degrees on the top and bottom.

The audio signal collected by the array microphone 10 is converted into an audio signal calculation unit 12 (12 ′).
And an arithmetic processing described later is performed. The audio signal calculation unit 12 calculates the audio signal from the array microphone 10A. The audio signal calculation unit 12 ′ calculates the audio signal from the array microphone 10B. The audio signal calculation unit 12 (12 ′) generates and outputs an audio signal having the number of channels determined by an angle of view that is a shooting range captured by the video camera 110 and a predetermined resolution set by calculation processing. The resolution here refers to the angular resolution of the surrounding sound.

The audio signal amplifier 13 amplifies the stereo audio signal output from the stereo microphone 11 and the audio signal (multi-channel audio signal) of a predetermined number of channels output from the audio signal calculator 12 (12 ′). . The audio signal amplified by the audio signal amplifying unit 13 is supplied to the video / audio recording processing unit 16 and the terminal Ta of the switch 65.

The voice signal amplified by the voice signal amplification unit 13 is also supplied to the voice recognition unit 61, the sound pressure level calculation unit 62, the frequency spectrum analysis unit 63, and the voice recognition level calculation unit 64. The voice recognition unit 61 has a dictionary 611 that registers a plurality of vocabularies.

The voice recognition unit 61 recognizes a voice uttered by a person from sounds input as a multi-channel voice signal by referring to a vocabulary registered in the dictionary 611. The sound pressure level calculation unit 62 calculates the sound pressure level of the signal input from the audio signal amplification unit 13. The frequency spectrum analyzer 63 calculates the sound pressure level for each audio frequency band divided into a plurality of signals input from the audio signal amplifier 13. The voice recognition level calculation unit 64 calculates a voice recognition level based on the memory data amount used for the voice recognition process. Thereby, the recognition degree of a human voice can be quantified.

The sound pressure level calculator 62, the frequency spectrum analyzer 63, and the voice recognition level calculator 64 are
It is an example of the sound collection level calculation part which calculates the sound collection level of the audio | voice signal of the 1 or several angle direction in a multichannel audio | voice signal. The sound pressure level calculated by the sound pressure level calculator 62, the frequency spectrum of the sound analyzed by the frequency spectrum analyzer 63, and the sound recognition level calculator 6
The speech recognition level calculated by 4 is collectively referred to as the sound collection level of speech uttered by a person.

The sound pressure level calculation unit 62 may obtain the sound pressure level of the voice uttered by the person recognized by the voice recognition unit 61 by calculation. The frequency spectrum analysis unit 63 includes a voice recognition unit 61.
The frequency spectrum of the speech uttered by the person recognized by may be analyzed.

In a state where the switch 65 is connected to the terminal Ta, the audio signal amplified by the audio signal amplifying unit 13 is supplied to the audio output processing unit 22. In a state where the switch 65 is connected to the terminal Tb, the audio signal recorded on the recording medium 40 is supplied to the audio output processing unit 22 via the video / audio reproduction processing unit 21.

The audio output processing unit 22 can generate a sound based on the input audio signal from the speaker 23. When the operation unit 30 gives an instruction to increase or decrease the volume,
The control unit 20 controls the amplification degree of the audio signal in the audio output processing unit 22. In addition, an audio signal can be output to the outside from the audio output terminal 29a.

The control unit 20 controls the entire video camera 110. When an operation for adjusting the zoom is performed by the operation unit 30, the zoom adjustment unit 17 controls the video photographing unit 18 so as to obtain the designated zoom magnification. The operation unit 30 is an operation unit provided in a casing of the video camera 110 or a remote controller that remotely controls the video camera 110. Video shooting unit 18
Has a zoom lens and a CCD or CMOS image sensor. Information indicating the effective focal length determined by the zoom adjustment by the zoom adjustment unit 17 is input to the directivity switching unit 14.

In FIG. 2, when the video camera 110 is shooting the subject 200, the shooting range in which the video information is actually captured by the video camera 110 is a portion of the angle of view 201 surrounded by a broken line. The angle of view 201 is increased or decreased by zoom adjustment by the zoom adjustment unit 17. Based on the effective focal length, the directivity switching unit 14 can know the horizontal and vertical angular ranges that are actually captured by the video camera 110 having the size of the angle of view 201.

In the directivity switching unit 14, the resolution is set to 10 degrees, for example. The control unit 20 may change the resolution set in the directivity switching unit 14. Directivity switching unit 14
The information of the angle of view 201 is obtained based on the effective focal length, but the information of the angle of view 201 may be obtained by other methods.

The directivity switching unit 14 calculates and outputs the audio signal of each channel for the number of channels determined by the angle of view 201 and the resolution by the audio signal calculation unit 12 (12 ′). ) To control. The directivity information generation unit 15 is a voice signal calculation unit 12 (
12 ′) generates the directivity information of the audio signals of the respective channels generated and supplies them to the video / audio recording processing unit 16. The directivity information indicates, for example, the angle of the audio signal from the horizontal direction or the vertical direction.

The video signal output from the video photographing unit 18 is input to the video signal processing unit 19. The video signal processing unit 19 performs A / D conversion on the input video signal and performs predetermined signal processing. The video signal subjected to the predetermined signal processing by the video signal processing unit 19 is supplied to the video / audio recording processing unit 16 and the terminal Ta of the switch 66.

In a state where the switch 66 is connected to the terminal Ta, the video signal output from the video signal processing unit 19 is displayed to the video signal processing unit 24 so that the video signal in a state of photographing the subject is displayed on the display unit 28. Supplied with. In a state where the switch 66 is connected to the terminal Tb, the video signal output from the video / audio reproduction processing unit 21 is the video signal so that the video number recorded and reproduced on the recording medium 40 is displayed on the display unit 28. It is supplied to the processing unit 24.

The video signal output from the switch 66 is input to the video signal processing unit 24 and processed. The video composition unit 25 superimposes additional information on the video signal output from the video signal processing unit 24 under the control of the control unit 20 when the image data generation unit 26 generates image data. Synthesize. The video display processing unit 27 performs processing so that the video signal output from the video signal processing unit 24 or the video signal on which the additional information is superimposed by the video synthesis unit 25 is displayed on the display unit 28.

As shown in FIG. 2, the display unit 28 is a liquid crystal panel provided on the side surface of the video camera 110, for example. The video signal can be output to the outside from the video output terminal 29v.

The video / audio recording processing unit 16 includes a stereo audio signal output from the audio signal amplifying unit 13 and a multi-channel audio signal having directivity, directivity information output from the directivity information generating unit 15, and a video signal processing unit. The video signal output from 19 is recorded on the recording medium 40 as data of a predetermined signal format. The video / audio recording processing unit 16 may compress the video signal and the audio signal by a predetermined compression processing method and record the compressed signal on the recording medium 40.

A stereo audio signal and a multi-channel audio signal are recorded on the recording medium 40 in a multitrack. The recording medium 40 may be a recording medium built in the video camera 110 such as a semiconductor memory or a hard disk drive, or may be a removable recording medium such as a memory card.

The video / audio reproduction processing unit 21 reads out the video signal, the stereo audio signal, the multichannel audio signal, and the directivity information recorded on the recording medium 40 based on the control by the control unit 20 and performs the reproduction process. The stereo audio signal and the multi-channel audio signal are supplied to the terminal Tb of the switch 65. The video signal is supplied to the terminal Tb of the switch 66. The directivity information is supplied to the control unit 20.

With reference to FIG. 4, description will be given of how the audio signal calculation unit 12 (12 ′) calculates and generates an audio signal in each direction. In FIG. 4, description will be given by taking the array microphone 10A as an example. The normal to the arrangement direction indicated by the broken line of the microphone element ME of the array microphone 10A is L0. The direction of the normal L0 is set to 0 degree. If the incident direction of sound is 0 degree, the sound reaches each microphone element ME almost simultaneously.

As shown in FIG. 4, a case is considered where sound is picked up from a direction of a straight line L30 of 30 degrees with respect to the normal L0. When sound is incident on the array microphone 10A from the direction of the straight line L30,
Three microphone elements M on the right side with the center of the six microphone elements ME as a reference position
The voice reaches E with delay times of DL1, DL2, and DL3, respectively. Voices arrive at the left three microphone elements ME earlier than the reference position, and the voices arrive at delay times of negative delay times -DL1, -DL2, and -DL3.

That is, if the sound collected by each microphone element ME is delayed by the delay time shown in FIG. 4, a straight line L30 having an angle of 30 degrees with respect to the normal line L0 of the array microphone 10A.
The sound can be picked up from the direction. Actually, since the voice cannot be delayed with a negative delay time, for example, the delay time of the leftmost microphone element ME is set to 0, and the right microphone element ME is delayed so as to increase the delay time. If so, it is possible to pick up sound from a direction of 30 degrees.

As described above, if the sound collected by the microphone elements ME of the array microphones 10A and 10B is delayed by the delay time corresponding to the left and right angles from the normal direction, the sounds from a plurality of horizontal directions are collected. Can sound. Similarly, in the case of the array microphone 10B, if the sound to be collected is delayed by a delay time corresponding to the angle in the vertical direction, the sound from a plurality of vertical directions can be collected.

The audio signal calculation unit 12 (12 ′) generates and generates an audio signal having directivity from the left, right, and upper and lower angular directions based on the principle described above. First, the specific configuration and operation of the audio signal calculation unit 12 that performs calculation processing on the audio signal from the array microphone 10A will be described with reference to FIG.

As shown in FIG. 5, the audio signal calculation unit 12 includes variable delay devices 121a to 121f, an adder 122, and an amplifier 123. The variable delay devices 121a to 121f are provided corresponding to the number of microphone elements ME. The variable delay devices 121a to 121f are supplied from the control unit 20 to the variable delay devices 121a to 121f.
A horizontal delay control signal Sdlch for setting each delay time is input.

For example, when the angle of view 201 is 60 degrees in the horizontal direction and the resolution is set to 10 degrees, the audio signal calculation unit 12 is 0 degrees that is the normal direction, 10 degrees, 20 degrees, and 30 degrees in the right direction. Audio signals in directions of 7 channels of 10 degrees, 20 degrees, and 30 degrees in the left direction are generated by arithmetic processing. If the angle of view 201 is 30 degrees in the horizontal direction, there are four channels. The control unit 20 supplies a horizontal delay control signal Sdlch for setting a delay time for each channel direction to the variable delay devices 121a to 121f.

When the angle of view 201 is 60 degrees in the horizontal direction, the control unit 20 is 0 degrees, 10 degrees in the right direction, 20 degrees,
The variable delay devices 121a to 121f are sequentially switched by switching the horizontal delay control signal Sdlch so as to set respective delay times for generating audio signals for seven channels of 30 degrees, 10 degrees, 20 degrees, and 30 degrees in the left direction. To supply.

The audio signal output from each microphone element ME is delayed by the variable delay devices 121a to 121f set to the delay time corresponding to one of the channels. Adder 1
22 adds all the audio signals output from the variable delay devices 121a to 121f, and the amplifier 123
The added audio signal output from the adder 122 is amplified and output as an audio signal of one channel. The audio signal calculation unit 12 is controlled by the variable delay devices 121a to 121d by the horizontal delay control signal Sdlch
By setting the delay time corresponding to each channel by 121f, audio signals for 7 channels are sequentially generated and output.

Next, a specific configuration and operation of the audio signal calculation unit 12 ′ that performs calculation processing on the audio signal from the array microphone 10B will be described with reference to FIG. As shown in FIG. 6, the audio signal calculation unit 1
2 ′ includes horizontal delay units 12b1, 12b2, 12b3, a vertical delay unit 124, an adder 125,
And an amplifier 126. The horizontal delay units 12b1, 12b2, and 12b3 are provided corresponding to the arrangements 10b1, 10b2, and 10b3 of the microphone elements ME shown in FIG.

The horizontal delay unit 12b1 includes variable delay devices 121a1 to 121f1 and an adder 1221. Horizontal delay unit 1
2b2 includes variable delay devices 121a2 to 121f2 and an adder 1222. The horizontal delay unit 12b3 includes variable delay devices 121a3 to 121f3 and an adder 1223. Variable delay devices 121a1 to 121f1, 121a2 to 121f2, 12
The horizontal delay control signal Sdlch described in FIG. 5 is input to 1a3 to 121f3. Horizontal delay unit 12
The operations of b1, 12b2, and 12b3 are the same as those of the variable delay devices 121a to 121f and the adder 122 in FIG.

The variable delay devices 1241 to 1243 of the vertical delay unit 124 include a horizontal delay unit 12b1,
Based on the audio signals output from 12b2 and 12b3, the vertical delay control signal Sdlcv for generating voice signals having respective directivities determined by the vertical angle and resolution of the angle of view 201.
Is entered. The audio signals output from the horizontal delay units 12b1, 12b2, and 12b3 are delayed by variable delay devices 1241 to 1243 set to delay times corresponding to any channel in the vertical direction.

The adder 125 adds all the audio signals output from the variable delay units 1241 to 1243, and the amplifier 126 amplifies the added audio signal output from the adder 125 and outputs it as an audio signal of one channel. . The audio signal calculation unit 12 ′ sequentially generates and outputs audio signals for a plurality of channels by setting the delay time corresponding to each vertical channel by the variable delay devices 1241 to 1243 by the vertical delay control signal Sdlcv. To do.

The audio signal calculation unit 12 ′ generates an n × m channel audio signal, where n is the number of channels in the horizontal angular direction and m is the number of channels in the vertical angular direction.

In the present embodiment, the switches 65 and 66 are connected to the terminal Ta, respectively, and the video signal of the subject being photographed is displayed on the display unit 28.
The sound collection level of the audio signal of one or more selected channels among the multi-channel audio signals can be displayed on the display unit 28.

When the user operates a predetermined key with the operation unit 30, as shown in FIG.
The image data generator 26 causes the pointer image 154 using, for example, a microphone image to be displayed at the lower end of the image 280 displayed on the display unit 28, for example, in the horizontal direction. When the direction of the pointer image 154 is adjusted and determined, the control unit 20 determines the pointer image 1.
A sound collection level display image indicating the sound collection level of the audio signal of the channel indicated by the direction in which 54 is directed is superimposed on the image 280.

In the example illustrated in FIG. 7, the control unit 20 displays the sound pressure level display image 71 based on the sound pressure level obtained by the sound pressure level calculation unit 62 as the sound collection level display image indicating the sound collection level.
80 is displayed at the lower right corner. Here, the lower right corner of the image 280 is a fixed position where the sound pressure level display image 71 is displayed. The position where the sound pressure level display image 71 is displayed is arbitrary. The position where the sound pressure level display image 71 is displayed may be changeable.

As an example, the sound pressure level display image 71 is an image in which a number of segments 71SG corresponding to the sound pressure level are displayed inside a rectangular shape. The sound pressure level may be indicated by a numerical value instead of the one or more segments 71SG.

In the example shown in FIG. 8, the control unit 20 causes the image data generation unit 26 to display two pointer images 155a and 155b made of wedge-shaped images instead of the pointer image 154 shown in FIG. , 155b are displayed pointer images 154a, 154b using microphone images. Here, two wedge-shaped images and two microphone images are used, but three or more images may be used.

When the direction of the pointer images 155a and 155b is adjusted and determined, the control unit 20 displays the pointer images 154a and 154b, and displays the sound pressure level of the audio signal of the channel indicated by the direction in which the pointer images 154a and 154b are directed. The image 71 is superimposed on the image 280. Pointer images 155a and 155b
And the pointer images 154a and 154b are displayed at the same time, the orientation of the pointer images 155a and 155b may be adjusted. The display positions of the pointer images 154a and 154b and the sound pressure level display image 71 may be adjustable.

The controller 20 may be equipped with a face recognition function for recognizing a person's face so that the person's face is recognized and the pointer images 154a and 154b and the sound pressure level display image 71 may be displayed in the vicinity of the person's face. In this case, the video signal output from the video signal processing unit 19 may be input to the control unit 20. A face recognition unit may be provided separately from the control unit 20.

In the example illustrated in FIG. 9, the control unit 20 causes the image data generation unit 26 to display a vertical horizontal region dividing line 155 made of, for example, a broken line, and displays the image 280 in three regions R1 to R3 in the horizontal direction.
It is divided into two. As indicated by the solid line double arrow, the horizontal area dividing line 155 is movable to the left and right. When the user operates the operation 30 to move the horizontal region dividing line 155, the control unit 20 controls the video composition unit 25 to move the position where the horizontal region dividing line 155 is superimposed.

When the position of the horizontal region dividing line 155 is adjusted and determined, the control unit 20 determines the regions R1 to R1.
3 A sound pressure level display image 72 is displayed at the lower end of each. The control unit 20 includes the regions R1 to R1.
The sound pressure level display image 72 may be displayed by obtaining the sound pressure level of the audio signal of the channel located at the center in the horizontal direction of R3 (or closest to the center).

The sound pressure level display image 72 corresponds to a sound pressure level display image 71 that is tilted in the horizontal direction. Similar to the sound pressure level display image 71, the sound pressure level display image 72 is an image in which a number of segments 72SG corresponding to the sound pressure level are displayed inside a rectangular shape. In FIG. 9, a sound pressure level display image 71 may be displayed instead of the sound pressure level display image 72. However, in the case of horizontal region division as shown in FIG. 9, it is preferable to display the sound pressure level display image 72.

When the array microphone 10B is used, a display as shown in FIG. 10 is also possible. In the example shown in FIGS. 10A and 10B, the control unit 20 causes the image data generation unit 26 to display, for example, vertical horizontal division lines 155 and horizontal vertical division lines 156 made of broken lines. The image 280 has three regions R11 to R13, R in three in the horizontal direction and two in the vertical direction.
It is divided into 21 to R23. As indicated by the solid double arrow, the horizontal region dividing line 155
Can move left and right, and the vertical area dividing line 156 can move up and down.

FIG. 10B shows a state where the horizontal area dividing line 155 and the vertical area dividing line 156 are moved from the state shown in FIG.

When the positions of the horizontal region dividing line 155 and the vertical region dividing line 156 are adjusted and determined, the control unit 20 displays the sound pressure level display image 72 at the upper end of each of R11 to R13, and each of the regions R21 to R23. A sound pressure level display image 72 is displayed at the lower end. Control unit 20
Is the sound pressure level of the audio signal of the channel located at the center (or closest to the center) in the horizontal direction of the regions R11 to R13, R21 to R23 and positioned at the center in the vertical direction (or closest to the center). The sound pressure level display image 72 may be displayed.

7 to 10 show examples in which the sound pressure level display images 71 and 72 indicating the sound pressure level obtained by the sound pressure level calculation unit 62 are superimposed on the image 280. FIG. Sound pressure level display image 71
, 72 may be superimposed with an image indicating the frequency spectrum analyzed by the frequency spectrum analyzing unit 63 or an image indicating the amount of speech recognition data calculated by the speech recognition level calculating unit 64.

FIG. 11A shows a sound pressure level display image 71. FIG. 11B is an example of a frequency spectrum display image 73 showing the frequency spectrum analyzed by the frequency spectrum analysis unit 63. The frequency spectrum display image 73 has frequencies f1 to f1 depending on the number of segments 73SG.
The level in each frequency band is shown up to fn.

FIG. 11C is an example of a voice recognition level display image 75 showing the voice recognition level calculated by the voice recognition level calculation unit 64. The voice recognition level display image 75 is a segment 75SG.
The voice recognition level is represented by the number of. Here, the voice recognition level display image 75 is a figure having the same shape as the sound pressure level display image 71, but the sound pressure level display image 71 and the voice recognition level display image 75 may be different from each other.

Also in the frequency spectrum display image 73 and the speech recognition level display image 75, as in the sound pressure level display image 72, the segments 73SG and 75SG may be a figure that increases or decreases in the horizontal direction.

In the present embodiment, the audio signal amplifying unit 13 can individually amplify each channel of the multichannel audio signal based on the control by the control unit 20. The audio signal amplifier 13 can amplify only the audio signals of the selected one or more channels.

In FIG. 12, the amplification level of the audio signal of the channel indicated by the direction in which the pointer image 154a in FIG. For example, the case of 3 [dB] is shown. In FIG. 12, the amplification factor is −3.
By setting [dB], the sound pressure level display image 71 is changed to a small sound pressure level display image 71S, and by setting the amplification degree to 3 [dB], the sound pressure level display image 71 is displayed with a large sound pressure level. The image is changed to 71L.

Thus, by setting the size of the sound pressure level display image 71 to a size corresponding to the amplification level of the audio signal, the user can easily recognize which channel the amplification level of the audio signal is adjusted. Can do. That is, the user can easily understand the situation of the collected sound.

FIG. 13 illustrates a state in which the sound pressure level display image 72 is displayed as shown in FIG. 9 and the amplification level of the audio signal of each channel is adjusted. In the region R1, a large sound pressure level display image 72L with a sound signal amplification degree of 3 [dB] is displayed. In the region R2, a small sound pressure level display image 72S with the amplification degree of the audio signal being −3 [dB] is displayed. In the region R3, a sound pressure level display image 72M having an intermediate magnitude, which is a normal magnitude as a reference when the amplification level of the audio signal is 0 [dB], is displayed. The sound pressure level display image 72M
This is the same as the sound pressure level display image 72 of FIG.

FIG. 14 shows the state where the sound pressure level display image 72 is displayed as shown in FIG. 10B and the amplification level of the audio signal of each channel is adjusted. Region R11, R12
, R22, a small sound pressure level display image 72S with a sound signal amplification level of −3 [dB] is displayed. In the regions R13 and R21, a large sound pressure level display image 72L is displayed with an audio signal amplification degree of 3 [dB]. In the region R23, a sound pressure level display image 72M having an intermediate magnitude with an audio signal amplification degree of 0 [dB] is displayed.

12 to 14, the sound pressure level display images 71 and 72 are changed in size while maintaining a similar relationship. However, the amplification level of the audio signal of each channel is expressed by other display methods. Also good. For example, as shown in FIGS. 15A to 15C, a sound pressure level display image 77S indicating a low amplification level, a sound pressure level display image 77M indicating a medium amplification level, and a sound pressure indicating a high amplification level. A level display image 77L may be used. Sound pressure level display image 77S, 77M, 77
L indicates the degree of amplification in the width of the central portion in the vertical direction.

As described above, the image data generation unit 26 may generate image data in which the size and shape of the sound pressure level display image are changed according to the amplification degree of the audio signal of each channel. In addition to the size and shape, the brightness, color, and pattern may be varied. Further, the control unit 20 may control the image data generation unit 26 so that the size, shape, luminance, color, and pattern of these sound pressure level display images are changed over time.

In addition, when making the brightness of the sound pressure level display image different, make the darker the smaller the amplification degree,
It is preferable to increase the brightness as the degree of amplification increases. When the color of the sound pressure level display image is made different, it is preferable that the lower the amplification degree, the colder color direction, and the higher the amplification degree, the warmer color direction. Furthermore, the control unit 20 controls the image data generation unit 26 so as to periodically change the size, shape, brightness, color, and pattern of the sound pressure level display image, and to shorten the changing cycle as the amplification degree increases. May be.

Similarly, in the frequency spectrum display image 73 indicating the frequency spectrum and the voice recognition level display image 75 indicating the voice recognition level, at least the size, shape, brightness, color, and pattern of each channel are determined according to the amplification degree of the voice signal of each channel. One may be different. The image data generation unit 26 generates image data for changing at least one of the size, shape, luminance, color, and pattern of the sound collection level display image according to the amplification degree set by the audio signal amplification unit 13. That's fine.

The display operation of the sound collection level display image in the video camera 110 configured as described above will be described again with reference to FIG. In FIG. 16, when the power of the video camera 110 is turned on, the control unit 20 starts photographing the subject in step S101. Control unit 2
In step S102, it is determined whether or not there is an instruction to display the sound collection level by the operation unit 30 in step S102. If there is an instruction to display the sound collection level (YES), the process proceeds to step S103, and if there is no instruction to display the sound collection level (NO), the process proceeds to step S121.

In step S121, the control unit 20 determines whether or not an instruction to end shooting has been issued due to power-off. If there is an instruction to end shooting (YES), the control unit 20 turns off the video camera 110 and ends the process. If there is no instruction to end shooting (NO), step S102 is repeated.

In step S103, the control unit 20 displays a position instruction image at a predetermined position on the image 280. The pointer image 154 in FIG. 7, the pointer images 155a and 155b in FIG. 8, the horizontal area dividing line 155 and the vertical area dividing line 156 in FIG. 9 or 10 are examples of position indication images.

In step S104, the control unit 20 determines whether or not there is an instruction to cancel the display of the position instruction image. If there is an instruction to cancel the display (YES), the control unit 20 cancels the display of the position instruction image in step S124, and returns the process to step S101. If there is no instruction to cancel the display (NO
) In step S105, the control unit 20 determines whether or not there is an instruction to move the position instruction image.

If there is an instruction to move the position instruction image (YES), the control unit 20 in step S106,
The position indicating image is moved, and the process proceeds to step S107. If there is no instruction to move the position indicating image (NO), the process proceeds to step S107. In step S107, the control unit 20 determines whether or not there is an instruction to cancel the display of the position instruction image. If there is an instruction to cancel the display (YE
S) The control unit 20 cancels the display of the position indicating image in step S124, and returns the process to step S101. If there is no display cancellation instruction (NO), the control unit 20 shifts the process to step S108.

In step S108, the control unit 20 determines whether there is an instruction to determine the position of the position instruction image. If there is a position determination instruction (YES), the control unit 20 shifts the process to step S109, and if there is no position determination instruction (NO), the control unit 20 shifts the process to step S122.

In step S122, the control unit 20 determines whether or not an instruction to end shooting has been issued due to power-off. If there is an instruction to end shooting (YES), the control unit 20 turns off the video camera 110 and ends the process. If there is no instruction to end shooting (NO), the process returns to step S107.

In step S109, the control unit 20 stores the position of the position indication image determined. The directivity switching unit 14 acquires the effective focal length in step S110 based on the control of the control unit 20. In step S111, the directivity switching unit 14 determines the number of channels from the angle of view and the resolution.
Here, the number of channels is the number of channels of a multi-channel audio signal generated by collecting sound by the array microphone 10.

The effective focal length is f and the horizontal length of the image receiving area of the image sensor at the angle of view 201 (view angle 2
If the width of 01 is L _W and the length in the vertical direction (height of the angle of view 201) is L _H , the horizontal angle α _W and the vertical angle α _H of the angle of view 201 are expressed by the following formula ( It can be determined as 1) and (2).
α _W = 2 tan ⁻¹ (L _W / 2f) (1)
α _H = 2 tan ⁻¹ (L _H / 2f) (2)

Determining the number of horizontal and vertical channels from the horizontal angle α _W and the vertical angle α _H of the angle of view 201 obtained from the equations (1) and (2) and the preset resolution. Can do.

In step S112, the directivity switching unit 14 determines the delay time in the angular direction of the channel corresponding to the position indicated by the position indication image. In step S113, the audio signal calculation unit 12 (12 ′) calculates the sound collection in the angular direction. The sound pressure level calculation unit 62, the frequency spectrum analysis unit 63, and the speech recognition level calculation unit 64 calculate the sound collection level in step S114 based on the control of the control unit 20. In step S115, the control unit 20 displays the sound collection level display image in a state of being superimposed on the image 280 as described with reference to FIGS.

In step S116, the control unit 20 determines whether or not there is an instruction to cancel the display of the sound collection level display image. If there is an instruction to cancel the display (YES), the control unit 20 cancels the display of the sound collection level display image in step S125, and returns the process to step S101. If there is no instruction to cancel the display (NO), the control unit 20 determines in step S117 whether or not the zoom magnification has been changed. If the zoom magnification is changed (YES), the control unit 20 determines whether or not the display position of the image indicating the sound collection level needs to be changed in step S118.

When the zoom magnification is changed, the sound collection level display image may deviate from the angle of view, and the sound collection level display image may not be displayed on the image 280. If it is necessary to change the display position of the sound collection level display image (YES), the control unit 20 changes the position of the sound collection level display image so that the sound collection level display image is within the angle of view in step S119. Then, the process proceeds to step S120.

If the zoom magnification is not changed in step S117 (NO), and if it is not necessary to change the display position of the sound collection level display image in step S118 (NO), the control unit 20 performs step S1.
At 20, it is determined whether or not there is an instruction to cancel the display of the sound collection level display image. If there is an instruction to cancel the display (YES), the control unit 20 cancels the display of the sound collection level display image in step S125, and returns the process to step S101.

If there is no instruction to cancel the display (NO), the control unit 20 determines in step S123 whether or not there has been an instruction to end photographing due to power-off. If there is an instruction to end shooting (YES), the control unit 20 turns off the video camera 110 and ends the process. If there is no instruction to end photographing (NO), the control unit 20 returns the process to step S116.

In the example shown in FIG. 16, when the zoom magnification is changed, the display position of the sound collection level display image is changed as necessary. However, if the zoom magnification is changed, the sound collection level display image is changed. May be canceled. At the same time, the display of the position instruction image may be canceled. Further, if the zoom magnification is changed, the display may be continued only when the sound collection level display image is within the angle of view, and the display may be canceled when it is out of the angle of view. Similarly, the display of the position indication image may be canceled together with the display of the image indicating the sound collection level.

In the example shown in FIG. 16, the delay time in the angular direction of the channel corresponding to the position indicated by the position indicating image is determined in step S112, and the sound collection in the angular direction is processed in step S113. Determines the delay time in all angular directions in the multi-channel audio signal, calculates the sound collection in all angular directions, and only the calculation result of the sound collection in the angular direction of the channel corresponding to the position indicated by the position indication image May be used.

A recording operation performed by the video camera 110 when an instruction to record a video signal and an audio signal is given will be described with reference to FIG. The recording operation can be executed during the operation of displaying the sound collection level display image of the audio signal of any channel in the multi-channel audio signal described in FIG. Further, it is possible to execute the recording operation regardless of the operation described with reference to FIG. In FIG. 17, for the sake of simplicity, only the recording operation will be described without relating to the display operation of FIG.

When the power of the video camera 110 is turned on, the control unit 20 starts photographing the subject in step S201. In step S202, the control unit 20 determines whether or not an instruction to start recording is given by the operation unit 30. If there is no instruction to start recording (NO), the control unit 20 shifts the process to step S211.

If there is an instruction to start recording (YES), the directivity switching unit 14 acquires the effective focal length in step S203. In step S204, the directivity switching unit 14 determines the number of channels from the angle of view and the resolution. As described with reference to FIG. 16, the angle of view 20 obtained from the above equations (1) and (2).
1 of the angle alpha _H horizontal angle alpha _W perpendicular direction, it is possible to determine the number of channels the horizontal and vertical directions and a resolution that is set in advance.

In step S205, the directivity switching unit 14 determines an angular delay time corresponding to each channel. In step S206, the audio signal calculation unit 12 (12 ′) calculates the sound collection in each angle direction.

In step S207, the video / audio recording processing unit 16 records the video signal, the stereo audio signal, the audio signal in each angular direction, and the directivity information indicating the angular direction in the recording medium 40. The video / audio recording processing unit 16 records audio signals in respective angular directions in a multitrack.

In step S208, the control unit 20 determines whether or not the zoom magnification has been changed. If the zoom magnification is changed (YES), the process returns to step S203. If the zoom magnification has not been changed (NO), the control unit 20 determines in step S209 whether or not an instruction to end recording has been given. If there is no instruction to end recording (NO), the process returns to step S206. If there is an instruction to end recording (YES), the control unit 20 executes a recording stop process in step S210, and shifts the process to step S211.

In step S211, the control unit 20 determines whether or not an instruction to end shooting has been issued due to power-off. If there is an instruction to end shooting (YES), the control unit 20 turns off the video camera 110 and ends the process. If there is no instruction to end shooting (NO), the process returns to step S202.

When the recording operation is performed during the operation of displaying the sound collection level display image of the audio signal of any channel in the multi-channel audio signal described in FIG. 16, the following operation may be performed. A step of determining whether or not a recording start instruction has been issued as in step S202 of FIG. 17 is provided at a predetermined position in the flow of FIG. Then, when there is an instruction to start recording, the same steps as steps S205 to S207, S209, and S210 in FIG. 17 are provided.

Next, the reproduction operation of the video signal and audio signal recorded on the recording medium 40 will be described.
In FIG. 1, when an operation for reproducing a video signal (captured content) recorded on the recording medium 40 is performed by the operation unit 30, the video / audio reproduction processing unit 21 is based on the control by the control unit 20. The video signal and stereo audio signal recorded in 40 are read out and subjected to reproduction processing.

When the video signal and the stereo audio signal are compressed by a predetermined compression processing method and recorded on the recording medium 40, the video / audio reproduction processing unit 21 expands the video signal and the stereo audio signal. When a multi-channel audio signal and directivity information are subjected to a specific operation described later,
Read from the recording medium 40. The stereo audio signal output from the video / audio reproduction processing unit 21 is input to the audio output processing unit 22. The audio output processing unit 22 generates a sound based on the stereo audio signal from the speaker 23.

The video signal output from the video / audio reproduction processing unit 21 is input to the video signal processing unit 24 and processed. When the image data generating unit 26 generates additional information such as the image data of the sound pickup level display image described above, the video synthesizing unit 25 adds the additional information to the video signal output from the video signal processing unit 24. Synthesize to superimpose. The video display processing unit 27 is connected to the video signal processing unit 2.
The video signal output from 4 or the video signal on which the additional information is superimposed by the video synthesis unit 25 is processed to be displayed on the display unit 28.

In FIG. 18, the video camera 110 is connected to the television receiver 300, the video signal is displayed on the screen of the television receiver 300, and the sound by the stereo audio signal is generated from the speaker built in the television receiver 300. It shows the state of letting. In FIG. 18, the remote controller is the operation unit 30. The television receiver 300 is a display unit outside the video camera 110.

When the user 400 performs a predetermined operation for reproducing a multi-channel audio signal generated by collecting sound with the array microphone 10 by the operation unit 30, the control unit 20 converts the video signal into the video signal shown in FIG. The additional information signal indicating the pointer image 51 is generated, and the pointer image 51 is superimposed on the image displayed on the screen. The pointer image 51 may be displayed at a preset initial position.

FIG. 19 shows images displayed on the screens of the display unit 28 and the television receiver 300. When the array microphone 10 is the array microphone 10A shown in FIG. 3A, the control unit 20 causes the image data generation unit 26 to add a vertical line horizontal position identification image 52 as shown in FIG. It is preferable to generate the additional information signal to be displayed and to superimpose the horizontal position identification image 52 on the image displayed on the screen. The horizontal position identification image 52 is a first identification image for dividing the angular direction in the horizontal direction.

The number of horizontal position identification images 52 is preferably the number corresponding to the number of channels of the multi-channel audio signal. If the number of channels of the multi-channel audio signal is seven, the screen can be divided into seven in the horizontal direction if the number of horizontal position identification images 52 is six. In this case, the screen is divided into seven areas.

The user 400 moves the pointer image 5 with the left and right cursor keys on the operation unit 30, for example.
When the operation of moving 1 to the left or right is performed, the control unit 20 displays the pointer image 51.
Is moved left and right. In FIG. 19, the pointer image 51 is an image made of a right-pointing arrow in a circle. However, when the pointer image 51 is moved leftward, the image is made of a left-pointing arrow in the circle. A pointer image 51 with a rightward or leftward arrow in the circle corresponds to additional information for selecting an audio signal from any one of the horizontal directions in the multichannel audio signal.

When the array microphone 10 is an array microphone 10B shown in FIG.
The control unit 20 causes the image data generation unit 26 to generate an additional information signal indicating a vertical horizontal position identification image 52 and a horizontal vertical position identification image 53 as shown in FIG. It is preferable to superimpose the horizontal position identification image 52 and the vertical position identification image 53 on the displayed image. The vertical position identification image 53 is a second identification image for dividing the angular direction in the vertical direction.

The number of vertical position identification images 53 is preferably the number corresponding to the number of vertical channels of the multichannel audio signal. If the number of channels in the vertical direction of the multi-channel audio signal is 4, the screen can be divided into four in the vertical direction if the number of vertical position identification images 53 is three. If the number of horizontal channels of a multi-channel audio signal is 7,
The screen is divided into 28 areas by the horizontal position identification image 52 and the vertical position identification image 53.

The user 400 moves the pointer image 5 using, for example, the up and down cursor keys on the operation unit 30.
When the operation of moving 1 upward or downward is performed, the control unit 20 displays the pointer image 51.
Is moved up and down. When the pointer image 51 is moved upward, the image is formed of an upward arrow in the circle, and when the pointer image 51 is moved downward, an image of the downward arrow is formed in the circle. At this time, it is displayed so that the tip of the arrow in the circle points to the center of the area. A pointer image 51 with an upward or downward arrow in a circle corresponds to additional information for selecting an audio signal from one of the vertical directions in the multichannel audio signal.

In FIG. 19, when the user 400 operates, for example, a determination key on the operation unit 30, the control unit 20 determines a channel in the angular direction to be reproduced according to the position of the pointer image 51 in the multichannel audio signal. The control unit 20 controls the video / audio reproduction processing unit 21 to reproduce a track on which the audio signal of the determined channel is recorded based on the directivity information indicating the determined angular direction of the channel.

Thus, the video camera 110 can select and reproduce the audio signals of the channels in the respective angular directions in the horizontal direction by moving the position of the pointer image 51 in the horizontal direction and operating the enter key. . In FIG. 20, in order to hear the voice of a child who is located at the right end, if the decision key is operated by moving the pointer image 51 toward the right end, the voice of the child is emitted. Can play and listen.

In the case of FIG. 20, by moving the position of the pointer image 51 in the vertical direction and operating the enter key, it is possible to select and reproduce the audio signals of the channels in the respective angular directions in the vertical direction.

The images of additional information for selecting the horizontal direction or the vertical angle direction are shown in FIGS.
It is not limited to the pointer image 51 as shown in FIG. For example, a small circular image may be used. Further, instead of the pointer image, the divided area may be highlighted and grayed out.

The reproduction operation by the video camera 110 when an instruction to reproduce the video signal and the audio signal is given will be described again with reference to FIG. In FIG. 21, the control unit 20 determines in step S301 whether or not an instruction to reproduce the recorded video signal (captured content) has been issued. If the reproduction instruction is not given (NO), the control unit 20 shifts the process to step S320. When the reproduction instruction is given (YES), the control unit 20 reproduces the video signal and the stereo audio signal in step S302.

In step S303, the control unit 20 determines whether or not there is an instruction to display the pointer image 51. Here, an operation when the pointer image 51 is used will be described. Control unit 2
0 indicates that if there is an instruction (YES), the process proceeds to step S304, and if there is no instruction (NO),
The process proceeds to step S316. In step S304, the control unit 20 performs the pointer image 51.
Is displayed at the initial position.

In step S305, the control unit 20 determines whether there is an instruction to move the pointer image 51. If there is an instruction to move the pointer image 51 (YES), the control unit 20 moves the pointer image 51 in step S306. If there is no instruction to move the pointer image 51 (NO), the control unit 20 shifts the process to step S307.

In step S307, the control unit 20 determines whether or not an instruction to reproduce the audio signal in the selected angular direction has been issued. The control unit 20 shifts the process to step S308 when an instruction to reproduce the audio signal in the selected angular direction is given, and shifts the process to step S317 when the instruction is not given.

In step S317, the control unit 20 determines whether there is an instruction to cancel the display of the pointer image 51. For example, if the return key on the operation unit 30 is operated, it may be assumed that there is an instruction to cancel the display of the pointer image 51. If there is an instruction to cancel the display of the pointer image 51 (YES
) In step S318, the control unit 20 cancels the display of the pointer image 51, and shifts the processing to step S316. If there is no instruction to cancel the display of the pointer image 51 (NO), the control unit 2
If 0, the process returns to step S305.

In step S308, the control unit 20 determines a track number to be reproduced from the position of the pointer image 51. In step S309, the control unit 20 reproduces the audio signal having the determined track number. In step S310, the control unit 20 determines whether or not an instruction to stop the reproduction of the audio signal of the track number being reproduced has been issued. If an instruction to stop reproduction is given (YES), the control unit 20 stops reproduction of the audio signal being reproduced in step S311 and returns the process to step S305.

If an instruction to stop reproduction is not made (NO), the control unit 20 determines whether or not there is an instruction to move the pointer image 51 in step S312. The control unit 20 displays the pointer image 5
If there is an instruction to move 1 (YES), the process returns to step S308, and if there is no instruction to move the pointer image 51 (NO), the process proceeds to step S313.

In step S313, the control unit 20 determines whether there is an instruction to cancel the display of the pointer image 51. If there is no instruction to cancel the display of the pointer image 51 (NO), the control unit 20 returns the process to step S309. If there is an instruction to cancel the display of the pointer image 51 (YES), the control unit 2
0 cancels the display of the pointer image 51 in step S314, stops the reproduction of the audio signal being reproduced in step S315, and shifts to step S316.

In step S316, the control unit 20 determines whether or not an instruction to stop reproduction of content data has been issued. If no instruction to stop playback is given (NO), the control unit 20 steps the process.
Return to S302. If an instruction to stop playback is given (YES), the control unit 20 proceeds to step S319.
The reproduction of the video signal and the stereo audio signal is stopped, and the process proceeds to step S320.

In step S320, control unit 20 determines whether or not an instruction to turn off the power has been issued. If an instruction to turn off the power is given (YES), the control unit 20 ends the process. If no instruction to turn off the power is given (NO), the control unit 20 returns the process to step S301.

By the way, in the above-described step S309, it has been described that the audio signal of the determined track number is simply reproduced. In general, the television receiver 300 shown in FIG. 18 is a stereo speaker, and a stereo speaker is used to handle an audio signal in a selected angle direction among multi-channel audio signals as if it were a viewing screen. It is necessary to reproduce so that sound can be heard from the angle direction.

In general, in 2-channel stereo, sound image localization can be controlled by providing a difference in sound intensity. A certain audio signal has a weighting factor w _{L for the} left (L) channel and a right (R) channel.
w When distributed by multiplying by the weighting factor of _R , if the angle from the front to the speaker centered at the listening point is φ ₀ and the angle from the front to the sound localization position is φ (positive from the front to the right) It is generally known that the following relationship is satisfied (see Non-Patent Document 1, p. 34). Where φ
₀ is preferably close to the viewing angle of the display device.
sinφ / sinφ ₀ = (w _L −w _R ) / (w _L + w _R ) (3)

Using this, the audio output processing unit 22 converts the audio signal of the selected channel among the multi-channel audio signals into the L channel and the R channel in the stereo audio signal at a ratio according to the angular direction in the horizontal direction. For example, the distribution may be as follows.

If the number of multi-channel channels is N, and the channel number n is 1, 2, 3,... In order from the leftmost channel of the multi-channel, equation (3) becomes the following equation (4).

For example, consider the case where the number of channels N is 7 in the horizontal direction. Here, THX (registered trademark of THX) recommends a viewing angle of 2φ ₀ = 36 ° to 40 °, and it is assumed that viewing is performed at a viewing angle of 40 °.
When the first channel from the front to the left, that is, the third channel from the left end, is selected, the angle between the speakers is set to 2φ ₀ = 40 °, and the variable portion of the left side sine function is φ ₀ {(n- 1)-(N-1) / 2}
From / N, 40 ° · {(3-1)-(7-1) / 2} / 7 = approx. -5.71 °, that is, approx. From Equation (4), φ = 5.71 ° and w _L : w _R = 1.29: 0.71 = 0.
645: 0.355. In other words, the audio signal of the selected channel is 35.5% for the L channel,
Allocate to R channel at 64.5%.

When the second channel from the front to the right, that is, the sixth channel from the left end is selected, similarly, it is sufficient to assign 21% to the L channel and 79% to the R channel from the equation (4).

In this way, the stereo audio signal and the audio signal of the selected channel among the multi-channel audio signals are naturally synthesized, and the audio at the selected position (direction) can be accurately reproduced. According to the present embodiment, when the playback zoom is performed, the zoomed-up video can be associated with the reproduced audio, and the user hardly feels uncomfortable.

In the case of a multi-channel audio signal having a plurality of channels in the vertical direction, for example, using a multi-channel speaker in which a plurality of speakers are arranged in the vertical direction, similarly, the ratio is appropriately set for the plurality of speakers and distributed. Just do it.

The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the present invention.

10, 10A, 10B Array microphone 12, 12 ′ Audio signal calculation unit 14 Directivity switching unit 17 Zoom adjustment unit 18 Video photographing unit 20 Control unit 22 Audio output processing unit 23 Speaker 25 Video composition unit 26 Image data generation unit 28 Display unit 61 Voice recognition unit 62 Sound pressure level calculation unit (sound collection level calculation unit)
63 Frequency spectrum analyzer (sound collection level calculator)
64 Speech recognition level calculation unit (sound collection level calculation unit)

Claims (6)

  An image data generating device for generating image data of a display image obtained by changing a sound collection level display image indicating a sound collection level of an audio signal in accordance with a set amplification level of the audio signal. A sound collection level calculation unit for calculating the sound collection level of the audio signal;
An audio signal amplifier for setting the amplification degree of the audio signal;
An image data generating unit that generates image data of a display image obtained by changing the sound collection level display image indicating the sound collection level calculated by the sound collection level calculation unit according to the amplification degree set by the audio signal amplification unit. When,
An image data generating device comprising:   An image data generating device for generating image data of a voice recognition level display image indicating a voice recognition level indicating a recognition degree of a human voice included in a voice signal. A voice recognition level calculation unit for calculating a voice recognition level indicating a degree of recognition of a human voice included in the voice signal;
An image data generator for generating image data of a voice recognition level display image indicating the voice recognition level calculated by the voice recognition level calculator;
An image data generating device comprising:   An image data generation method for generating image data of a display image obtained by changing a sound collection level display image indicating a sound collection level of an audio signal in accordance with a set amplification level of the audio signal.   An image data generation method for generating image data of a voice recognition level display image indicating a voice recognition level indicating a recognition degree of a human voice included in a voice signal.

Images