JP7337491B2 - AUDIO VIDEO PROCESSING SYSTEM AND CONTROL METHOD FOR VIDEO AUDIO PROCESSING SYSTEM - Google Patents

Description

The present invention relates to a video/audio processing system and a control method for the video/audio processing system.

In Patent Document 1, an imaging apparatus includes an imaging unit that acquires an image, an image display unit that displays an image, and a microphone unit that converts an input external sound into a plurality of audio signals and outputs the signals. The imaging device designates a specific subject in the image displayed on the display unit, and determines the direction of the specific subject from the imaging unit based on the coordinate information of the designated subject and the shooting angle of view. Calculate Then, based on the calculated direction of the specific subject, the imaging device generates a synthesized sound signal corresponding to the direction of the specific subject from the plurality of sound signals, and outputs the synthesized sound signal to the outside.

JP 2008-193196 A

In Patent Document 1, it is assumed that one microphone is prepared for one camera. In order for multiple cameras to record along with recording, you will need to connect a microphone to every camera you want to record, or have a camera with a built-in microphone. If the user prepares as many microphones as the number of cameras, the cost will increase, so the user will give up the recording function.

Patent Document 1 assumes that the positions of the imaging unit and the microphone are fixed, and that the user cannot change the relative position of the microphone with respect to the imaging unit. Therefore, when the user decides the number of imaging units and microphones and installs them in arbitrary positions, the direction of the subject from the imaging unit and the direction of the subject from the microphones will differ depending on the installation location. put away. In this case, it is not possible to specify the sound collection direction in accordance with the image of the camera and distribute the sound.

SUMMARY OF THE INVENTION An object of the present invention is to appropriately associate a plurality of video data with a plurality of audio data.

In the video/audio processing system of the present invention, a plurality of imaging devices for generating a plurality of video data and a plurality of sound collection ranges for performing sound directivity processing are set, and a sound collection unit collects data for each sound collection range. an audio input device that generates audio data for each of the sound collection ranges by performing directivity processing on the sounded audio; and a processing device, wherein the audio input device receives the generated audio data. is set, and the plurality of generated audio data and the IP address of the communication device corresponding to the audio data are transmitted to the plurality of imaging devices; each of the imaging devices stores the IP address of the communication device corresponding to its own imaging device, and the audio data received from the audio input device together with the IP address matching the stored IP address; and video data captured by an imaging device, the generated file is transmitted to the processing device, and the processing device receives the file.

According to the present invention, it is possible to appropriately associate a plurality of video data and a plurality of audio data.

It is a figure which shows the structural example of an audio-video processing system. It is a figure which shows the structural example of a camera, a microphone, and a server apparatus. It is a figure which shows CPU, ROM, and RAM. FIG. 2 is a diagram showing how sound from a sound source reaches a microphone; It is a figure which shows the imaging|photography range of a camera, and the sound collection range of a microphone. It is a figure which shows the memory|storage part which memorize|stores ID, an IP address, and directivity conditions. 4 is a flow chart showing a control method of the video/audio processing system; 4 is a flow chart showing a control method of the video/audio processing system; It is a figure which shows the calculation method of the polar coordinate of the imaging|photography range of a camera. FIG. 4 is a diagram showing the directional direction and directional range of a microphone;

(First embodiment)
FIG. 1(a) is a diagram showing a configuration example of a video/audio processing system 100 according to the first embodiment of the present invention. The audiovisual processing system 100 has a camera 1101 , a camera 1102 , a camera 1103 , a microphone 2000 , a server device 3000 and a network 4000 . Camera 1101 , camera 1102 , camera 1103 , microphone 2000 and server device 3000 can communicate with each other via network 4000 . The number of microphones 2000 is less than the number of cameras 1101-1103.

A camera 1101 is a network camera that generates video data and transmits the video data via the network 4000, and corresponds to an imaging device. Cameras 1102 and 1103 are each similar to camera 1101 . Note that the audiovisual processing system 100 may include a plurality of cameras in addition to the cameras 1101-1103. The cameras 1101-1103 may include zoom drive mechanisms, pan drive mechanisms, tilt drive mechanisms, and the like.

Microphone 2000 is a network microphone that inputs voice and transmits the input voice via network 4000, and corresponds to a voice input device. FIG. 1B is a diagram showing a configuration example of the microphone 2000. As shown in FIG. Microphone 2000 has a plurality of microphones (sound collecting units) 2011-2018. The number of microphones 2011-2018 is not limited to eight, and may be increased or decreased.

Server device 3000 can communicate with cameras 1101 to 1103 and microphone 2000 via network 4000 . Cameras 1101 to 1103, microphone 2000, and server device 3000 each transmit various commands to other devices via network 4000, and when commands are received, they transmit responses to the transmitting device. The server device 3000 is an example of a processing device such as a personal computer (PC).

The network 4000 is composed of a plurality of routers, switches, cables, etc. that satisfy communication standards such as Ethernet (registered trademark). As long as the network 4000 can communicate between the cameras 1101 to 1103, the microphone 2000, and the server apparatus 3000, its communication standard, scale, and configuration are not limited. For example, the network 4000 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like. The cameras 1101 to 1103 may be, for example, monitoring cameras compatible with PoE (Power Over Ethernet (registered trademark)), and may be supplied with power via LAN cables.

FIG. 2 is a diagram showing a configuration example of the camera 1101, the microphone 2000, and the server device 3000. As shown in FIG. The camera 1101 has an imaging unit 1001 , an image processing unit 1002 , a control unit 1003 , a communication unit 1004 and a storage unit 1005 . Cameras 1102 and 1103 have the same configuration as camera 1101 .

The image pickup unit 1001 has a lens and an image pickup device such as a CCD or CMOS, picks up an image of a subject with an angle of view determined by lens settings or the like, and generates a video signal by photoelectric conversion. The image processing unit 1002 performs predetermined image processing and compression encoding processing on the video signal generated by the imaging unit 1001 to generate video data. Note that the control unit 1003 controls the imaging unit 1001 and the image processing unit 1002 based on imaging conditions set by the user or imaging conditions automatically determined by the control unit 1003 . Here, the imaging conditions include imaging gain conditions, gamma conditions, dynamic range conditions, exposure conditions, focus conditions, and the like.

The control unit 1003 has a CPU 301, a ROM 302, and a RAM 303, as shown in FIG. The control unit 1003 analyzes camera control commands received from the microphone 2000, the server device 3000, and the like via the network 4000, and performs processing according to the camera control commands. For example, the control unit 1003 instructs the image processing unit 1002 to adjust image quality, instruct zoom and focus control, instruct pan/tilt operation, and combine and transmit audio data and video data. Also, the control unit 1003 has a CPU 301, controls each component of the camera 1101 in an integrated manner, and sets various parameters. The control unit 1003 also has a ROM 302 and a RAM 303 that store data, and executes programs stored in the RAM 302 or ROM 303 . The RAM 303 has a storage area for programs executed by the control unit 1003, a work area during program execution, a data storage area, and the like. In addition, the control unit 1003 has a timer, and can add a time stamp or the like to each acquired data.

The communication unit 1004 receives audio data transmitted by the microphone 2000 via the network 4000 , performs appropriate packet processing, and then outputs the data to the control unit 1003 . Also, the communication unit 1004 receives a command from the microphone 2000 and transmits a response to the received command to the microphone 2000 . Communication unit 1004 also transmits video data to server device 3000 via network 4000 . Also, the communication unit 1004 receives a camera control command transmitted by the server device 3000 , performs appropriate packet processing and the like, and then outputs the command to the control unit 1003 . Also, the communication unit 1004 transmits to the server device 3000 a response to the command received from the server device 3000 .

Storage unit 1005 stores information for associating video data generated by camera 1101 with audio data generated by microphone 2000 .

Next, with reference to FIG. 2, the configuration and function of each part of microphone 2000 will be described. Microphone 2000 has sound collecting section 2001 , sound processing section 2002 , control section 2003 , communication section 2004 and storage section 2005 . It should be noted that each of the eight microphones 2011 to 2018 has at least a separate sound collecting section 2001 .

The sound collector 2001 is composed of electrodes such as a vibrating membrane and a fixed plate. When the distance between the electrodes changes according to the vibration of the vibrating membrane due to the sound pressure, the voltage fluctuates, converting the sound into an electric audio signal. Convert. Also, the sound collector 2001 may include an amplifier for amplifying the voltage of the audio signal.

The audio processing unit 2002 performs audio processing and compression encoding processing on the audio signal generated by the sound collecting unit 2001 to generate audio data. Note that the control unit 2003 controls the sound collecting unit 2001 and the sound processing unit 2002 based on the voice input condition set by the user or the voice input condition automatically determined by the control unit 2003 . Here, the audio input conditions include a volume gain condition, an audio frequency characteristic condition, an audio orientation direction condition, an audio orientation range condition, and the like.

The control unit 2003 has a CPU 301, a ROM 302, and a RAM 303, as shown in FIG. The control unit 2003 analyzes control commands received from the camera 1101, the server device 3000, etc. via the network 4000, and performs processing according to the control commands. For example, the control unit 2003 instructs control of the transmission destination of audio data that has undergone audio processing. Also, the control unit 2003 has the CPU 301, controls each component of the microphone 2000 in an integrated manner, and sets various parameters and the like. The control unit 2003 also has a ROM 302 and a RAM 303 that store data, and executes programs stored in the ROM 302 or the RAM 303 . The RAM 303 has a storage area for programs executed by the control unit 2003, a work area during program execution, a data storage area, and the like. In addition, the control unit 2003 has a timer, and can add a time stamp or the like to each acquired data. Further, the control unit 2003 performs directivity processing (signal processing that emphasizes sound from the target direction and suppresses sound from other Outputs the processed audio signal.

Directivity processing will be described with reference to FIGS. 4(a) and 4(b). FIG. 4A shows how sound from a sound source reaches the microphones 2011 and 2012 from the direction of angle θ. The microphone 2011 and the microphone 2012 are arranged with a distance D2. In this case, the difference L between the distance between the sound source and the microphone 2011 and the distance between the sound source and the microphone 2012 is expressed by the following equation.
L=D2×cos θ

Assuming that the speed of sound is V, the time T from when the sound from the sound source reaches the microphone 2011 to when the sound from the sound source reaches the microphone 2012 is expressed by the following equation.
T=L/V=D2×cos θ/V

FIG. 4B shows the values of L and T and the difference in T with respect to the angle θ when D2=50 mm and V=346.75 m/s. For example, when θ=0 deg, L=50 mm and T=144 μs. When θ=15 deg, L=48 mm and T=139 μs. The difference in T between θ=0 deg and θ=15 deg is 5 μs.

Based on the time T, the control unit 2003 performs calculations for directivity processing. For example, a case will be described where it is desired to emphasize the sound in the front direction (90 degrees) (to provide directivity). In this case, the control unit 2003 performs a calculation for emphasizing the sound (sound of T=0 μs) arriving at the microphones 2011 and 2012 at the same time and suppressing the sound (sound of T≠0 μs) arriving with a time lag.

Communication unit 2004 transmits the audio data to server device 3000 via network 4000 . Also, the communication unit 2004 receives a control command transmitted from the server device 3000 , performs appropriate packet processing and the like, and then outputs the control command to the control unit 2003 . The communication unit 2004 also transmits a response to the command received from the server device 3000 to the server device 3000 .

Storage unit 2005 stores information for associating video data generated by cameras 1101 to 1103 with audio data generated by microphone 2000 .

Next, with reference to FIG. 2, the configuration and function of each part of the server device 3000 will be described. Server device 3000 is, for example, a general-purpose computer such as a personal computer. The server device 3000 has a communication section 3001 , a system control section 3002 and a storage section 3003 .

Communication unit 3001 receives video data from cameras 1101 to 1103 and audio data from microphone 2000 via network 4000 . The communication unit 3001 also transmits control commands to the cameras 1101 to 1103 or the microphone 2000 and receives responses to these control commands.

The system control unit 3002 has a CPU 301, a ROM 302 and a RAM 303, as shown in FIG. A system control unit 3002 generates camera control commands according to user operations, and transmits the camera control commands to the cameras 1101 to 1103 via the communication unit 3001 . Also, the system control unit 3002 stores video data received from the cameras 1101 to 1103 via the communication unit 3001 in the storage unit 3003 . Also, the system control unit 3002 has a CPU 301, controls each component of the server apparatus 3000 in an integrated manner, and sets various parameters. The system control unit 3002 also has a ROM 302 and a RAM 303 that store data, and executes programs stored in the ROM 302 or the RAM 303 . The RAM 303 has a storage area for programs executed by the system control unit 3002, a work area during program execution, a data storage area, and the like. Further, the system control unit 3002 has a timer, and can add a time stamp or the like to each acquired data.

Storage unit 3003 stores data acquired by cameras 1101 to 1103 and microphone 2000 . The system control unit 3002 reads data stored in the storage unit 3003 and transfers it.

FIG. 5 shows the arrangement of cameras 1101 to 1107 and microphone 2000 installed in room 5000, imaging ranges 1201 to 1207 of cameras 1101 to 1107, and sound collection ranges A to H of microphone 2000. FIG. Cameras 1102 to 1107 have the same configuration as camera 1101 and are connected to network 4000 . The number of microphones 2000 is less than the number of cameras 1101-1107. A method for determining the sound collection range in directivity processing of the microphone 2000 will be described with reference to FIG. When the cameras 1101 to 1107 are installed in a room 5000, the server device 3000 displays the room 5000 as seen from above as shown in FIG. 5 by the camera management software. The server device 3000 displays the shooting ranges 1201-1207 of the cameras 1101-1107, and automatically sets the shooting ranges 1201-1207 according to the user's settings or the angles of view of the cameras 1101-1107. The areas of the sound collection ranges A, B, C, D, E, F, G, and H in which the directivity of the microphone 2000 can be set are divided, and one of the sound collection ranges A to H is selected for the microphone 2000. By doing so, it is possible to specify the sound collection area. For example, if the directivity of the microphone 2000 is designated as sound collection ranges A and B, only sounds from sound sources within the sound collection ranges A and B can be collected. Sound collection ranges A through H correspond to microphones 2011 through 2018 . The microphone 2000 collects sounds from multiple locations or from multiple directions, and performs directional processing on the sounds from multiple locations or multiple directions to generate multiple audio data corresponding to multiple cameras 1101 to 1107. do.

6A to 6E are diagrams showing correspondence relationships between IDs, IP addresses, and directivities for associating video data from cameras 1101 to 1107 with audio data from microphone 2000. FIG. In this embodiment, one IP address is assigned to each of cameras 1101-1107, and microphone 2000 is provided with a plurality of communication devices so as to have a plurality of IP addresses. As shown in FIG. 6A, one ID is assigned to one camera. For example, the ID of the camera 1101 is 1. The ID of the camera 1102 is 2. The ID of the camera 1103 is 3. The ID of the camera 1104 is 4. The ID of the camera 1105 is 5. The ID of the camera 1106 is 6. The ID of the camera 1107 is 7.

FIG. 6A is a diagram of a table showing the correspondence between the IDs of the cameras 1101 to 1107 and IP addresses (identification information). FIG. 6B is a diagram of a table showing the directivity sound collection range of the microphone 2000 for each ID of the cameras 1101-1107. The directivity sound collection range of the microphone 2000 is information indicating the directivity of the microphone 2000 . For example, for ID “1” of camera 1101 , directional sound collection ranges A and B corresponding to imaging range 1201 of camera 1101 are set.

FIG. 6(c) is a table showing the correspondence between the IDs of the cameras 1101 to 1107 and the IP addresses (identification information) of the communication devices corresponding to the audio data. For example, ID "1" of camera 1101 is associated with the IP address of a communication device for outputting sound data of sound collection ranges A and B. FIG. Also, IP addresses of communication devices for outputting sound data of sound collection ranges A and H are associated with ID "2" of camera 1102 . FIG. 6(d) is a diagram of a table showing the correspondence relationship between the IP addresses of cameras for the IDs of cameras 1101 to 1107 and the IP addresses of communication devices that output the corresponding audio data. FIG. 6(e) shows the correspondence between the ID and IP address of the camera and the ID and IP address of the corresponding communication device that outputs the audio data, and shows an example where the camera ID is "1". .

7A to 7C are flow charts showing the control method of the video/audio processing system 100. FIG. FIG. 7(a) is a flow chart showing a control method of the microphone 2000. FIG. In step S701, the sound collecting units 2001 of the microphones 2011 to 2018 collect sounds and convert the sounds into electrical sound signals. The audio processing unit 2002 performs audio processing and compression encoding processing on the audio signal to generate audio data.

In step S702, the control unit 2003 refers to the table of FIG. 6B stored in the memory 2005, and reads out the directivity sound collection range of the microphone 2000 corresponding to the ID of each of the cameras 1101-1107.

In step S703, the control unit 2003 performs directivity processing on the audio data of the microphones 2011 to 2018 based on the sound collection range of the IDs of the cameras 1101 to 1107 read out in step S702. to generate audio data. For example, the control unit 2003 performs directivity processing of the directional sound collection ranges A and B of the microphone 2000 on the ID of the camera 2011 to generate audio data of the ID of the camera 2011 .

In step S704, the control unit 2003 refers to the table of FIG. 6C stored in the storage unit 2005, and reads the IP address of the audio data corresponding to the ID of each camera 1101-1107.

In step S705, the control unit 2003 sets the IP address of the communication device that outputs audio data corresponding to the IDs of the cameras 1101 to 1107 read out in step S704, and the audio data of each ID generated in step S703. Associate. Control unit 2003 then transmits the associated IP address and voice data to server device 3000 .

FIG. 7B is a flow chart showing a control method for the cameras 1101-1107. In step S711, the image capturing units 1001 of the cameras 1101 to 1107 capture images of subjects and generate video signals. The image processing units 1002 of the cameras 1101 to 1107 perform image processing and compression encoding processing on video signals to generate video data.

In step S712, the control units 1003 of the cameras 1101 to 1107 refer to the table of FIG. 6A stored in the storage unit 1005, and read out the IP address corresponding to the ID of the own camera. The control unit 1003 of each of the cameras 1101 to 1107 associates the IP address corresponding to its own camera ID with the video data generated by the camera having its own ID, and transmits them to the server device 3000 .

FIG. 7(c) is a flow chart showing a control method of the server apparatus 3000. FIG. In step S721, the system control unit 3002 receives the IP address of each ID and video data from the cameras 1101 to 1107 via the communication unit 3001. FIG. Also, the system control unit 3002 receives, via the communication unit 3001, the IP address of the communication device that outputs the audio data corresponding to each ID and the audio data from the microphone 2000. FIG.

In step S722, based on the table of FIG. 6D stored in the storage unit 3003, the system control unit 3002 communicates to output the IP addresses corresponding to the IDs of the cameras 1101 to 1107 and the corresponding audio data. Read the device's IP address combination.

In step S723, the system control unit 3002 associates the corresponding video data and audio data based on the IP address of the camera with the same ID and the IP address of the audio data, and records them in the storage unit 3003 as an MPEG file. The system control unit 3002 can reproduce an MPEG file containing video data and audio data based on the IP address of the camera.

(Second embodiment)
In the first embodiment, as shown in FIG. 7C, the server apparatus 3000 associates video data and audio data with the same ID. In the second embodiment of the present invention, cameras 1101-1107 associate video data and audio data with the same ID. The configurations and connections of the cameras 1101 to 1107, the microphone 2000, and the server device 3000 are the same as those of the first embodiment, and thus description thereof is omitted. Further, since directivity processing is also the same as that of the first embodiment, description thereof is omitted. Differences of this embodiment from the first embodiment will be described below.

FIGS. 8A to 8C are flow charts showing the control method of the video/audio processing system 100 according to the second embodiment of the present invention. FIG. 8(a) is a flow chart showing a control method of the microphone 2000. FIG. The microphone 2000 performs steps S801 to S804. The processing of steps S801 to S804 is the same as the processing of steps S701 to S704 in FIG. 7(a). In step S805, the control unit 2003 associates the IP addresses of the audio data corresponding to the IDs of the cameras 1101 to 1107 read out in step S804 with the audio data of the IDs generated in step S803. ~1107.

FIG. 8B is a flow chart showing a control method for the cameras 1101-1107. As shown in FIG. 6E, the storage unit 1005 of the camera 1101 stores a table of combinations of IP addresses and audio data corresponding to the ID of the camera 1101 of its own. Similarly, the storage units 1005 of the cameras 1102 to 1107 each store a table of combinations of IP addresses and audio data corresponding to the IDs of the cameras 1101 of their own.

In step S811, the image capturing units 1001 of the cameras 1101 to 1107 capture images of subjects and generate video signals. The image processing units 1002 of the cameras 1101 to 1107 perform image processing and compression encoding processing on video signals to generate video data.

In step S812, the control unit 1003 of the camera 1101 refers to the table shown in FIG. read out. Similarly, the control unit 1003 of each of the cameras 1102 to 1107 refers to the table stored in the storage unit 1005, and determines the IP address of the communication device that outputs the audio data corresponding to the ID of the camera 1102 to 1107. read out.

In step S813, the control units 1003 of the cameras 1101 to 1107 receive from the microphone 2000 audio data output from the communication device having the IP address corresponding to the ID of the camera 1102 to 1107 read out in step S812. The control units 1003 of the cameras 1101 to 1107 advance to step S814 if the audio data can be received, and advance to step S815 if the audio data cannot be received. In step S<b>815 , control unit 1003 transmits the video data generated in step S<b>811 and the IP address corresponding to the ID of its own camera to server apparatus 3000 via communication unit 1004 .

In step S814, the control unit 1003 of each of the cameras 1101 to 1107 communicates to output the IP address corresponding to the ID of its own camera and the corresponding audio data based on the table shown in FIG. Read the device's IP address combination. The control unit 1003 of each of the cameras 1101 to 1107, based on the IP address corresponding to the ID of the own camera and the IP address of the communication device that outputs the audio data, determines the video data (S811) and the audio data (S811) of the own camera ID. S813) is associated. Then, the control units 1003 of the cameras 1101 to 1107 generate MPEG files containing the associated video data and audio data. After that, the control units 1003 of the cameras 1101 to 1107 proceed to step S815. In step S<b>815 , control unit 1003 transmits to server device 3000 via communication unit 1004 the MPEG file containing the video data and audio data of its own camera ID and the IP address of the video data.

FIG. 8(c) is a flow chart showing a control method of the server apparatus 3000. FIG. In step S821, the system control unit 3002 transmits, via the communication unit 3001, the MPEG file containing the video data and audio data of each ID and the IP address of the camera, or the video data of each ID and the IP address of the camera from the cameras 1101 to 1107. receive an address. The system control unit 3002 can reproduce an MPEG file containing video data and audio data or video data based on the IP address of the camera.

In the first and second embodiments, the IP addresses of the cameras 1101 to 1107 and the IP addresses of the communication devices of the microphone 2000 are used to associate video data and audio data, but the present invention is not limited to this. . If the microphone 2000 does not have a plurality of communication devices, the IDs of the cameras 1101 to 1107 may simply be used as identification information and associated with the directional ranges for collecting sound. That is, in this case, one may only have FIGS. 6(a) and (b).

(Third embodiment)
FIG. 9(a) is a diagram showing a method of calculating the polar coordinates of the shooting range 1201 of the camera 1101 with respect to the position of the microphone 2000 according to the third embodiment of the present invention. The server device 3000 calculates the polar coordinates of the shooting range 1201 of the camera 1101, and calculates the directivity direction and the directivity range of the camera 1101 based on the polar coordinates. The differences of this embodiment from the first and second embodiments will be described below.

First, the system control unit 3002 sets the position coordinates a (Xa, Ya) of the camera 1101, the shooting direction of the camera 1101, the shooting angle of the camera 1101, and the shooting distance of the camera 1101 in accordance with the user's instruction. . Next, system control section 3002 sets the position coordinates of microphone 2000 according to the user's instruction.

Next, the system control unit 3002 calculates vertex coordinates a (Xa, Ya), b (Xb, Yb), and c (Xc, Yc) of the shooting range 1201 of the camera 1101 based on the above information. Next, the system control unit 3002 converts the vertex coordinates a (Xa, Ya), b (Xb, Yb), and c (Xc, Yc) of the shooting range 1201 with respect to the position coordinates of the microphone 2000 into polar coordinates (ra, θa), (rb, θb), and (rc, θc).

The system control unit 3002 converts the vertex coordinates of the photographing ranges 1202 to 1207 of the cameras 1102 to 1107 with respect to the position coordinates of the microphone 2000 into polar coordinates in the same manner as described above.

FIG. 9B is a diagram showing another method of calculating the polar coordinates of the imaging range 1201 of the camera 1101 with respect to the microphone 2000 according to this embodiment. The server device 3000 calculates the polar coordinates of the shooting range 1201 of the camera 1101, and calculates the directivity direction and the directivity range of the camera 1101 based on the polar coordinates.

First, the system control unit 3002 sets the coordinates of the installation area of the camera 1101 according to the user's instruction. Next, system control section 3002 sets the position coordinates of microphone 2000 according to the user's instruction.

Next, based on the above information, the system control unit 3002 calculates vertex coordinates a (Xa, Ya), b (Xb, Yb), c (Xc, Yc), d (Xd, Yd) is calculated. The system control unit 3002 converts the vertex coordinates a (Xa, Ya), b (Xb, Yb), c (Xc, Yc), and d (Xd, Yd) into polar coordinates based on the position coordinates of the microphone 2000 according to the following equation. Convert to (ra, θa), (rb, θb), (rc, θc), (rd, θd).

The system control unit 3002 converts the vertex coordinates of the photographing ranges 1202 to 1207 of the cameras 1102 to 1107 with respect to the position coordinates of the microphone 2000 into polar coordinates in the same manner as described above.

FIG. 10A is a diagram showing the directivity direction θ1 and the directivity range φ1 of the microphone 2000 corresponding to the photographing range 1201 of the camera 1101. FIG. After calculating the polar coordinates of FIG. 9A or 9B, the server device 3000 calculates the directivity direction θ1 and the directivity range φ1 of the microphone 2000 corresponding to the shooting range 1201 of the camera 1101 .

First, the system control unit 3002 calculates angles of two straight lines 901 and 902 connecting the position coordinates of the microphone 2000 and both ends of the imaging range 1201 . Next, the system control unit 3002 calculates the average angle θ1 of the angles of the two straight lines 901 and 902 as the directivity direction of the microphone 2000 corresponding to the shooting range 1201 of the camera 1101 . Next, the system control unit 3002 calculates an angle φ1, which is the angle difference between the two straight lines 901 and 902, as the directivity range of the microphone 2000 corresponding to the shooting range 1201 of the camera 1101. FIG.

The system control unit 3002 calculates the directivity directions θ2 to θ7 and the directivity ranges φ2 to φ7 of the microphone 2000 corresponding to the photographing ranges 1202 to 1207 of the cameras 1102 to 1107 in the same manner as described above.

FIG. 10(b) is a diagram of a table showing the directivity direction and directivity range of the microphone 2000 for each ID of the cameras 1101-1107. The directional direction and directional range of microphone 2000 are information indicating the directivity of microphone 2000 . The server device 3000 generates the table of FIG. 10(b) after the calculation of FIG. 10(a). After the calculation in FIG. 10(a), the system control unit 3002 indicates the directivity directions θ1 to θ7 and the directivity range φ1 to φ7 of the microphone 2000 with respect to the IDs of the cameras 1101 to 1107 as shown in FIG. 10(b). Generate a table. The table of FIG. 10(b) is used instead of the table of FIG. 6(b). Server device 3000 transmits the table of FIG. 10B to microphone 2000 . The microphone 2000 stores the table of FIG. 10B in the storage unit 2005, and performs the directivity processing of step S703 of FIG. 7A based on the table of FIG. 10B.

According to the first to third embodiments, the number of microphones 2000 is less than the number of cameras 1101-1107, and the user can arbitrarily specify the installation locations of the microphones 2000 and cameras 1101-1107. The video/audio processing system 100 can collect sounds matched with the video images of the cameras 1101 to 1107 and appropriately associate (combine) the video data and audio data of the cameras 1101 to 1107 .

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

It should be noted that the above-described embodiments are merely examples of specific implementations of the present invention, and the technical scope of the present invention should not be construed to be limited by these. That is, the present invention can be embodied in various forms without departing from its technical concept or main features.

1101-1107 cameras, 2000 microphones, 3000 server devices, 4000 networks

Claims (3)

a plurality of imaging devices that generate a plurality of video data;
A plurality of sound collection ranges are set for sound directivity processing, and sound data is generated for each sound collection range by performing directivity processing on sound collected by the sound collection unit for each sound collection range. a voice input device that
a processor;
has
The voice input device is set with an IP address of a communication device corresponding to each of the plurality of generated voice data, and the plurality of generated voice data and the IP address of the communication device corresponding to the voice data. and transmitting to the plurality of imaging devices,
Each of the plurality of imaging devices stores an IP address of the communication device corresponding to its own imaging device, and audio data received from the audio input device together with an IP address matching the stored IP address; generating a file containing video data captured by its own imaging device, and transmitting the generated file to the processing device;
The audio/video processing system, wherein the processing device receives the file. Each of the plurality of imaging devices stores an IP address of the communication device corresponding to its own imaging device, and audio data received from the audio input device together with an IP address matching the stored IP address; generating a file containing video data captured by its own imaging device, and transmitting the generated file to the processing device together with the IP address of its own imaging device;
2. The audio/video processing system according to claim 1 , wherein said processing device receives said file and an IP address of said own imaging device and reproduces said file. Generate a plurality of video data by a plurality of imaging devices,
A plurality of sound collection ranges are set by the sound input device for performing sound directivity processing, and for each of the sound collection ranges, the sound collected by the sound collection unit is subjected to directivity processing, so that for each sound collection range generate audio data of
An IP address of a communication device corresponding to each of the plurality of generated voice data is set by the voice input device, and the plurality of generated voice data and the IP address of the communication device corresponding to the voice data are set. and transmitting to the plurality of imaging devices,
each of the plurality of image pickup devices stores an IP address of the communication device corresponding to the image pickup device, audio data received from the audio input device together with the IP address matching the stored IP address; generating a file containing video data captured by its own imaging device, and transmitting the generated file to the processing device;
A control method for an audiovisual processing system, wherein the file is received by the processing device.

Images