WO2008050620A1

WO2008050620A1 - Camera server and data processing method

Info

Publication number: WO2008050620A1
Application number: PCT/JP2007/070007
Authority: WO
Inventors: Katsura Obikawa; Jong-Sun Hur
Original assignee: Opt Corporation
Priority date: 2006-10-25
Filing date: 2007-10-12
Publication date: 2008-05-02
Also published as: JP2008109362A

Abstract

It is possible to distribute image data having little noise to a client and reduce the video disturbance displayed at the client side. A camera server (3) distributes image data on an image captured by an imaging element (imaging sensor) (25) of a camera device (2) to a client (5) via a network (6). The camera server (3) includes data selection means (38) which outputs image data to the network (6) side if the image data is judged to be predetermined image data, and does not output the image data to the network (6) side if the image data is judged to be other than predetermined image data.

Description

Specification

Camera server and data processing method

Technical field

The present invention relates to a camera server and a data processing method.

Background art

Conventionally, a camera server for distributing image data of an image captured by a camera device including an image sensor to a client via a network is known (for example, see Patent Document 1). The KARU Camera Server uses all the data signals output from the camera device.

It is configured to be distributed to the client side.

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2005-318412

Disclosure of the invention

Problems to be solved by the invention

However, a data signal output from the camera device may include a noise data signal that is not a data signal of image data. Therefore, the client is distributed not only for image data but also for a noise data signal. Therefore, when displaying a video on the client side based on the data transmitted from the camera server, there is a problem that the video is disturbed by a data signal of noise.

[0005] Accordingly, an object of the present invention is to provide a camera server and a data processing method for distributing image data with less noise to a client and reducing disturbance of an image displayed on the client side.

Means for solving the problem

[0006] In order to solve the above-mentioned problem, whether or not the image data is predetermined image data in a camera server that distributes image data of an image captured by a camera device including an image sensor to a client via a network. If the image data is determined to be predetermined image data, the image data is output to the network side. If the image data is determined not to be predetermined image data, the image data is output to the network side. Data selection means not to output is provided. In such a configuration, image data with less noise can be delivered to the client, so that the client can obtain image data with less noise.

[0008] In addition to the above-described invention, in another invention, the data selecting means determines whether or not the image data is predetermined based on information of a header added to the image data! To do

In such a configuration, the force S can be used to easily determine whether the image data is predetermined.

[0010] In addition to the above-described invention, in another invention, the camera server receives image data based on header information of image data and sound data when sound data is input in addition to the image data. Data synchronization means for synchronizing the sound data with the sound data, and the data synchronization means synchronizes the predetermined sound data of the sound data with the data portion determined by the data selection means to be not the predetermined image data. I will do it.

In such a configuration, image data with less noise can be delivered to the client, so that the client can obtain image data with less noise. As for sound data, sound data synchronized with image data that has not been distributed is also distributed. Therefore, the client can listen to the collected sound without interruption.

[0012] In order to solve the above-described problem, in a data processing method for distributing image data of an image captured by a camera device including an image sensor to a client via a network, the image data is predetermined image data. If the image data is determined to be the predetermined image data, the image data is output to the network side, and the image data determined not to be the predetermined image data is Do not output to the network side.

[0013] With such a configuration, image data with less noise can be delivered to the client, so the client can obtain image data with less noise.

[0014] In addition to the above-mentioned invention, in another invention, whether or not the image data is predetermined image data is determined based on header information added to the image data. [0015] In the case of such a configuration, the force S can be used to easily determine whether the image data is predetermined.

In order to solve the above-described problem, in a data processing method for distributing image data of an image captured by a camera device including an image sensor and sound data of sound collected by a microphone to a client via a network, For the image data, it is determined whether or not the image data is predetermined image data. The image data determined to be the predetermined image data is output to the network side, and the image data is determined to be the predetermined image data. Data that is determined not to be image data is processed not to be output to the network side, and the sound data and the image data output to the network side are processed based on the header information added to each data. The predetermined sound data of the sound data is also applied to the data portion that is synchronized with each other and is determined not to be the predetermined image data. And to synchronize the data.

When configured in this manner, image data with less noise can be delivered to the client, so that the client can obtain image data with less noise. As for sound data, sound data synchronized with image data that has not been distributed is also distributed. Therefore, the client can listen to the collected sound without interruption. The invention's effect

[0018] According to the present invention, it is possible to distribute image data with less noise to a client, and to reduce disturbance of video displayed on the client side.

Brief Description of Drawings

FIG. 1 is a diagram showing a configuration of a camera server system according to an embodiment of the present invention.

2 is a block diagram showing an electric circuit of a camera device in the camera server system shown in FIG.

3 is a block diagram showing an electric circuit of a camera server in the camera server system shown in FIG.

Explanation of symbols

[0020] 1 · · · Camera server system

2 · · · Camera equipment

3 · · · Camera server 6 · · · Network

25 · · · Image sensor

38 · · · Data selection part (data selection means)

39 · · · Data synchronization part (data synchronization means)

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, a camera server and a data processing method according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a system configuration diagram showing a camera server system 1 according to an embodiment of the present invention. The camera server system 1 includes a camera device 2 and a camera server 3. The camera device 2 and the camera server 3 are connected by a USB (Universal Serial Bus) cable 4. The camera server system 1 and the client 5 are connected via the network 6, and the camera server system 1 receives a command from the personal computer 7 on the client 5 side. The camera server 3 distributes image data of an image captured by the camera device 2 to the personal computer 7 (client 5) based on a command received from the personal computer 7. One or a plurality of clients 5 exist, and the camera server system 1 distributes the image data to each client 5 in accordance with a request from each client 5.

The camera device 2 has a cubic-shaped housing 8. Housing 8 has a fisheye lens

9, USB connector 10, audio 'video output connector 11, sound output connector 12, etc. The fisheye lens 9 is disposed on the upper surface of the housing 8. In addition to the fisheye lens 9, a ventilation hole 14 for the microphone 13 (see FIG. 2) is formed on the upper surface of the housing 8. The USB connector 10, the audio / video output connector 11 and the sound output connector 12 are arranged on the side surface of the housing 8.

The camera server 3 has a rectangular parallelepiped housing 15. The housing 15 includes a USB connector 16 as an input unit that is an input terminal for inputting data from the camera device 2, a microphone connector 18 to which an expansion microphone 17 to be used as needed is connected, an infrared sensor, and the like. General purpose input / output port (GPIO) 19 for connecting external equipment, audio output connector 20 for outputting audio signals, video output connector 21 for outputting video signals, and speakers to which speakers are connected A single connector 22, a communication interface 23 connected to the network, a power switch 24 that also serves as a reset switch, and the like are provided.

FIG. 2 is a block diagram showing an electric circuit of the camera device 2.

[0026] The camera device 2 includes a fisheye lens 9, a USB connector 10, an audio 'video output connector 11, a sound output connector 12, a microphone 13, an image sensor 25 as an image sensor, and an F PGA (Field Programmable Gate Array). Color conversion processing unit 26, sound processing unit 27 composed of sound IC (Integrated Circuit) 27, audio 'video encoder 28, streaming generation unit 29, and CPU (CCU) that controls the operation of each of the above components Central Processing Unit) 30 etc.

The microphone 13 collects sounds and sounds around the camera device 2. The microphone 13 generates a sound signal that is an electrical signal corresponding to the collected sound and outputs the sound signal to the sound processing unit 27. The sound processing unit 27 generates digital sound data based on the electrical signal from the microphone 13. The sound processing unit 27 outputs an electrical signal corresponding to the sound from the microphone 13 to the sound output connector 12 without converting it into sound data (digital format). Therefore, the sound collected by the microphone 13 can be heard by connecting, for example, a speaker, headphones or the like to the sound output connector 12.

[0028] For the image sensor 25, for example, a CMOS (Complementary Metal-Oxide Semiconductor) or the like is used. An image by the fisheye lens 9 is formed on the image sensor 25. The image sensor 25 outputs luminance distribution data, which is an imaging signal, to the color conversion processing unit 26 as digital image data. In the present embodiment, it is assumed that the image sensor 25 performs imaging at an imaging speed of 1/15 frames per second, for example. Therefore, one frame of image data is output from the image sensor 25 every 1/15 second.

[0029] The color conversion processing unit 26 replaces the data of each pixel of the luminance distribution data using a color conversion table (not shown) to generate digital image data. The generated image data is added with a header for each frame by a header adding unit 31 formed by the CPU 30. A header number M (M = 1, 2, 3,...), Which is a serial number, is assigned to the header according to the imaging order. The CPU 30 is configured with a JPEG encoder section 32 as an encoder that converts image data into JPEG (Joint Photographic Experts Group) open-type compressed image data. When the image data is converted into JPEG format image data by the JPEG encoder unit 32, information indicating that the data has been subjected to SJPEG conversion is added to the header.

[0031] Incidentally, noise data other than image data is output from the image sensor 25, or noise data is mixed between the image sensor 25 and the JPEG encoder unit 32. In the JPEG encoder unit 32, when the data is image data input to the JPEG encoder unit 32, the data is converted into JPEG format image data, and the header is added to the header. Gives information that it is image data.

On the other hand, when the data to be converted is not image data, the JPEG encoder unit 32 does not convert the image data into JPEG format image data. That is, of the data input as image data output from the image sensor 25 to the JPEG encoder unit 32, the resolution data is not image data! /, And is not converted to JPEG image data. In the header, information indicating that the image data is in JPEG format is not added. The JPEG encoder unit 32 may be configured so as not to convert the image data into JPEG format data if the image data includes noise data exceeding a predetermined level! /.

[0033] Therefore, based on the header information, it can be determined whether the content of the data to which the header is added is image data or noise data.

[0034] Also for sound data, a header is added by the header adding unit 31 in a predetermined amount of data. For example, in the present embodiment, a header is added to sound data for each data length of 1/30 second. The header of the sound data includes information that the data with the header added is sound data, and a header number N (N = 1, 2, 3, ... ) Information is given.

[0035] Whereas image data has a data amount of 15 frames per second, sound data has a data amount of 1/30 second. Therefore, one unit of image data corresponds to two units (1/30 seconds x 2) of sound data. [0036] The image data and the sound data are output from the CPU 30 to the streaming generation unit 29 in a state where the timing at the time of imaging and the time of sound collection are synchronized with each other based on the header number! It is powered. For example, the header number M of the image data is 1, 2, 3, 4, 5, ..., and the header number N of the sound data is 1, 2, 3, 4, 5, 6, 7, 8, 9, Let's say 10, ... Then, the sound data of header number N = l, 2 corresponds to the image data of header number M = l, and the image data of header number M = l and the sound of header number N = l, 2 Data is data with synchronized timing. Similarly, the image data with the header number M = 2 and the sound data with the header numbers N = 3 and 4 are data with synchronized timing. In other words, the image data of the header number M and the sound data of the header numbers N = 2M— 1 and 2M are data in which the time of the captured image and the collected sound are synchronized.

[0037] The streaming generation unit 29 reads the image data and sound data output from the CPU 30, and generates data having these content data as streaming data. The streaming generation unit 29 supplies the generated digital format data signal to the communication processing unit 33, and transmits it from the USB connector 10 to the camera server 3 via the USB cable 4.

In this way, data having content data of image data and sound data is generated, and the data in the digital format is sent to the camera server 3 via the USB cable 4. As a result, the image data in the digital format can be transmitted to the camera server 3 without being converted into the data signal in the analog format.

[0039] Audio 'The video encoder 28 reads the image data output from the color conversion processing unit 26 without converting it to JPEG format image data, and NTSC (National TV standards Committee) method ¹ -PAL (Phase Alternating Line ) Method high signal. The audio / video encoder 28 converts the sound data in the digital format into a sound signal that is an electrical signal. The video output connector 11 outputs the video signal and the sound signal together.

FIG. 3 is a block diagram showing an electric circuit of the camera server 3.

[0041] The camera server 3 includes a USB connector 16, a microphone connector 18 to which an extension microphone 17 is connected, a general-purpose input / output port 19 to which an external device such as an infrared sensor is connected, Audio output connector 20, video output connector 21, speaker connector 22, communication interface 23, power switch 24, audio 'video input connector 34, audio encoder 35, memory 36, and control of the above components MCU (Microcontroller Unit) 37 as a control means for managing

The MCU 37 includes a data selection unit 38 that is a data selection unit, a data synchronization unit 39 that is a data synchronization unit, a data distribution unit 40 that is a data distribution unit, and the like. The data selection unit 38, the data synchronization unit 39, and the data distribution unit 40 are functionally realized by reading the control program power MCU 37 stored in the memory 36.

The data selection unit 38 determines the content of the image data transmitted from the camera device 2 based on the header information of the image data of each frame. As described above, in the case of image data that has been subjected to JPEG conversion in the data power in the header, information to that effect is given to the header, and in the case where the data in the header is noise data, the header No information is given. That is, the data selection unit 38 determines whether the content of the data for each header is predetermined image data or data other than the predetermined image data based on the header information.

Then, the data selection unit 38 selects data to be output to the data synchronization unit 39 based on the determination result. That is, when the data selection unit 38 determines that the content of the data is predetermined image data, the image data is transmitted from the communication interface unit 23 via the data synchronization unit 39 and the data distribution unit 40. Then, it is distributed to the personal computer 7 of the client 5 through the network 6. On the other hand, data that is not predetermined image data is not output to the data synchronization unit 39 side.

[0045] In this manner, the image data distributed to the client 5 side can be obtained by providing the data selection unit 38 and not distributing the predetermined image data! / To the client 5 side. , Noise data will not be included. For this reason, the video displayed on the monitor of the personal computer 7 of the client 5 is a beautiful video with little noise. Since noise data is not transmitted, the amount of data transmitted from the camera server 3 to the client 5 is reduced, and the transmission speed of image data from the camera server 3 to the client 5 can be increased. On the other hand, since the sound data is not selected as in the case of image data, all the sound data is transmitted from the communication interface unit 23 via the data synchronization unit 39 and the data distribution unit 40. It is distributed to the personal computer 7 of the client 5 through the network 6. On the client 5 side, sound such as voice collected by the microphone 13 can be heard from a speaker built in the personal computer 7 or the like.

[0047] By the way, since the sound data is not selected as in the case of image data, the header number N of the sound data input from the data selection unit 38 to the data distribution unit 40 is a continuous number without any loss. On the other hand, as described above, with respect to the image data, the data selection unit 38 selects the data to be output to the data distribution unit 40, and therefore the image data input to the data distribution unit 40 from the data selection unit 38 is selected. The header number M may be missing.

Therefore, for example, for image data, when the image data of the portion in which the header number M is missing is packed and distributed, and for sound data, the sound data is distributed to the client 5 side without packing the data. As a result, the video based on the image data is ahead of the content of the sound based on the video, and the video and sound are not synchronized. In addition, for example, if sound data that synchronizes with missing image data is deleted, there is a problem in that the content of power sound that can synchronize video and sound is skipped.

[0049] Therefore, in the data synchronization unit 39, the sound collection timing and the imaging timing of the sound data and the image data output from the data selection unit 38 are synchronized with each other, and the sound content is skipped. Process to synchronize so that nothing happens.

[0050] Specifically, for example, the header number power of the image data output from the data selection unit 38 is 1, 2,-, 4, 5, ... If the header number of the sound data is 1, 2, 3, 4, 5, 6, ..., the sound with the header numbers 1 and 2 for the image data with the header number 1 Synchronize the data, and synchronize the sound data of header numbers 3 and 4 with the image data of header number 2. Image data with header number 3 is missing force Sound data with header numbers 5 and 6 is synchronized with this part. The sound data of header numbers 7 and 8 are synchronized with the image data of header number 4. In this way, the sound data and image data are synchronized. Data is distributed from the data distribution unit 40 to the network 6 side through the communication interface 23.

[0051] On the client 5 side, only the sound data is distributed for the portion of header number 3 where the image data is missing, and the sound based on the sound data of header numbers 5 and 6 is played from the speaker of the personal computer 7 Will be.

[0052] It should be noted that the viewer software of the personal computer 7 of the client 5 is configured to continuously display the video based on the previous image data on the monitor until new image data is distributed. When image data is lost, nothing can be displayed on the monitor of the PC 7 so that it does not appear unnatural. When the visual software is configured in this way, the video data based on the image data with the header number 2 is displayed until the image data with the header number 2 is distributed and then the image data with the header number 4 is distributed. During this time, sound based on the sound data of header numbers 5 and 6 is reproduced.

Note that the data distribution unit 40 simultaneously distributes data signals to these clients 5 when a plurality of clients 5 are accessing the camera server 3.

By the way, the USB connector 16 is supplied with a data signal having image data in digital format and sound data in digital format from the camera device 2 via the USB cable 4 and is input to the MCU 37. The digital data signal input to the MCU 37 is configured to be output from the communication interface 23 to the network 6 as a digital data signal.

Therefore, since the camera server 3 outputs the digital data signal output from the camera device 2 to the network side as a digital signal, there is no signal deterioration. Further, it is not necessary to provide an AD conversion function for converting an analog signal into a digital signal in the camera server 3. Note that image data is transmitted using the TCP or UDP protocol in a half-duplex communication system. Sound data is transmitted by the TCP protocol or UDP protocol in a duplex communication system.

[0056] As described above, image data is converted into compressed image data in JPEG format in the camera device 2, and then transmitted to the camera server 3. From the data distribution unit 40, the image data is transmitted to the client through the communication interface 23. Delivered to 5. In this way, the image data is By converting the apparatus 2 to the JPEG format and sending the image data in the JPEG format to the camera server 3, the capacity of data transmitted from the camera apparatus 2 to the camera server 3 can be reduced. Therefore, it is possible to improve the transmission speed of image data in the camera server system 1. As a result, it is possible to increase the speed at which image data is sent to the client 5.

On the other hand, for example, the conversion of the image data into the JPEG format is performed in the MCU 37 of the camera server 3, and in response to a request for transmission of the image data from the client 5, the JPEG format is used each time. When the conversion is executed, the speed of distributing the image data from the camera server 3 to the client 5 becomes slow.

In addition to the overall control of the camera server 3, the MCU 37 performs many processes related to the control of the camera device 2 and the communication with the client 5. Therefore, in addition to these processes, if an encoding function for JPEG conversion is provided, the load on the MCU 37 increases, the processing speed of the MCU 37 decreases, and the encoding processing speed for JPEG conversion also slows down. As a result, the transmission speed of the image data from the camera server 3 to the client 5 becomes slow.

[0059] In addition, when conversion to JPEG format is executed every time a request for transmission of image data from the client 5 is performed, image data transmission processing to the client 5 of 1 Since the other clients 5 have to wait while the image data is being transmitted, the transmission speed of the image data to the client 5 is reduced.

On the other hand, JPEG conversion speed can be increased by performing JPEG conversion using the CPU 30 on the camera device 2 side, which has fewer processing items than the MCU 37 on the camera server 3 side. In addition, since the MCU 37 does not perform the JPEG conversion process, the processing speed does not decrease, the processing speed of the data selection unit 38, the data distribution unit 40, etc. can be increased, and the image data is distributed to the client 5. The processing speed can be increased.

[0061] Further, since the image data is simultaneously distributed from the data distribution unit 40 to the plurality of clients 5, the client 5 waits for the distribution to the other clients 5 to finish and stores the image data. The delivery is not received and the delivery speed can be increased.

[0062] Note that the JPEG encoder unit 32 is provided in the CPU 30 of the camera device 2, and a JPEG encoder 32A is connected to the camera server 3 side separately from the MCU 37 as shown in FIG. It may be provided between the connector 16 and the MCU 37, and the image data that has been subjected to JPEG conversion by the JPEG encoder 32A may be input to the MCU 37. The JPEG encoder unit 32A is provided as a processing device different from the MCU 37 provided with the data selection unit 38, the data synchronization unit 39, the data distribution unit 40, and the like, so that the processing speed of the MCU 37 is not reduced. .

[0063] Further, a dedicated JPEG encoder may be provided on the camera device 2 side separately from the CPU 30, and JPEG conversion may be performed by the JP EG encoder.

The audio encoder 35 reads the digital sound data input from the communication interface 23 to the camera server 3 from the MCU 37, converts the sound data into a sound signal that is an electrical signal, and outputs the sound signal to the speaker connector 22. For this reason, for example, by connecting a speaker or the like to the speaker connector 22, the sound data input from the communication interface 23 can be reproduced with the force S.

[0065] When the sound signal and video signal output from the audio 'video output connector 11 on the camera device 2 side are input to the audio' video input connector 34, these sound signal and video signal are output as audio signals, respectively. Output to connector 20 and video output connector 21. Accordingly, when a television or the like is connected to the audio output connector 20 and the video output connector 21, the video and audio captured and collected by the camera device 2 can be viewed.

A camera device control signal for controlling the camera device 2 is transmitted from the camera server 3 to the camera device 2 via the USB cable 4. This camera device control signal is generated by the device control unit 41 configured in the MCU 37, and controls, for example, the start and stop of the operation of the camera device 2, or the camera device 2 receives image data. That require the transmission of Also, for a command transmitted from the personal computer 7 on the client 5 side connected to the network to the camera server 3, the command is interpreted and a camera device control signal is transmitted to the camera device 2. For example, from a computer 7 etc., pan control for panning to move the shooting direction horizontally, tilt control to move the shooting direction up and down, zoom for narrowing or widening the shooting angle of view When control or the like is requested, the device control unit 41 interprets the command and transmits a camera device control signal to the camera device 2. The device controller 41 is connected to the camera In addition to device 2, if an external device is connected to GPIO, this external device is also controlled.

[0067] By the way, the microphone 13 incorporated in the camera device 2 is limited in size that can be used due to restrictions on the image sensor 25 and circuit configuration disposed in the housing 8, and has sufficient sound collecting power. You may not get. In such a case, the extension microphone 17 is connected to the microphone connector 18.

When the MCU 37 recognizes that the extension microphone 17 is connected to the microphone connector 18, the MCU 37 transmits a camera device control signal for stopping the sound collection by the microphone 13 to the camera device 2. In response to the camera device control signal, the CPU 30 of the camera device 2 controls the sound processing unit 27 to stop generating sound data. Therefore, the power camera device 2 outputs data without sound data.

[0069] The sound signal is read from the microphone connector 18, and the sound processing unit 42 configured in the MCU 37 of the camera server 3 generates sound data in a digital format. The sound data and the image data transmitted from the camera device 2 are supplied from the data distribution unit 40 to the communication interface 23 and distributed to the network 6.

[0070] As described above, when the extension microphone 17 is connected, the sound is collected by the extension microphone 17 instead of the microphone 13 built in the camera device 2. Sound data suitable for the environment such as noise can be obtained and placed on the data for distribution.

[0071] When the operation of the camera server 3 is started (the power switch 25 is turned on), the application of the camera server 3 sets the sound data of the microphone 13 sent from the USB cable 4 as a default sound input. ing. Thereafter, the application of the camera server 3 searches for the connection of the extension microphone 17 of the microphone connector 18. When the application of the power server 3 recognizes the connection of the expansion microphone 17, as described above, the device control unit 41 transmits a camera device control signal for stopping the sound collection by the microphone 13 to the camera device 2. To do. When the application of the camera server 3 does not recognize the connection of the extension microphone 17, the sound data of the microphone 13 sent from the USB cable 4 is used as it is. Note that the determination by the data selection unit 38 may be performed based on other information instead of based on the header information added to the image data. For example, when image data is inspected for each header and noise data exceeding a predetermined level is detected, the image data of the header may not be transmitted to the network 6 side.

Claims

The scope of the claims

[1] In a camera server that distributes image data of an image captured by a camera device equipped with an image sensor to a client via a network,

It is determined whether or not the image data is predetermined image data, and the image data determined to be the predetermined image data is output to the network side, and the image data is A camera server, comprising: a data selection unit that does not output data determined not to be the predetermined image data to the network side.

2. The camera server according to claim 1, wherein the data selection means determines whether or not the image data is the predetermined image data based on information of a header added to the image data. .

[3] Data synchronization means for synchronizing the image data and the sound data based on header information of the image data and the sound data when sound data is input in addition to the image data;

2. The data synchronization means synchronizes predetermined sound data of the sound data even with a data portion determined by the data selection means to be not the predetermined image data. Or the camera server described in 2.

[4] In a data processing method for distributing image data of an image captured by a camera device including an image sensor to a client through a network,

It is determined whether or not the image data is predetermined image data, and the image data determined to be the predetermined image data is output to the network side, and the image data is A data processing method, wherein data determined not to be the predetermined image data is not output to the network side.

5. The data processing method according to claim 4, wherein whether or not the image data is predetermined image data is determined based on information of a header added to the image data. .

[6] Data that distributes image data of an image captured by a camera device equipped with an image sensor and sound data of sound collected by a microphone to clients over a network In the processing method,

It is determined whether the image data is predetermined image data. The image data determined to be the predetermined image data is output to the network side, and the image data is However, the data that is determined not to be the predetermined image data is processed so as not to be output to the network side,

The sound data and the image data output to the network side are synchronized with each other based on the header information added to each data! /

A data processing method comprising synchronizing predetermined sound data of the sound data with a data portion determined not to be the predetermined image data.