WO2008050621A1

WO2008050621A1 - Camera server system, data processing method and camera server

Info

Publication number: WO2008050621A1
Application number: PCT/JP2007/070008
Authority: WO
Inventors: Katsura Obikawa; Jong-Sun Hur
Original assignee: Opt Corporation
Priority date: 2006-10-25
Filing date: 2007-10-12
Publication date: 2008-05-02
Also published as: JP2008109364A

Abstract

The speed of distributing data to a client is improved. A camera server (3) distributes image data of an image picked up by a camera device (2) having an imaging element (image sensor (25)) to a client (5) through a network (6). In the camera server, a JPEG encoder (32), which converts image data into image data in a JPEG format, is arranged between an input section (16) wherein the image data transmitted from the camera device(2) is inputted to the camera server (3), and a control means (MC37) of the camera server (3).

Description

Specification

Camera server system, data processing method, and camera server

Technical field

The present invention relates to a camera server system, a data processing method, and a camera server.

Background art

Conventionally, a camera server for distributing image data of an image captured by a camera device including an image sensor to a client via a network is known (for example, see Patent Document 1).

[0003] Patent Document 1: Japanese Patent Application Laid-Open No. 2005-318412

Disclosure of the invention

Problems to be solved by the invention

[0004] In such a camera server, there is a problem of increasing the data delivery speed to the client.

Accordingly, an object of the present invention is to provide a camera server system, a data processing method, and a camera server that increase the delivery speed of image data to a client. Means for solving the problem

[0006] In order to solve the above-described problem, in a camera server system including a camera device including an image sensor and a camera server that distributes image data output from the camera device to a client, the image sensor and the camera server An encoder that converts image data output from the image sensor into compressed image data that has been subjected to data compression is provided between the control unit and the control unit.

In such a configuration, the processing load on the control means of the camera server is reduced, and the processing speed of the camera server is increased. Therefore, the processing speed for distributing image data to the client can be increased.

[0008] In addition to the above-described invention, another invention is that the encoder is a JPEG encoder and is provided in a camera device. When configured in this manner, the image data is converted into the JPEG format in the camera device, and the image data in the JPEG format is sent to the camera server. Therefore, the volume of data transmitted from the camera device to the camera server can be reduced, and the transmission speed of image data in the camera server system can be improved. As a result, the speed at which image data is transmitted to the client can be increased.

[0010] In order to solve the above-described problems, image data is compressed in a data processing method when image data of an image captured by a camera device including an image sensor is distributed to a client via a network. After conversion to compressed image data, this compressed image data is distributed to the client in accordance with a request from the client.

When configured in this way, the processing load on the control means of the camera server is reduced, and the processing speed of the camera server is increased. Therefore, the processing speed for distributing image data to the client can be increased.

[0012] In addition to the above-described invention, in another invention, the data compression is JPEG compression, and the JPEG compression is performed in parallel with the camera device.

When configured in this way, the image data is converted into the JPEG format in the camera device, and the image data in the JPEG format is sent to the camera server. Therefore, the volume of data transmitted from the camera device to the camera server can be reduced, and the transmission speed of image data in the camera server system can be improved. As a result, the speed at which image data is transmitted to the client can be increased.

[0014] In order to solve the above-described problems, image data of an image captured by a camera device including an image sensor is converted into compressed image data that is data-compressed in a camera server that distributes the image data to a client via a network. This encoder is provided between an input unit for inputting image data sent from the camera device to the camera server and the control means of the camera server.

[0015] With this configuration, the processing load on the control means of the camera server is reduced, and the processing speed of the camera server is increased. Therefore, the processing speed for distributing image data to the client can be increased.

[0016] Further, in addition to the above-described invention, another invention is configured such that the encoder is a JPEG encoder, It is supposed to be provided in the equipment.

When configured in this manner, the image data is converted into the JPEG format in the camera device, and the image data in the JPEG format is sent to the camera server. Therefore, the volume of data transmitted from the camera device to the camera server can be reduced, and the transmission speed of image data in the camera server system can be improved. As a result, the speed at which image data is transmitted to the client can be increased.

The invention's effect

[0018] According to the present invention, it is possible to increase the data delivery speed to the client.

Brief Description of Drawings

FIG. 1 is a diagram showing a configuration of a camera server system according to an embodiment of the present invention.

2 is a block diagram showing an electric circuit of a camera device in the camera server system shown in FIG.

3 is a block diagram showing an electric circuit of a camera server in the camera server system shown in FIG.

Explanation of symbols

[0020] 1 · · · Camera server system

16 · · · USB connector (input section)

25 · · · Image sensor

32 · · · JPEG encoder (encoder)

37 · · · MCU (control means)

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, a camera server and a data processing method according to an embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is a system configuration diagram showing a camera server system 1 according to an embodiment of the present invention. The camera server system 1 includes a camera device 2 and a camera server 3. The camera device 2 and the camera server 3 are connected by a USB (Universal Serial Bus) cable 4. Camera server system 1 and client 5 are connected via network 6. The camera server system 1 receives a command from the personal computer 7 on the client 5 side. The camera server 3 distributes image data of an image captured by the camera device 2 to the personal computer 7 (client 5) based on a command received from the personal computer 7. One or a plurality of clients 5 exist, and the camera server system 1 distributes the image data to each client 5 in accordance with a request from each client 5.

The camera device 2 has a cubic-shaped housing 8. Housing 8 has a fisheye lens

9, USB connector 10, audio 'video output connector 11, sound output connector 12, etc. The fisheye lens 9 is disposed on the upper surface of the housing 8. In addition to the fisheye lens 9, a ventilation hole 14 for the microphone 13 (see FIG. 2) is formed on the upper surface of the housing 8. The USB connector 10, the audio / video output connector 11 and the sound output connector 12 are arranged on the side surface of the housing 8.

The camera server 3 has a rectangular parallelepiped housing 15. The housing 15 includes a USB connector 16 as an input unit that is an input terminal for inputting data from the camera device 2, a microphone connector 18 to which an expansion microphone 17 to be used as needed is connected, an infrared sensor, and the like. General purpose input / output port (GPIO) 19 for connecting external equipment, audio output connector 20 for outputting audio signals, video output connector 21 for outputting video signals, and a speaker connected to a speaker A power connector 22, a communication interface 23 connected to the network, a power switch 24 also serving as a reset switch, and the like are provided.

FIG. 2 is a block diagram showing an electric circuit of the camera device 2.

[0026] The camera device 2 includes a fisheye lens 9, a USB connector 10, an audio 'video output connector 11, a sound output connector 12, a microphone 13, an image sensor 25 as an image sensor, and an F PGA (Field Programmable Gate Array). Color conversion processing unit 26, sound processing unit 27 composed of sound IC (Integrated Circuit) 27, audio 'video encoder 28, streaming generation unit 29, and CPU (CCU) that controls the operation of each of the above components Central Processing Unit) 30 etc.

The microphone 13 collects sounds and sounds around the camera device 2. The microphone 13 generates a sound signal that is an electrical signal corresponding to the collected sound and outputs the sound signal to the sound processing unit 27. The sound processing unit 27 generates digital sound data based on the electrical signal from the microphone 13. The sound processing unit 27 outputs an electrical signal corresponding to the sound from the microphone 13 to the sound output connector 12 without converting it into sound data (digital format). Therefore, the sound collected by the microphone 13 can be heard by connecting, for example, a speaker, headphones or the like to the sound output connector 12.

As the image sensor 25, for example, a CMOS (Complementary Metal-Oxide Semiconductor) or the like is used. An image by the fisheye lens 9 is formed on the image sensor 25. The image sensor 25 outputs luminance distribution data, which is an imaging signal, to the color conversion processing unit 26 as digital image data. In the present embodiment, it is assumed that the image sensor 25 performs imaging at an imaging speed of 1/15 frames per second, for example. Therefore, one frame of image data is output from the image sensor 25 every 1/15 second.

[0029] The color conversion processing unit 26 replaces the data of each pixel of the luminance distribution data using a color conversion table (not shown) to generate digital image data. The generated image data is added with a header for each frame by a header adding unit 31 formed by the CPU 30. A header number M (M = 1, 2, 3,...), Which is a serial number, is assigned to the header according to the imaging order.

The CPU 30 is configured with a JPEG encoder section 32 as an encoder that converts image data into JPEG (Joint Photographic Experts Group) open-type compressed image data. When the image data is converted into JPEG format image data by the JPEG encoder unit 32, information indicating that the data has been subjected to SJPEG conversion is added to the header.

[0031] Incidentally, noise data other than image data is output from the image sensor 25, or noise data is mixed between the image sensor 25 and the JPEG encoder unit 32. In the JPEG encoder unit 32, when the data is image data input to the JPEG encoder unit 32, the data is converted into JPEG format image data, and the header is added to the header. Gives information that it is image data. On the other hand, when the data to be converted is not image data, the JPEG encoder unit 32 does not convert the image data into JPEG format image data. That is, of the data input as image data output from the image sensor 25 to the JPEG encoder unit 32, the resolution data is not image data! /, And is not converted to JPEG image data. In the header, information indicating that the image data is in JPEG format is not added. The JPEG encoder unit 32 may be configured so as not to convert the image data into JPEG format data if the image data includes noise data exceeding a predetermined level! /.

[0033] Therefore, based on the header information, it can be determined whether the content of the data to which the header is added is image data or noise data.

[0034] Also for sound data, a header is added by the header adding unit 31 in a predetermined amount of data. For example, in the present embodiment, a header is added to sound data for each data length of 1/30 second. The header of the sound data includes information that the data with the header added is sound data, and a header number N (N = 1, 2, 3, ... ) Information is given.

[0035] Whereas image data has a data amount of 15 frames per second, sound data has a data amount of 1/30 second. Therefore, one unit of image data corresponds to two units (1/30 seconds x 2) of sound data.

[0036] The image data and the sound data are output from the CPU 30 to the streaming generation unit 29 in a state where the timing at the time of imaging and the time of sound collection are synchronized with each other based on the header number! It is powered. For example, the header number M of the image data is 1, 2, 3, 4, 5, ..., and the header number N of the sound data is 1, 2, 3, 4, 5, 6, 7, 8, 9, Let's say 10, ... Then, the sound data of header number N = l, 2 corresponds to the image data of header number M = l, and the image data of header number M = l and the sound of header number N = l, 2 Data is data with synchronized timing. Similarly, the image data with the header number M = 2 and the sound data with the header numbers N = 3 and 4 are data with synchronized timing. In other words, the image data of the header number M and the sound data of the header numbers N = 2M— 1 and 2M are data in which the time of recording and the collected time are synchronized.

[0037] The streaming generation unit 29 reads image data and sound data output from the CPU 30. Data having these contents data is generated as streaming data. The streaming generation unit 29 supplies the generated digital format data signal to the communication processing unit 33, and transmits it from the USB connector 10 to the camera server 3 via the USB cable 4.

In this way, data having content data of image data and sound data is generated, and the data in the digital format is sent to the camera server 3 via the USB cable 4. As a result, the image data in the digital format can be transmitted to the camera server 3 without being converted into the data signal in the analog format.

[0039] Audio 'The video encoder 28 reads the image data output from the color conversion processing unit 26 without converting it to JPEG format image data, and NTSC (National TV standards Committee) method ¹ -PAL (Phase Alternating Line ) Method high signal. The audio / video encoder 28 converts the sound data in the digital format into a sound signal that is an electrical signal. The video output connector 11 outputs the video signal and the sound signal together.

FIG. 3 is a block diagram showing an electric circuit of the camera server 3.

[0041] The camera server 3 includes a USB connector 16, a microphone connector 18 to which an extension microphone 17 is connected, a general-purpose input / output port 19 to which an external device such as an infrared sensor is connected, an audio output connector 20, and a video output. In addition to connector 21, speaker connector 22, communication interface 23, power switch 24, audio 'video input connector 34, audio encoder 35, memory 36, and control means for controlling the operation of each of the above components An MCU (Microcontroller Unit) 37 is included.

The MCU 37 includes a data selection unit 38 that is a data selection unit, a data synchronization unit 39 that is a data synchronization unit, a data distribution unit 40 that is a data distribution unit, and the like. The data selection unit 38, the data synchronization unit 39, and the data distribution unit 40 are functionally realized by reading the control program power MCU 37 stored in the memory 36.

The data selection unit 38 determines the content of the image data transmitted from the camera device 2 based on the header information of the image data of each frame. Above As described above, in the case where the header is the data power in the header JPEG converted image data, information to that effect is given, and in the case where the data in the header is noise data, the information is Not granted. That is, the data selection unit 38 determines whether the content of the data for each header is predetermined image data or data other than the predetermined image data based on the header information.

Then, the data selection unit 38 selects data to be output to the data synchronization unit 39 based on the determination result. That is, when the data selection unit 38 determines that the content of the data is predetermined image data, the image data is transmitted from the communication interface unit 23 via the data synchronization unit 39 and the data distribution unit 40. Then, it is distributed to the personal computer 7 of the client 5 through the network 6. On the other hand, data that is not predetermined image data is not output to the data synchronization unit 39 side.

[0045] In this manner, the image data distributed to the client 5 side can be obtained by providing the data selection unit 38 and not distributing the predetermined image data! / To the client 5 side. , Noise data will not be included. For this reason, the video displayed on the monitor of the personal computer 7 of the client 5 is a beautiful video with little noise. Since noise data is not transmitted, the amount of data transmitted from the camera server 3 to the client 5 is reduced, and the transmission speed of image data from the camera server 3 to the client 5 can be increased.

On the other hand, since the sound data is not selected as in the case of image data, all the sound data is transmitted from the communication interface unit 23 via the data synchronization unit 39 and the data distribution unit 40. It is distributed to the personal computer 7 of the client 5 through the network 6. On the client 5 side, sound such as voice collected by the microphone 13 can be heard from a speaker built in the personal computer 7 or the like.

[0047] By the way, since the sound data is not selected as in the case of image data, the header number N of the sound data input from the data selection unit 38 to the data distribution unit 40 is a continuous number without any loss. On the other hand, as described above, with respect to the image data, the data selection unit 38 selects the data to be output to the data distribution unit 40, and therefore the image data input to the data distribution unit 40 from the data selection unit 38 is selected. The header number M may be missing. Therefore, for example, for image data, when the image data of the portion in which the header number M is missing is packed and distributed, and for sound data, the sound data is distributed to the client 5 side without packing the data. As a result, the video based on the image data is ahead of the content of the sound based on the video, and the video and sound are not synchronized. In addition, for example, if sound data that synchronizes with missing image data is deleted, there is a problem in that the content of power sound that can synchronize video and sound is skipped.

[0049] Therefore, in the data synchronization unit 39, the sound collection timing and the imaging timing of the sound data and the image data output from the data selection unit 38 are synchronized with each other, and the sound content is skipped. Process to synchronize so that nothing happens.

[0050] Specifically, for example, the header number power of the image data output from the data selection unit 38 is 1, 2,-, 4, 5, ... If the header number of the sound data is 1, 2, 3, 4, 5, 6, ..., the sound with the header numbers 1 and 2 for the image data with the header number 1 Synchronize the data, and synchronize the sound data of header numbers 3 and 4 with the image data of header number 2. Image data with header number 3 is missing force Sound data with header numbers 5 and 6 is synchronized with this part. The sound data of header numbers 7 and 8 are synchronized with the image data of header number 4. As described above, in a state where the sound data and the image data are synchronized, the data is distributed from the data distribution unit 40 to the network 6 through the communication interface 23.

[0051] On the client 5 side, only the sound data is distributed for the portion of header number 3 where the image data is missing, and the sound based on the sound data of header numbers 5 and 6 is played from the speaker of the personal computer 7 Will be.

[0052] It should be noted that the viewer software of the personal computer 7 of the client 5 is configured to continuously display the video based on the previous image data on the monitor until new image data is distributed. When image data is lost, nothing can be displayed on the monitor of the PC 7 so that it does not appear unnatural. When the visual software is configured in this way, the video data based on the image data with the header number 2 is displayed until the image data with the header number 2 is distributed and then the image data with the header number 4 is distributed. Is this During this period, the sound based on the sound data of the header numbers 5 and 6 is reproduced.

Note that the data distribution unit 40 simultaneously distributes data signals to these clients 5 when a plurality of clients 5 are accessing the camera server 3.

By the way, the USB connector 16 is supplied with a data signal having image data in digital format and sound data in digital format from the camera device 2 via the USB cable 4 and is input to the MCU 37. The digital data signal input to the MCU 37 is configured to be output from the communication interface 23 to the network 6 as a digital data signal.

Therefore, since the camera server 3 outputs the digital data signal output from the camera device 2 to the network side as a digital signal, there is no signal deterioration. Further, it is not necessary to provide an AD conversion function for converting an analog signal into a digital signal in the camera server 3. Note that image data is transmitted using the TCP or UDP protocol in a half-duplex communication system. Sound data is transmitted by the TCP protocol or UDP protocol in a duplex communication system.

[0056] As described above, image data is converted into compressed image data in JPEG format in the camera device 2, and then transmitted to the camera server 3. From the data distribution unit 40, the image data is transmitted to the client through the communication interface 23. Delivered to 5. As described above, the image data is converted into the JPEG format in the camera device 2 and the image data in the JPEG format is sent to the camera server 3, so that the capacity of the data transmitted from the camera device 2 to the camera server 3 can be reduced. Can be reduced. For this reason, the transmission speed of image data in the camera server system 1 can be improved. As a result, it is possible to increase the speed at which image data is sent to the client 5.

On the other hand, for example, the conversion of the image data into the JPEG format is performed in the MCU 37 of the camera server 3, and in response to a request for transmission of the image data from the client 5, the JPEG format is used each time. When the conversion is executed, the speed of distributing the image data from the camera server 3 to the client 5 becomes slow.

In addition to the overall control of the camera server 3, the MCU 37 performs many processes related to the control of the camera device 2 and the communication with the client 5. Therefore, in addition to these processes, J If an encoding function for PEG conversion is provided, the load on the MCU 37 increases, the processing speed of the MCU 37 decreases, and the encoding processing speed of the JPEG conversion also decreases. As a result, the transmission speed of the image data from the camera server 3 to the client 5 becomes slow.

[0059] In addition, when conversion to JPEG format is executed every time a request for transmission of image data from the client 5 is performed, image data transmission processing to the client 5 of 1 Since the other clients 5 have to wait while the image data is being transmitted, the transmission speed of the image data to the client 5 is reduced.

On the other hand, the camera device has fewer processing items than the MCU 37 on the camera server 3 side.

JPEG conversion speed can be increased by performing JPEG conversion using the CPU 30 on the second side. In addition, since the MCU 37 does not perform the JPEG conversion process, the processing speed does not decrease, the processing speed of the data selection unit 38, the data distribution unit 40, etc. can be increased, and the image data is distributed to the client 5. The processing speed can be increased.

[0061] Further, since the image data is simultaneously distributed from the data distribution unit 40 to the plurality of clients 5, the client 5 waits for the distribution to the other clients 5 to finish and stores the image data. The delivery is not received and the delivery speed can be increased.

[0062] Note that the JPEG encoder unit 32 is provided in the CPU 30 of the camera device 2, and, as indicated by a dotted line in Fig. 3, the JPEG encoder 32A is connected to the camera server 3 side separately from the MCU 37, the USB connector 16 and the MCU 37 The image data obtained by performing JPEG conversion by the JPEG encoder 32A may be input to the MCU 37. The JPEG encoder unit 32A is provided as a processing device different from the MCU 37 provided with the data selection unit 38, the data synchronization unit 39, the data distribution unit 40, and the like, so that the processing speed of the MCU 37 is not reduced. .

[0063] Further, a dedicated JPEG encoder may be provided on the camera device 2 side separately from the CPU 30, and JPEG conversion may be performed by the JP EG encoder.

[0064] In the present embodiment described above, the data compression method and encoder are not limited to the JPEG compression method and other compression such as MPEG (Moving Picture Experts Group) shown as JPEG encoder unit 32 (32A). A method may be adopted. In addition to the irreversible compression method, a reversible compression method may be used. The audio encoder 35 reads the digital sound data input from the communication interface 23 to the camera server 3 from the MCU 37, converts it into a sound signal that is an electrical signal, and outputs it to the speaker connector 22. For this reason, for example, by connecting a speaker or the like to the speaker connector 22, the sound data input from the communication interface 23 can be reproduced with the force S.

[0066] When the sound signal and video signal output from the audio 'video output connector 11 on the camera device 2 side are input to the audio' video input connector 34, these sound signal and video signal are output as audio signals, respectively. Output to connector 20 and video output connector 21. Accordingly, when a television or the like is connected to the audio output connector 20 and the video output connector 21, the video and audio captured and collected by the camera device 2 can be viewed.

A camera device control signal for controlling the camera device 2 is transmitted from the camera server 3 to the camera device 2 via the USB cable 4. This camera device control signal is generated by the device control unit 41 configured in the MCU 37, and controls, for example, the start and stop of the operation of the camera device 2, or the camera device 2 receives image data. That require the transmission of Also, for a command transmitted from the personal computer 7 on the client 5 side connected to the network to the camera server 3, the command is interpreted and a camera device control signal is transmitted to the camera device 2. For example, from a computer 7 etc., pan control for panning to move the shooting direction horizontally, tilt control to move the shooting direction up and down, zoom for narrowing or widening the shooting angle of view When control or the like is requested, the device control unit 41 interprets the command and transmits a camera device control signal to the camera device 2. The device control unit 41 controls the external device in addition to the camera device 2 when an external device is connected to the GPIO.

[0068] By the way, the microphone 13 built in the camera device 2 is limited in the size that can be used due to restrictions on the image sensor 25 disposed in the housing 8 and the circuit configuration, and has sufficient sound collecting power. You may not get. In such a case, the extension microphone 17 is connected to the microphone connector 18.

[0069] The MCU 37 confirms that the extension microphone 17 is connected to the microphone connector 18. When recognized, a camera device control signal for stopping the sound collection by the microphone 13 is transmitted to the camera device 2. In response to the camera device control signal, the CPU 30 of the camera device 2 controls the sound processing unit 27 to stop generating sound data. Therefore, the power camera device 2 outputs data without sound data.

A sound signal is read from the microphone connector 18, and digital sound data is generated by the sound processing unit 42 configured in the MCU 37 of the camera server 3. The sound data and the image data transmitted from the camera device 2 are supplied from the data distribution unit 40 to the communication interface 23 and distributed to the network 6.

As described above, when the extension microphone 17 is connected, the extension microphone 17 collects sound instead of the microphone 13 built in the camera device 2, so that the target of the sound to be collected and the surrounding sound are collected. Sound data suitable for the environment such as noise can be obtained and placed on the data for distribution.

[0072] When the operation of the camera server 3 is started (the power switch 25 is turned on), the application of the camera server 3 sets the sound data of the microphone 13 sent from the USB cable 4 as the default sound input. ing. Thereafter, the application of the camera server 3 searches for the connection of the extension microphone 17 of the microphone connector 18. When the application of the power server 3 recognizes the connection of the expansion microphone 17, as described above, the device control unit 41 transmits a camera device control signal for stopping the sound collection by the microphone 13 to the camera device 2. To do. When the application of the camera server 3 does not recognize the connection of the extension microphone 17, the sound data of the microphone 13 sent from the USB cable 4 is used as it is.

Note that the determination by the data selection unit 38 may be performed based on other information instead of based on the header information added to the image data. For example, when image data is inspected for each header and noise data exceeding a predetermined level is detected, the image data of the header may not be transmitted to the network 6 side.

Claims

The scope of the claims

[1] a camera device including an image sensor;

In a camera server system comprising: a camera server that distributes image data output from the camera device to a client;

A camera server system comprising an encoder for converting image data output from the image sensor into compressed image data that is data-compressed between the image sensor and the control means of the camera server.

2. The camera server system according to claim 1, wherein the encoder is a JPEG encoder and is provided in the camera device.

[3] In a data processing method for distributing image data of an image captured by a camera device including an image sensor to a client through a network,

A data processing method comprising: converting the image data into compressed image data that has been compressed, and then delivering the compressed image data to the client in accordance with a request from the client.

4. The data processing method according to claim 3, wherein the data compression is JPEG compression, and the above-described d! PEG compression is performed on the camera device.

[5] In a camera server that distributes image data of an image captured by a camera device including an image sensor to a client via a network,

An encoder that converts the image data into compressed image data that has undergone data compression is provided between an input unit through which the image data sent from the camera device is input to the camera server and the control means of the camera server. A camera server characterized by that.

6. The camera server according to claim 5, wherein the encoder is a JPEG encoder and is provided in the camera device.