JP2006332914A - Photographing apparatus and image data transfer method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photographing apparatus for making a terminal display a photographed video thereon fast after the end of the operation of a camera when the photographing apparatus carries out settings of a photographed position and the operation of the camera, in the apparatus for displaying the video on the terminal via a network. <P>SOLUTION: While a camera state detection section 4 of the photographing apparatus 1 detects a change in the photographing state and the control state of the camera, the photographing apparatus 1 controls a coding pre-processing section 5 or an image signal coding section 6 on the basis of a data transfer rate of the network, a frame rate of transferring image data obtained by a network interface section 7 and a code quantity of compressed image data coded by the image signal coding section 6 so as to adjust the coding quantity of compressed image data coded by the image signal coding section 6 and attain transfer of the data at the designated frame rate and transmits the result to the terminal 2. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a photographing apparatus that transmits a captured image to a terminal device connected via a network such as the Internet, and an image data transfer method that compresses and encodes the captured image data and transmits the image data on a network.

  2. Description of the Related Art Conventionally, video conferencing systems and video chat systems are known as devices that transmit captured images in real time to a terminal (receiving device) connected using a general network such as a dedicated line or the Internet. In particular, since networks such as the Internet are rapidly spreading in recent years, these systems can be easily used by anyone by connecting a USB camera to a personal computer and using the Internet. Also, in a video conference dedicated system having both a photographing function and a communication function, a network such as the Internet is generally used.

Response time is extremely problematic when considering the transmission of video captured by such a system in real time, but especially when transmitting and receiving data required for video conference systems and video chat systems in remote locations Rate matters. However, in the Internet, traffic fluctuations are large, and if multiple accesses are concentrated on the same traffic or if the amount of data transferred by each access is large, the transfer rate assigned to each of them may be lowered to maintain real-time performance. It becomes difficult, and when the transfer target is lowered, if the shooting target is changed by the operation of the camera, the video after the camera operation is finished may not be displayed on the receiving side in real time.
On the other hand, in such a system, it is required to transfer video data, audio data, and other texts and image data for presentation used in a meeting in real time.

  Also known is a system that remotely controls the imaging device to control the imaging angle and the zoom function of the camera, but responsiveness also becomes a problem when remotely operating the imaging device. It may take a considerable amount of time to obtain an image at the target position, and the operability may deteriorate.

Therefore, as a method of operating the photographing device comfortably and comfortably and quickly transmitting image data being operated to the terminal, the camera is controlled when receiving a camera control signal from the remotely operated terminal device. There has been proposed a method of performing image data reduction processing and transmitting image data to a network. (For example, see Patent Document 1)
JP 2002-354461 A

  However, cameras that automatically follow in response to audio or the like are widespread among cameras used in video conferencing systems. Instead of remotely operating a camera from a remote video conference system, the TV on the photographing side is used. A method of use in which a conference system automatically specifies a shooting position of a camera has become common.

  In addition, when a home camera such as a USB camera is connected to a personal computer and a personal environment system such as video chat or videophone is constructed, a general home camera has a function of controlling the camera from a remote location. In many cases, the subject himself picks up the camera and determines the shooting position.

  Therefore, it is expected that the video after the camera operation is completed can be displayed in real time on the receiving side, regardless of the network traffic status, even if the shooting position changes or the camera operation is changed on the shooting device side. Yes. In addition, for the person who is watching the video on the receiving device side, the video during the setting of the shooting position and the camera operation on the shooting device side is not important. Rather, the video after the operation of the camera is finished quickly. It is expected to be displayed.

However, in Japanese Patent Laid-Open No. 2002-354461, processing for reducing the amount of data to be transmitted when a camera control signal is received from a remote location is performed, so that the photographing position is set and the camera is operated on the photographing device side. In this case, there is a problem that the amount of data to be transmitted cannot be adjusted so that the video after the operation of the camera can be quickly displayed on the receiving device.
Further, when traffic is not congested, it is not necessary to reduce the amount of data, and it is desirable to change the reduction rate according to the transfer rate between networks.

  The present invention has been made in order to solve the above-described conventional problems, and is related to the traffic state of the network even when the photographing target is changed by setting the photographing position or operating the camera on the photographing apparatus side. It is another object of the present invention to provide a photographing apparatus and an image data transfer method for displaying an image after the camera operation is completed in real time on the receiving side and transmitting photographed image data.

  In order to solve the above problems, an imaging apparatus according to claim 1 of the present invention is configured to generate an image signal from an image captured by a camera, and an image signal generated by the image signal generation unit. An encoding preprocessing unit that performs preprocessing of encoding, an image signal encoding unit that performs compression processing on the image signal processed by the encoding preprocessing unit, and transmits the encoded image data over a network A network interface unit that acquires a network transfer rate, a camera state detection unit that detects a shooting state of the camera and a control state of the camera, the pre-encoding processing unit, and the image signal encoding unit; A code amount adjusting unit that adjusts the code amount of the camera, while the camera state detecting unit detects changes in the shooting state of the camera and the control state of the camera. Based on the network data transfer rate from the network interface unit, the frame rate for transferring the image data, and the code amount of the compressed image data encoded by the image signal encoding unit, The image signal encoding unit is controlled to adjust the code amount of the compressed image data, and the compressed image data having the adjusted code amount is transmitted from the network interface unit to the network.

  According to a second aspect of the present invention, in the photographing apparatus according to the first aspect, when the camera state detection unit detects pan / tilt or movement of the camera, information on a change in the photographing state of the camera is stored in the code amount. An angular velocity sensor for notifying the adjustment unit and camera control means for notifying the code amount adjustment unit of information on a change in the control state of the camera when the control of the camera is started.

  The imaging apparatus according to claim 3 of the present invention is the imaging apparatus according to claim 1, wherein the image signal encoding unit performs DCT processing for performing discrete cosine transform on the image signal processed by the pre-encoding processing unit. Means for quantizing the DCT coefficients computed by the DCT processing means using a quantization table, and multiplying means for multiplying each coefficient of the quantization table used by the quantization means by a predetermined number And an encoding means for encoding the quantized data calculated by the quantization means, and the code amount adjustment section changes the shooting state of the camera and the control state of the camera by the camera state detection section. During detection, the network data transfer rate from the network interface unit, the frame rate for transferring image data, and the image signal code A coefficient amount of the compressed image data to be encoded by the image signal encoding unit is adjusted by multiplying each coefficient of the quantization table by a predetermined number based on a code amount of the image data encoded by the unit, To do.

  The imaging apparatus according to claim 4 of the present invention is the imaging apparatus according to claim 1, wherein the pre-encoding processing unit discretizes and samples the image signal from the image signal generation unit, and the sampling Image forming means for constructing an image from the data sampled by the means, and the code amount adjusting unit detects a change in the shooting state of the camera and the control state of the camera by the camera state detection unit. The pre-encoding processing unit based on the network data transfer rate from the network interface unit, the frame rate for transferring image data, and the code amount of the compressed image data encoded by the image signal encoding unit. Control is performed to change the image size of the image signal and adjust the code amount of the compressed image data.

  According to a fifth aspect of the present invention, there is provided the photographing apparatus according to the first aspect, wherein the network interface unit is connected to image data transmitting means for transmitting the encoded compressed image data and a network connected thereto. Network transfer rate acquisition means for acquiring a data transfer rate, wherein the network transfer rate detection means detects the change in the shooting state and control state of the camera by the camera state detection unit. Is sent to the code amount adjustment unit.

  According to a sixth aspect of the present invention, in the photographing apparatus according to the third aspect, the image signal encoding unit includes temporary storage means for temporarily storing the image signal processed by the pre-encoding processing unit. The encoding adjustment unit multiplying each coefficient of the quantization table by a predetermined number and encoding the compressed image data encoded by the image signal encoding unit, and network data transfer obtained from the network interface unit The image signal encoded by the image signal encoding unit is re-encoded based on the rate and the frame rate at which the image data is transferred.

  According to a seventh aspect of the present invention, in the photographing apparatus according to any one of the first to sixth aspects, the code amount of the compressed image data encoded by the image signal encoding unit exceeds a predetermined data amount. The image size of the image signal generated by the pre-encoding processing unit is changed.

  An image data transfer method according to claim 8 of the present invention converts video captured by a camera into an image signal, performs compression processing by encoding on the image signal, and transmits the encoded compressed image data over a network. An image data transfer method for detecting a change in camera shooting state and camera control state, a network data transfer rate for transmitting image data, a frame rate for transferring image data, and encoding The compressed image data having the code amount adjusted based on the code amount of the compressed image data is transferred.

  The image data transfer method according to claim 9 of the present invention is the image data transfer method according to claim 8, wherein the network data transfer rate and the image data are detected while the change in the shooting state of the camera and the control state of the camera is detected. A code amount upper limit value that enables transmission of image data at a specified frame rate based on a frame rate for transmitting the image data is calculated, and compressed image data in which the code amount is adjusted is transferred.

  The image data transfer method according to claim 10 of the present invention is the image data transfer method according to claim 8, wherein the code amount of the encoded compressed image data allows the image data to be transmitted at a specified frame rate. If the value exceeds the value, the compressed image data in which the code amount is adjusted by setting the quantization table so as to reduce the data amount of the encoded image data is transferred, and the code amount of the encoded compressed image data is the code When the lower limit value based on the upper limit value is not reached, the compressed image data with the code amount adjusted is transferred by setting a quantization table so that the data amount of the encoded image data is increased. It is.

  The image data transfer method according to claim 11 of the present invention is the image data transfer method according to claim 8, wherein the code amount of the encoded compressed image data allows the image data to be transmitted at a specified frame rate. If the value exceeds the value, the image signal is converted so that the image size of the image signal to be encoded is reduced, the compressed image data whose code amount is adjusted is transferred, and the code amount of the encoded compressed image data is the code amount upper limit When the lower limit value based on the value is not reached, the compressed image data in which the code amount is adjusted by converting the image signal so as to increase the image size of the image signal to be encoded is transferred.

  The image data transfer method according to claim 12 of the present invention is the image data transfer method according to claim 10, wherein the code amount of the encoded compressed image data allows the image data to be transmitted at a specified frame rate. If the value exceeds the value, the quantization table is reset so that the amount of image data to be encoded is reduced, the image signal is re-encoded, and the compressed image data with the code amount adjusted is transferred, and the encoded compressed image If the code amount of the data does not reach the lower limit value based on the upper limit value of the code amount, the quantization table is reset so that the data amount of the encoded image data is increased, and the image signal is re-encoded to reduce the code amount. The adjusted compressed image data is transferred.

  An image data transfer method according to a thirteenth aspect of the present invention is the image data transfer method according to any one of the eighth to twelfth aspects, wherein the encoded data amount of the compressed image data exceeds a predetermined data amount. The compressed image data in which the code amount is adjusted by changing the image size of the image signal to be transferred is transferred.

  According to the present invention, in the case of a scene that does not require high-quality video data, such as camera control such as camera pan / tilt by the photographing apparatus side or operation of the photographer's camera, the transfer rate depends on the transfer rate. By reducing the data amount, the delay of the video data after the pan / tilt operation and the camera operation is reduced, and the real-time video can be transmitted as much as possible.

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of a photographing apparatus and an image data transfer method of the present invention will be described below in detail with reference to the drawings.
(Embodiment 1)
FIG. 1 shows a block diagram of a photographing device and a terminal device that receives and displays image data from the photographing device in Embodiment 1 of the present invention.

  In FIG. 1, reference numeral 1 denotes a photographing device that transmits image data obtained by photographing a subject, 2 denotes a terminal device that receives and displays image data transmitted from the photographing device 1, and 3 denotes a network such as the Internet. The camera 1 and the terminal device 2 are connected to enable data transfer.

  Reference numeral 4 denotes an image pickup apparatus that is provided in the image pickup apparatus 1 and takes an image of a subject, converts it into an image signal, and outputs the image signal. 4a is a camera composed of a CCD or CMOS image sensor that photoelectrically converts the subject, 4b is a signal that performs various signal processing on the image signal photoelectrically converted by the camera 4a, and the luminance (Y) and color difference signal of the image data. An image signal generation unit having an image processing function for converting to (Cb, Cr). Reference numeral 4 c denotes a camera state detection unit that includes an angular velocity sensor 4 d and detects the pan / tilt operation of the imaging device 4 and the movement / operation of the photographing device 1. Note that panning is to change the shooting direction of the imaging device 4 to the left and right directions, and tilt is to change the shooting direction of the imaging device 4 to the vertical direction. Further, the camera state detection unit 4c includes the camera control unit 4e, so that it is possible to detect camera control operation states such as camera zoom control, focus control, and white balance control. In addition, you may make it have a camera control information acquisition means which acquires camera control information instead of the camera control means 4e.

  The encoding preprocessing unit 5 preprocesses the image signal input from the image signal generation unit 4b into a form suitable for encoding before encoding. For example, image size change, image signal filtering, signal level correction, noise reduction, and the like are performed. The image size will be described later.

  The image signal encoding unit 6 compresses and encodes the image signal generated by the pre-encoding processing unit 5 and generates compressed image data that can be sent to the network. Reference numeral 6a denotes DCT processing means for performing discrete cosine transform in units of 8 × 8 in order to express an image signal with frequency components (hereinafter referred to as DCT coefficients). Reference numeral 6b denotes quantization means for linearly scalar quantizing the DCT processing coefficient calculated by the DCT processing means 6a, and reference numeral 6c denotes a quantization table used when the DCT coefficient is linearly scalar quantized. The quantization means 6b performs linear scalar quantization on a coefficient basis with the coefficient of the quantization table 6c as a step size. This can be realized, for example, by dividing each DCT coefficient by each coefficient that is an element of the quantization table 6c. Reference numeral 6d denotes multiplication means for multiplying a coefficient of the quantization table 6c by a predetermined number in order to reduce the amount of data to be transferred in the linear scalar quantization in the quantization means 6b by the quantization table 6c. Reference numeral 6e denotes encoding means for encoding the quantized coefficient calculated by the linear scalar quantization of the quantizing means 6b.

  Quantization calculated by the quantization means 6b by quantizing the DCT processing coefficient calculated by the DCT processing means 6a performed by the image signal encoding unit 6 by multiplying each coefficient of the quantization table 6c by a predetermined number. The technique for adjusting the coefficient and adjusting the code amount of the compressed image data encoded by the encoding unit 6e is a known technique and is described in JP-A-2002-354461. A detailed method of adjusting the code amount performed by the image signal encoding unit 6 in the first embodiment will be omitted.

  The network interface unit 7 includes an image data transmission unit 7a and a network transfer rate acquisition unit 7b. The network interface unit 7 adds a header or performs format conversion on the data encoded by the image signal encoding unit 6 if necessary for transmission of the image data, thereby performing image data transmission means 7a. The compressed image data encoded by the above is transmitted. Further, the network interface unit 7 acquires the data transfer rate on the connected network by the network transfer rate acquisition means 7b. Here, in Embodiment 1 of the present invention, the data transfer rate when data is transmitted to the terminal device 2 is acquired by the network transfer rate acquisition means 7b. The network transfer rate acquisition means 7b acquires a data transfer rate between other connected network interfaces, and the data transfer while the camera state detection unit 4c detects a change in the shooting state and control state of the camera. The rate is notified to the code amount adjustment unit 8.

  For this data transfer rate, the compressed image data encoded by the image signal encoding unit 6 by the photographing device 1 is converted into a file such as an HTML file and temporarily stored, and the terminal device 2 acquires the file. In this case, the file size can be obtained by measuring the size of the file that has already been transmitted and the time from the transmission request for the file from the terminal device 2 until the transmission request for the next file is received.

  Also, when the terminal device 2 continues to transmit the compressed image data encoded by the photographing device 1 to the network address of the terminal device 2, when the terminal device 2 receives the image data, the photographing device 1 indicates completion of reception of the image data. An ACK signal is returned. Thereby, the network interface unit 7 can acquire the data transfer rate by measuring the data amount of the transmitted compressed image data and the time from when the compressed image data is transmitted until the ACK signal is received.

  The code amount adjustment unit 8 transfers the network data obtained from the network interface unit 7 while the camera state detection unit 4c detects pan / tilt detection of the imaging device 4 and movement / operation of the imaging device 1. The pre-encoding processing unit 5 or the image signal encoding unit 6 is controlled based on the rate, the frame rate at which the image data is transferred, and the code amount of the compressed image data encoded by the image signal encoding unit 6. Thereby, the code amount of the compressed image data encoded by the image signal encoding unit 6 can be adjusted, and the compressed image data whose code amount has been adjusted can be transmitted from the network interface unit 7 to the network 3.

Further, the code amount adjustment unit 8 detects the change in the shooting state of the camera and the control state of the camera by the camera state detection unit 4c, and the network data transfer rate obtained from the network interface unit 7 and the image data. Is compressed by the image signal encoding unit 6 by multiplying each coefficient of the quantization table 6c by a predetermined number based on the frame rate at which the image signal is encoded and the code amount of the image data encoded by the image signal encoding unit 6. Adjust the code amount of image data.
Note that the frame rate at which image data is transferred may be determined by the photographing apparatus 1 or the terminal apparatus 2.

Next, a detailed control procedure of the code amount adjusting unit 8 is shown in FIG.
In FIG. 2, in step S101, the code amount adjustment unit 8 determines an initial value of a quantization table multiplication value for multiplying each coefficient of the basic quantization table by a predetermined value and a frame rate at which image data is transferred. Here, the frame rate for transferring the image data is, for example, the (Fn / S) rate. The initial value of the quantization table multiplication value is set so that a quantization table used in a normal state in which no pan / tilt operation or movement / operation of the camera is performed is generated from the basic quantization table. For example, when the basic quantization table is used in a normal state, “1” is set as the initial value of the quantization table multiplication value.

  Also, when quantization is performed using the quantization table used in the normal state, transfer may not be possible at the desired frame rate, but the camera is already shooting the subject at the optimal shooting position and camera control. Since there is no movement due to camera operation or control, the viewer does not need a high frame rate.

  Next, in step S102, the code amount adjustment unit 8 receives the camera state signal sent from the camera state detection unit 4c, detects the pan / tilt operation of the imaging device 4, and detects the movement / operation of the imaging device 1. Then (Yes in step S102), in step S103, the data transfer rate for transferring data to the image data transmission destination terminal device 2 is acquired by the network interface unit 7.

  In step S103, the code amount upper limit value (Jmax) is set to (Jmax) = (Dm / S) from the acquired data transfer rate, for example, (Dm / S) and the frame rate (Fn / S) set in step S101. ) / (Fn / S) -α. α is a data amount for adjusting the value of the code amount upper limit value in order to cope with a decrease in the data transfer rate. The code amount lower limit (Jmin) is desirably set so as to satisfy the code amount that can prevent the excessive deterioration and secure the desired image quality. For example, the data amount of 90% of Jmax is (Jmin), Jmin) = 0.90 × (Jmax). The code amount upper limit value and the code amount lower limit value may be the same value (Jmin) = (Jmax).

  Next, in step S104, the code amount (J) of the compressed image data generated by the encoding unit 6e is acquired. In step S105, the code amount (J) of the compressed image data, the code amount upper limit value (Jmax), and Compared with the code amount lower limit (Jmin), when (J)> (Jmax) or (J) <(Jmin), that is, the code amount of the compressed image data is in the range between the code amount upper limit value and the code amount lower limit value. If it is outside, the multiplication value of the quantization table is changed in step S106.

  When the multiplication value of the quantization table is changed in step S106 (Yes in step S105), when (J)> (Jmax), the multiplication value of the quantization table is increased so that the code amount is decreased, and (J) When <(Jmin), addition / subtraction is performed so as to decrease the multiplication value of the quantization table so that the code amount increases.

  At this time, a value to be added or subtracted from the multiplication value (Qx) of the current quantization table according to the correlation between the code amount (J) of the compressed image data, the code amount upper limit value (Jmax), and the code amount lower limit value (Jmin). And the multiplication value of the next quantization table is obtained.

FIG. 3 is a multiplication value calculation table of the quantization table in the photographing apparatus according to Embodiment 1 of the present invention.
A method of determining a new quantization table multiplication value from the current quantization table multiplication value (Qx) will be described with reference to FIG. Here, according to the correlation between the code amount (J) of the compressed image data, the code amount upper limit value (Jmax), and the code amount lower limit value (Jmin), the multiplication value (Qx) of the current quantization table is divided into six stages. ) Is added / subtracted to determine the multiplication value of the new quantization table.

  As shown in the first to third stages of the table of FIG. 3, the generated code amount (J) is 1.5 times or more, or 1.2 to 1.5 times the code amount upper limit (Jmax). Or from 1.0 to 1.2, a constant is added to the multiplication value (Qx) of the current quantization table. As a result, when each DCT coefficient is quantized with a quantization table multiplied by a predetermined value (Qx) of this quantization table, each coefficient in the quantization table becomes large, each DCT coefficient after quantization becomes small, What is evaluated as “0” increases and the amount of encoded data decreases. Conversely, as shown in the fourth to sixth stages of the table in FIG. 3, the generated code amount (J) is 0.5 times or less of the code amount lower limit (Jmin), or from 0.8 times. When 0.5 times or 0.8 times to 1.0 times, a constant is subtracted from the multiplication value (Qx) of the current quantization table. As a result, when each DCT coefficient is quantized with a quantization table multiplied by a predetermined value (Qx) of this quantization table, each coefficient in the quantization table becomes smaller and each DCT coefficient after quantization becomes larger. In addition, the number evaluated as “0” decreases, and the amount of encoded data increases.

  When (Jmin) ≦ (J) ≦ (Jmax), the code amount (J) of the compressed image data generated by the encoding unit 6e is transferred at the frame rate specified in the current traffic state. Since the code amount is appropriate, the multiplication value (Qx) of the current quantization table is not particularly changed.

  Then, when the pan / tilt operation of the imaging device 4 and the movement / operation of the photographing device 1 are not detected in step S102 of FIG. 2 and the photographing position and camera control are determined (No in step S102), the quantum The multiplication value (Qx) in the conversion table is returned to the initial value (step S107). As a result, the shooting position and camera control are determined, and a high-quality image is transmitted.

  As described above, when the code amount of the compressed image data is larger than the code amount upper limit value, or when the code amount of the compressed image data is smaller than the code amount lower limit value, the multiplication value of the quantization table is changed, so that the compressed image data is changed. The amount of data can be reduced and can be transferred in a desired frame. In addition, by determining the multiplication value of the quantization table according to the correlation between the code amount of the compressed image data and the code amount upper limit value and the code amount lower limit value, it is possible to prevent excessive image quality degradation, and immediately to a desired value. You can transfer at the transfer rate.

  When obtaining the multiplication value (Qx) of the quantization table, not only adding and subtracting a constant to each coefficient of the multiplication value (Qx) of the quantization table, but also each coefficient of the multiplication value (Qx) of the quantization table By adding and subtracting a weighted value, the degradation in image observation is small, and the data amount of the quantized data can be effectively reduced. In addition, by separately setting the multiplication value (Qx) of the quantization table for the luminance signal quantization table and the color difference signal quantization table, it is possible to reduce the quantized data with little deterioration in image quality.

  Next, the terminal device 2 will be described. The network interface unit 9 can receive the compressed image data sent from the network interface unit 7 of the photographing apparatus 1 via the network 3. When receiving the image data from the photographing apparatus 1, the network interface unit 9 It is possible to transmit an ACK signal indicating reception completion to one network interface unit 7. The network interface unit 9 can also receive data from other imaging devices connected to the network 3.

  Reference numeral 10 denotes an image expansion unit that can expand the received compressed image data. Reference numeral 11 denotes an image display unit that displays image data that can be displayed by the image expansion unit 10. The image display unit 11 is a browser that can display image data in an HTML file, for example.

  The terminal device 2 can also receive general data other than compressed image data, and receives text data from a personal computer connected to the network 3 while receiving and displaying the image data sent from the photographing device 1. It is also possible to do.

  As described above, when the camera state detection unit 4c detects the pan / tilt operation of the imaging device 4 and the movement / operation of the imaging device 1, the imaging device 1 acquires the data transfer rate for the terminal device 2 from the network interface unit 7, Based on the data transfer rate and the designated frame rate, a code amount upper limit value and a code amount lower limit value, which are data amounts that do not cause excessive image quality degradation at the designated frame rate, are obtained. Then, the data amount of the compressed image data encoded by the encoding unit 6e is compared with the code amount upper limit value and the code amount lower limit value, and the multiplication value of the quantization table for multiplying each coefficient of the quantization table 6c by a predetermined value is obtained. Then, the quantization table of the image signal to be encoded next is updated.

  Thereby, when the data amount of the compressed image data encoded by the encoding unit 6e is larger than the code amount upper limit value, by using a quantization table obtained by adding a constant to the multiplication value of the current quantization table, Generate compressed image data smaller than the amount of data encoded with the current quantization table, and if it is less than the lower limit of the code amount, use a quantization table obtained by subtracting a constant from the multiplication value of the current quantization table. Thus, compressed image data larger than the data amount encoded by the current quantization table can be generated. In addition, compressed image data encoded with a quantization table obtained by multiplying each coefficient of the quantization table 6c by a predetermined number when the pan / tilt operation of the imaging device 4 is detected and the movement / operation of the photographing device 1 is detected. The amount of data is smaller than that when the image data is encoded by the quantization table 6c when the pan / tilt operation of the image pickup device 4 and the movement / operation of the image pickup device 1 are not detected.

  In the terminal device 2, when the camera of the photographing device 1 is performing pan / tilt operation or zoom control, the compressed image data can be received at a specified frame rate and displayed on the image display unit 11. it can. At this time, the image data is reduced and the compressed image data is sent from the image capturing device 1 so that the image data can be received at the frame rate specified by the terminal device 2, but the camera of the image capturing device 1 performs the pan / tilt operation. When the terminal device 2 performs zooming or zoom control, the terminal device 2 does not require high-quality image data. Since the compressed image data from the photographing apparatus 1 is transmitted at the specified frame rate, when the camera operation or control is determined by the photographing apparatus 1, the determined image data is not excessively delayed. Can be received.

  Further, the pan / tilt operation of the imaging device 4 is detected and photographed by determining the multiplication value of the quantization table according to the correlation between the code amount of the compressed image data and the code amount upper limit value and code amount lower limit value. When the movement / operation of the apparatus 1 is detected, compressed image data that can be transferred immediately at a designated transfer rate can be generated, and the pan / tilt operation of the imaging apparatus 4 and the movement / operation of the imaging apparatus 1 are short. Even when it is time, the determined image data can be received without delay.

  As described above, according to the first embodiment, the image signal generation unit 4b that generates an image signal from the video captured by the camera, and the preprocessing for encoding the image signal generated by the image signal generation unit 4b A pre-encoding processing unit 5 that performs image processing, an image signal encoding unit 6 that performs compression processing on the image signal processed by the pre-encoding processing unit 5, and transmits the encoded image data over the network. The network interface unit 7 for acquiring the transfer rate of the camera, the camera state detection unit 4c for detecting the photographing state of the camera and the control state of the camera, the pre-encoding processing unit 5 and the image signal encoding unit 6; Since the code amount adjustment unit 8 for adjusting the code amount of the image data is provided, the code amount of the compressed image data encoded by the image signal encoding unit is adjusted to reduce the data amount. There is an effect that it is possible to reduce the delay of the data.

(Embodiment 2)
FIG. 4 is a block diagram of the photographing apparatus according to the second embodiment of the present invention, in which a temporary storage means 6f is provided before the DCT processing means 6a of the image signal encoding unit 6 of the photographing apparatus according to the first embodiment. It is.

  The temporary storage unit 6f temporarily stores the image signal data processed by the pre-encoding processing unit 5, and the code amount adjustment unit 8 may re-encode the image signal data stored in the temporary storage unit 6f. it can. Also, the code amount adjustment unit 8 can update the quantization table and encode the image signal data once encoded. At this time, the code amount adjustment unit 8 multiplies each coefficient of the quantization table 6c by a predetermined amount and encodes the compressed image data encoded by the image signal encoding unit 6, and the network amount obtained from the network interface unit 7. Based on the data transfer rate and the frame rate for transferring the image data, each coefficient of the quantization table 6c is multiplied by a predetermined number, and the image signal of the image data encoded by the image signal encoding unit 6 is re-encoded.

  Next, FIG. 5 shows a detailed control procedure of the code amount adjustment unit 8 in the second embodiment. The processing from step S101 to S107 is the same as in the first embodiment. When (J)> (Jmax) or (J) <(Jmin) in step S105, that is, when the code amount of the compressed image data is outside the range between the code amount upper limit value and the code amount lower limit value (step S105). In step S106, the code amount adjustment unit 8 changes the multiplication value of the quantization table. In step S108, the code amount adjustment unit 8 uses the multiplication value changed in step S106 with respect to the image signal encoding unit 6. The image signal is encoded again using the updated quantization table. Then, the image signal encoding unit 6 re-encodes the image signal stored in the temporary storage unit 6f using the updated quantization table, and the code amount (J of the compressed image data re-encoded in step S109). ), The code amount upper limit value (Jmax), and the code amount lower limit value (Jmin). At this time, similarly to step S105, when (J)> (Jmax) or (J) <(Jmin), that is, the code amount of the re-encoded compressed image data is the code amount upper limit value and the code amount lower limit value. When it is out of range (No in step S109), in step S106, the code amount adjustment unit 8 changes the multiplication value of the quantization table and re-encodes the image signal.

  In this way, when the code amount of the encoded compressed image data exceeds the code amount upper limit value at which the image data can be transmitted at the specified frame rate, the data amount of the encoded image data is reduced. If the code amount of the encoded compressed image data does not reach the lower limit value based on the upper limit value of the code amount, the quantizer is set so that the data amount of the encoded image data is increased. The encoding table is reset, and the image signal of the encoded image data is re-encoded.

  In the first embodiment, during the pan / tilt operation and movement / operation of the camera, the next image signal is encoded with the quantization table updated based on the changed multiplication value. Although compressed image data having a specified frame rate and no excessive image quality degradation is generated and transferred, in the second embodiment, by re-encoding the image signal based on the changed multiplication value, It becomes possible to transfer compressed image data without excessive image quality degradation at a specified frame rate.

  In the first embodiment, the next image signal is encoded with the updated quantization table. Therefore, depending on the next input image signal, the updated quantization table has an expected data amount. There are cases where encoding cannot be performed. However, in the second embodiment, since re-encoding is performed using the updated quantization table, it is possible to reliably encode the expected data amount.

  As described above, according to the second embodiment, the image signal encoding unit 60 includes the temporary storage unit 6f that temporarily stores the image signal processed by the pre-encoding processing unit 5. By updating the quantization table with respect to the encoded image signal data and re-encoding the image signal, there is an effect that it is possible to reliably encode the expected data amount.

(Embodiment 3)
FIG. 6 is a block diagram of the photographing apparatus according to the third embodiment of the present invention. The pre-encoding processing unit according to the second embodiment includes sampling means 5a and image construction means 5b.

  In FIG. 6, the sampling means 5a samples the image signal at a predetermined sampling rate, and the image construction means 5b reconstructs the image signal based on the data sampled by the sampling means 5a. For example, when the image signal input from the image signal generation unit 4b is sampled by the sampling unit 5a at a sampling rate of ½ in each of the vertical direction and the horizontal direction, the image signal is reconstructed by the image configuration unit 5b. An image signal having an image size that is 1/4 of the input image signal can be generated. Thereby, the data amount of the image signal generated by the image signal generation unit 4b is reduced, and as a result, the data amount of the compressed image data transmitted on the network is reduced.

  The code amount adjustment unit 8 transfers the network data transfer rate and image data obtained from the network interface unit 7 while the camera state detection unit 4c detects changes in the shooting state of the camera and the control state of the camera. Image data is sampled by the sampling means 5a, the image size is reduced by the image construction means 5b, and the image signal is encoded based on the frame rate to be encoded and the code amount of the compressed image data encoded by the image signal encoding unit 60. The code amount of the compressed image data to be encoded by the unit 60 is adjusted.

  Next, FIG. 7 shows a detailed control procedure of the code amount adjustment unit 8 in the third embodiment. The processing from step S101 to S108 is the same as that in the second embodiment. If (J) <(Jmin) in step S110, that is, if the code amount of the compressed image data is smaller than the code amount lower limit value (Yes in step S110), step S109 in the second embodiment is performed as in step S109. The image signal is re-encoded using the quantization table updated with the multiplication value changed in S106. Next, when (J)> (Jmax) in step S110, that is, when the code amount of the compressed image data is larger than the code amount upper limit value (Yes in step S111), the code amount adjustment unit 8 performs encoding in step S112. The image size is reduced with respect to the preprocessing unit 5. The sampling rate at this time may be, for example, (Jmax) / (J).

  In this manner, when the code amount of the compressed image data encoded by the image signal encoding unit 60 exceeds a predetermined data amount, the image size of the image signal generated by the pre-encoding processing unit 50 can be changed. it can.

  Even if the code size (J) and the code size upper limit (Jmax) of the compressed image data are greatly deviated and the quantization table is updated and re-encoded, the image size can be changed at a predetermined transfer rate. This is effective when compressed image data having a data amount that can be transmitted cannot be generated. In addition, in the case of causing extreme image quality degradation by changing the multiplication value of the quantization table, the image quality of the image to be compression-encoded is reduced to prevent the extreme image quality degradation by reducing the data amount of the image signal, Can maintain a certain level of image quality.

  Thereby, for example, the image capturing device 1 transmits image data to the terminal device 2 via the network, and image data or text data is transmitted from another image capturing device connected to the network 3. Even when the data transfer rate from the device 1 to the terminal device 2 is extremely lowered, when the camera of the photographing device 1 performs pan / tilt operation or zoom control, the image size is reduced by reducing the image size. The image data having the above image quality can be transferred at a predetermined frame rate, and after the operation and control of the camera are determined, the image data can be received without excessive delay.

In the third embodiment, the code amount (J) and the code amount upper limit value (Jmax) of the compressed image data encoded at the first time in step S111 are compared, but the quantization table is updated and re-encoding is performed. The code amount (J) and the code amount upper limit (Jmax) of the compressed image data that has been converted may be compared to determine whether to change the image size. In addition, a threshold value may be set to determine whether to change the image size with respect to the encoded code amount, and it may be determined whether to change the image size.
In each of the embodiments, the image signal encoding units 6 and 60 have been described as being encoded by JPEG, but the same applies to other encoding methods for image compression.

  As described above, according to the third embodiment, the pre-encoding processing unit 50 discretizes and samples the image signal from the image signal generation unit 4b, and is sampled by the sampling unit 5a. Since the image forming unit 5b that forms an image from data is included, there is an effect that the image size of the image signal can be changed to reduce the data amount of the compressed image data transmitted on the network.

  The image capturing apparatus and the image data transfer method according to the present invention provide compressed image data based on a data transfer rate for transmitting image data to a terminal when the camera is performing pan / tilt operation or zoom control. Such as a TV conference system and a video chat system that are required to transmit the determined image data to the terminal without excessive delay when the operation and control of the camera are determined. Suitable for sending and receiving image data.

Configuration diagram of photographing apparatus according to Embodiment 1 of the present invention The flowchart which showed the process sequence of the code amount control part of the imaging device in Embodiment 1 of this invention. Multiplication value calculation table of quantization table of photographing apparatus according to Embodiment 1 of the present invention Configuration diagram of photographing apparatus according to Embodiment 2 of the present invention The flowchart which showed the process sequence of the code amount control part of the imaging device in Embodiment 2 of this invention. Configuration diagram of photographing apparatus according to Embodiment 3 of the present invention The flowchart which showed the process sequence of the code amount control part of the imaging device in Embodiment 3 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image pick-up device 2 Terminal device 3 Network 4 Image pick-up device 4a Camera 4b Image signal generation part 4c Camera state detection part 4d Angular velocity sensor 4e Camera control means 5, 50 Encoding pre-processing part 5a Sampling means 5b Image structure means 6, 60 Image signal Encoding unit 6a DCT processing unit 6b Quantizing unit 6c Quantization table 6d Multiplying unit 6e Encoding unit 7 Network interface unit 7a Image data transmission unit 7b Network transfer rate acquisition unit 8 Code amount adjustment unit 9 Network interface unit 10 Image expansion unit 11 Image display

Claims (13)

An image signal generator for generating an image signal from video captured by the camera;
A pre-encoding unit that performs pre-encoding processing on the image signal generated by the image signal generation unit;
An image signal encoding unit that performs compression processing on the image signal processed by the pre-encoding processing unit;
A network interface unit for transmitting encoded image data over a network and obtaining a network transfer rate;
A camera state detection unit for detecting the shooting state of the camera and the control state of the camera;
A code amount adjustment unit that controls the pre-encoding processing unit and the image signal encoding unit and adjusts the code amount of image data;
While the camera state detection unit detects changes in the shooting state of the camera and the control state of the camera, the network data transfer rate from the network interface unit, the frame rate for transferring image data, and the image signal code Based on the code amount of the compressed image data encoded by the encoding unit, the code amount of the compressed image data is adjusted by controlling the pre-encoding processing unit and the image signal encoding unit, and the code amount is adjusted. Transmitting image data from the network interface unit to the network;
An imaging apparatus characterized by that. The imaging device according to claim 1,
The camera state detection unit
An angular velocity sensor for notifying the code amount adjustment unit of information on a change in the shooting state of the camera when detecting pan / tilt or movement of the camera;
Camera control means for notifying the code amount adjustment unit of information on the control state change of the camera when the control of the camera is started,
An imaging apparatus characterized by that. The imaging device according to claim 1,
The image signal encoding unit is
DCT processing means for performing discrete cosine transform on the image signal processed by the pre-encoding processing unit;
Quantization means for linear scalar quantization of the DCT coefficient calculated by the DCT processing means using a quantization table;
Multiplication means for multiplying each coefficient of a quantization table used in the quantization means by a predetermined number;
Encoding means for encoding the quantized data calculated by the quantization means;
The code amount adjustment unit includes:
While the camera state detection unit detects changes in the shooting state of the camera and the control state of the camera, the network data transfer rate from the network interface unit, the frame rate for transferring image data, and the image signal code Based on the code amount of the image data encoded by the encoding unit, each coefficient of the quantization table is multiplied by a predetermined amount, and the code amount of the compressed image data encoded by the image signal encoding unit is adjusted.
An imaging apparatus characterized by that. The imaging device according to claim 1,
The encoding preprocessing unit
Sampling means for discretizing and sampling the image signal from the image signal generation unit;
Image constructing means for constructing an image from the data sampled by the sampling means,
The code amount adjustment unit includes:
While the camera state detection unit detects changes in the shooting state of the camera and the control state of the camera, the network data transfer rate from the network interface unit, the frame rate for transferring image data, and the image signal code Based on the code amount of the compressed image data encoded by the encoding unit, the image pre-processing unit is controlled to change the image size of the image signal and adjust the code amount of the compressed image data.
An imaging apparatus characterized by that. The imaging device according to claim 1,
The network interface unit
Image data transmitting means for transmitting the encoded compressed image data;
Network transfer rate acquisition means for acquiring the data transfer rate on the connected network,
The network transfer rate detection means notifies the data transfer rate to the code amount adjustment unit while the camera state detection unit detects changes in the shooting state and control state of the camera.
An imaging apparatus characterized by that. In the imaging device according to claim 3,
The image signal encoding unit has temporary storage means for temporarily storing the image signal processed by the pre-encoding processing unit,
The encoding adjustment unit
The code amount of the compressed image data encoded by the image signal encoding unit by multiplying each coefficient of the quantization table by a predetermined number, the network data transfer rate obtained from the network interface unit, and the frame rate for transferring the image data And re-encoding the image signal encoded by the image signal encoding unit,
An imaging apparatus characterized by that. In the imaging device according to any one of claims 1 to 6,
If the code amount of the compressed image data encoded by the image signal encoding unit exceeds a predetermined data amount, change the image size of the image signal generated by the pre-encoding processing unit,
An imaging apparatus characterized by that. An image data transfer method for converting video captured by a camera into an image signal, performing compression processing by encoding on the image signal, and transmitting the encoded compressed image data over a network,
While detecting changes in the camera shooting state and camera control state, based on the data transfer rate of the network that transmits the image data, the frame rate that transfers the image data, and the code amount of the encoded compressed image data Transfer the compressed image data with the code amount adjusted,
An image data transfer method characterized by the above. The image data transfer method according to claim 8.
Code amount upper limit that allows image data to be transmitted at a specified frame rate based on the network data transfer rate and the frame rate at which image data is transmitted while detecting changes in the camera shooting state and camera control state Transfer the compressed image data with the code amount adjusted by calculating the value,
An image data transfer method characterized by the above. The image data transfer method according to claim 8.
Set the quantization table so that the amount of encoded image data is smaller when the amount of encoded compressed image data exceeds the upper limit for the amount of encoded image data that can be transmitted at the specified frame rate. To transfer the compressed image data with the code amount adjusted,
When the code amount of the encoded compressed image data does not reach the lower limit value based on the upper limit value of the code amount, the code amount is adjusted by setting the quantization table so that the data amount of the encoded image data is increased. Transfer compressed image data,
An image data transfer method characterized by the above. The image data transfer method according to claim 8.
If the code amount of the encoded compressed image data exceeds the upper limit of the code amount that enables transmission of the image data at the specified frame rate, the image signal is converted so that the image size of the image signal to be encoded is reduced. Transfer the compressed image data with the code amount adjusted,
A compressed image in which the code amount is adjusted by converting the image signal so that the image size of the image signal to be encoded becomes large when the code amount of the encoded image data to be encoded does not reach the lower limit value based on the upper limit value of the code amount Transfer data,
An image data transfer method characterized by the above. The image data transfer method according to claim 10.
If the code amount of the encoded compressed image data exceeds the code amount upper limit value at which the image data can be transmitted at the specified frame rate, the quantization table is regenerated so that the data amount of the encoded image data becomes small. Set and re-encode the image signal and transfer the compressed image data with adjusted code amount,
If the code amount of the encoded compressed image data does not reach the lower limit value based on the upper limit value of the code amount, the quantization table is reset so that the data amount of the encoded image data is increased, and the image signal is re-encoded. Compressed image data with adjusted code amount
An image data transfer method characterized by the above. The image data transfer method according to any one of claims 8 to 12,
When the code amount of the encoded compressed image data exceeds a predetermined data amount, the compressed image data in which the code amount is adjusted by changing the image size of the image signal to be encoded is transferred.
An image data transfer method characterized by the above.

Images