JP5816858B2 - Video transmission system - Google Patents

Description

  The present invention relates to a moving image transmission system, and more particularly to a moving image transmission system that transmits a moving image in real time.

  2. Description of the Related Art Conventionally, in moving image transmission systems that transmit moving images in real time, moving image data is generally compressed to reduce the data size. In order to reduce the data size of the frame data, it has been proposed to use a difference from other frame data. As such, there is one using an I picture (intra-frame encoded image) and a P picture (forward prediction encoded image). Here, the I picture is obtained by compressing frame data into a form that can be independently decompressed without any dependency on other frame data. A P picture is obtained by compressing frame data into a form represented by a difference from a previous frame in time series.

  A compression method similar to the compression method described above is employed in MPEG-2 and the like. Unlike an I picture, a P picture cannot be expanded independently, but the data size can be significantly reduced compared to an I picture. Therefore, if the first frame data of moving image data is compressed into an I picture and the subsequent frame data is compressed into a P picture, the data size can be made very small.

  By the way, when decompressing a P picture, it is necessary to refer to the previous frame data. Therefore, when an image is deteriorated due to packet loss or the like, the influence of such deterioration affects all subsequent frame data, and the quality of the displayed moving image is deteriorated. Therefore, a method of suppressing image degradation by periodically compressing and transmitting frame data to an I picture is employed. However, since I pictures are periodically transmitted, relatively large data is periodically transmitted. Will be transmitted. Also, when the frequency band that can be used for moving image data transmission is relatively narrow, it takes a long time to transmit the I picture. Therefore, there is a possibility that a delay occurs in the moving image being displayed.

  As a solution to this, Patent Document 1 discloses a method of dividing frame data into a plurality of divided frame data and compressing at least one of the plurality of divided frame data divided from the same frame data into a P picture. Has been. In the one described in Patent Document 1, the data size after compression of a plurality of frame data included in moving image data can be averaged, and the entire frame data is compressed into a reference picture such as an I picture. It can be set to a smaller value. Therefore, even when the frequency band that can be used for moving image data transmission is narrow, it is not necessary to reduce the compression data rate, frame rate, and number of colors, and it is possible to transmit moving images without degrading the quality. .

JP 2010-081140 A

  However, in the above conventional example, the average value of the data size can be reduced by averaging the data size after compression, but there is a problem that the effect of reducing the transmission amount of necessary data as a whole cannot be expected. It was. For this reason, in the conventional example, relatively large data is transmitted as in the case of regularly transmitting I pictures. In addition, the above conventional example has a problem that there is no means for preventing the deterioration of quality when the quality of the frame data to be transmitted is deteriorated due to the intense movement of an object (for example, a person) on the frame data. It was.

  The present invention has been made in view of the above points, and provides a moving image transmission system capable of reducing the average value of the data size and reducing the data transmission amount while preventing the deterioration of moving image quality. The purpose is to do.

The moving image transmission system of the present invention includes a transmitting device and a receiving device that transmit / receive moving image data to / from each other, and the transmitting device transmits the compressed frame data compressed into the reference picture once and then acquires the immediately preceding frame data and the acquired frame. Transmit compressed frame data compressed into a differential picture obtained from the difference with the data, analyze the compressed frame data to obtain a parameter for evaluating the quality of the moving image, and if the parameter falls below a threshold value, The frame data is compressed into a reference picture and transmitted to the receiving apparatus.

  In the moving image transmission system, the transmission device compresses moving image data by compressing each of a plurality of time-series frame data included in the moving image data obtained by the input unit and moving image data obtained by the input unit. A compression unit that performs compression, a transmission unit that transmits the compressed frame data compressed by the compression unit to the reception device in time series, and a parameter for analyzing the compressed frame data and evaluating the quality of a moving image An analysis unit for obtaining the compressed frame data transmitted from the transmission device, and a decompression unit for decompressing the moving image data compressed by decompressing the compressed frame data. A decompression unit that performs the display, and a display unit that displays the decompressed moving image data. The compression unit includes first frame data among the plurality of frame data. A difference between the first compression processing for compressing the frame data into the reference picture that can be decompressed independently without dependence on other frame data, and the remaining frame data of the plurality of frame data from other frame data It is preferable that the second compression process for compressing the difference picture is performed, and if the parameter falls below a threshold value in the analysis unit, the first compression process is performed on the next frame data.

  In the moving image transmission system, it is preferable that the analysis unit performs analysis based on color information of the compressed frame data.

  In this moving image transmission system, it is preferable that the analysis unit performs analysis based on luminance information of the compressed frame data.

  In the moving image transmission system, it is preferable that the analysis unit performs analysis based on a data size of the compressed frame data.

  The present invention has an effect that the average value of the data size can be reduced and the amount of data transmission can be reduced while preventing the deterioration of the quality of the moving image.

It is a block diagram of an intercom system to which an embodiment of a moving picture transmission system according to the present invention is applied. (A)-(c) is explanatory drawing when the analysis part does not determine with the quality of the moving image having deteriorated in the intercom system same as the above. (A)-(c) is explanatory drawing when the analysis part determines with the quality of a moving image having deteriorated in the intercom system same as the above. (A), (b) is explanatory drawing of the conventional compression method.

  Hereinafter, an embodiment of a moving picture transmission system according to the present invention will be described based on an example applied to a security intercom system (hereinafter simply referred to as “interphone system”).

  As shown in FIG. 1, this intercom system includes an imaging device 1 installed at the eaves of a house, a display device 2 composed of a master unit (interphone master unit) installed inside the house (indoor), and outside the house ( It is comprised with the subunit | mobile_unit (interphone subunit | mobile_unit) not shown installed in the outdoors. Note that the imaging device 1 and the slave unit are connected to the display device 2 by a transmission line. Moreover, since a conventionally well-known thing can be employ | adopted for a subunit | mobile_unit, detailed description of a subunit | mobile_unit is abbreviate | omitted.

  The imaging device 1 is a transmission device that transmits moving image data, and includes an input unit 10, a storage unit 11, a compression unit 12, a transmission unit 13, a processing unit 14, and an analysis unit 15. As prepared.

  The input unit 10 is an imaging unit having a function of imaging a predetermined imaging region, for example, and includes an imaging device (not shown) made of a solid-state imaging device such as a CCD image sensor or a CMOS image sensor. In addition, the input unit 10 includes a control device (not shown) such as a DSP (Digital Signal Processor) that performs a function as a camera together with the imaging device as a main component. When a CMOS image sensor is used as the image sensor, a one-chip camera in which the image sensor and the control device are integrally configured using system-on-chip (SoC) technology or the like is used as the input unit 10. You can also.

  The control device of the input unit 10 captures the output (charge) of the image sensor at a predetermined frame rate (for example, 30 fps). Then, the control device creates and outputs still image data such as digital data in a predetermined format (for example, YUV format), for example, RAW data, based on the captured output of the image sensor. In this way, still image data representing the image of the imaging region is sequentially output from the control device. Therefore, moving image data representing moving images (moving images) in the imaging region can be obtained by the input unit 10. The moving image data obtained from the input unit 10 is composed of a plurality of still image data in time-series order. In the following description, still-image data in time-series order constituting the moving image data (that is, one frame of the moving image data) Hit data) is called frame data.

  The storage unit 11 is a rewritable storage device (storage device) such as a flash memory, SDRAM, or SRAM, and is mainly used for storage (storage) of moving image data obtained by the input unit 10.

  The compression unit 12 compresses the moving image data in order to transmit the moving image data obtained by the input unit 10 to the display device 2. The compression part 12 consists of microcomputers, for example, and performs the following compression process by executing the program memorize | stored in memory. In addition to the microcomputer, a dedicated ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), DSP, or the like can be used for the compression unit 12. The compression of the moving image data is performed by compressing each of a plurality of time-sequential frame data constituting the moving image data. The compression unit 12 creates compressed frame data by executing one of the first compression process and the second compression process on the frame data.

  The first compression processing is processing for compressing (encoding) frame data into a reference picture such as an I picture (intra-frame encoded image). The reference picture is a still image compressed image that is independent of other frame data and can be expanded independently. As a compression method to the reference picture, a conventionally well-known method (for example, a still image compression method for obtaining JPEG, I picture, or the like) can be adopted, and detailed description thereof is omitted here.

  The second compression processing is processing for compressing the frame data into a differential picture such as a P picture (forward prediction encoded image). The difference picture is a compressed moving image obtained from a difference from other frame data. Note that a conventionally known method (for example, a moving image compression method for obtaining a P picture) can be employed as a compression method to a differential picture, and thus detailed description thereof is omitted here.

  The transmission unit 13 is a communication unit for transmitting the moving image data (compressed moving image data) compressed by the compression unit 12 to the display device 2, and encodes the compressed moving image data (transmission path encoding) to display the display device 2. Output to. Such a transmission unit 13 transmits the compressed moving image data in real time by transmitting the compressed frame data created by the compression unit 12 in time series.

  For example, the processing unit 14 performs a storage process in which a management identification code (for example, a frame number) is attached to the frame data of the moving image data sequentially output by the input unit 10 and stored in the storage unit 11. Further, the processing unit 14 determines whether or not an object (a person in this embodiment) is captured in a frame based on the frame data of the input unit 10 at a predetermined time interval (that is, a person is within the imaging range of the input unit 10). A detection process for determining whether or not it exists is executed. Such detection processing can be realized, for example, by performing image processing on frame data obtained from the input unit 10, and various methods are conventionally provided, and thus detailed description thereof is omitted. When the processing unit 14 detects an object in the detection process, the processing unit 14 performs a transmission process of transmitting moving image data obtained from the input unit 10 to the display device 2 within a predetermined period including the detection time of the target. In the transmission process, the compression unit 12 starts to compress moving image data, and the transmission unit 13 starts transmission of moving image data. For example, a microcomputer is used as the processing unit 14.

  The analysis unit 15 analyzes the frame data obtained by the input unit 10 to obtain a parameter for evaluating the quality of the moving image, and executes a process of comparing the parameter with a preset threshold value To do. When the parameter is below the threshold value, the analysis unit 15 determines that the quality of the frame data is deteriorated and the image is disturbed, that is, the quality of the moving image is deteriorated. Then, the analysis unit 15 generates a request command for requesting the next frame data to be compressed into an I picture and transmitted, and gives the command to the processing unit 14.

  The display device 2 is a receiving device having a function of receiving and displaying moving image data transmitted from the imaging device 1. Such a display device 2 includes a receiving unit 20, a storage unit 21, an expansion unit 22, a display unit 23, and a processing unit 24 as main components.

  The receiving unit 20 is a communication unit for receiving the moving image data transmitted from the imaging device 1, and sequentially decodes the signal including the compressed moving image data transmitted from the transmitting unit 13 (transmission path decoding). Execute the output process.

  Similar to the storage unit 11, the storage unit 21 is a rewritable storage device such as a flash memory, SDRAM, or SRAM.

  The decompressing unit 22 is for decompressing the compressed moving image data received by the receiving unit 20, and decompresses the compressed moving image data by decompressing each compressed frame data. Note that, similarly to the compression unit 12, the decompression unit 22 can include a microcomputer, an ASIC, an FPGA, a DSP, and the like as main components.

  The display unit 23 includes a display device (not shown) such as a liquid crystal display device (LCD) or a CRT. The display unit 23 displays the moving image data (reproduces the moving image) by displaying the compressed frame data expanded by the expansion unit 22, that is, the frames based on the frame data in time series order.

  For example, the processing unit 24 performs a recording process for recording the compressed moving image data received by the receiving unit 20 in the storage unit 21. In addition, when the receiving unit 20 receives the compressed moving image data, the processing unit 24 executes display processing for displaying the compressed moving image data on the display unit 23. In this display process, the decompression unit 22 starts decompressing the compressed moving image data, and the display unit 23 starts displaying the moving image data. Such a processing unit 24 uses a microcomputer in the same manner as the processing unit 14.

  Hereinafter, the operation of this embodiment will be described. In the following description, an I picture is used as a reference picture, and a P picture is used as a difference picture.

  While the imaging apparatus 1 is activated, the imaging unit 1 always captures a predetermined imaging area with a predetermined frame rate by the input unit 10. The frame data captured by the input unit 10 is sequentially stored in the storage unit 11 by the processing unit 14. The processing unit 14 performs the above-described detection process based on the frame data obtained from the input unit 10. As a result of the detection process, when it is determined that the object is captured, the processing unit 14 causes the compression unit 12 to start compressing the moving image data and causes the transmission unit 13 to start transmitting the moving image data.

  Here, the compression unit 12 executes the first compression process on the frame data to be transmitted first, thereby creating compressed frame data I1 obtained by compressing the frame data to be transmitted first into an I picture. Then, the compression unit 12 performs the second compression process on the frame data to be transmitted later, thereby creating compressed frame data that is compressed into a P picture. Therefore, compressed frame data I1 compressed into an I picture is first transmitted to the display device 2, and then compressed frame data P1 to P7 compressed into a P picture are transmitted to the display device 2 in time series order ( (See FIG. 2 (a)).

  In the display device 2, the compressed moving image data is received by the receiving unit 20. Then, the compressed moving image data is recorded in the storage unit 21 by the processing unit 24. The processing unit 24 causes the decompression unit 22 to start decompressing the compressed moving image data, and causes the display unit 23 to start displaying the decompressed compressed movie data, that is, the movie data. The decompression unit 22 first decompresses the compressed frame data I1 alone and causes the display unit 23 to display the decompressed frame data A1. The processing unit 24 expands the subsequent compressed frame data P1 to P7 to frame data A2 to A8 with reference to the immediately preceding frame data A1 to A7, and displays the expanded frame data A2 to A8 in time series order. 23 (see FIG. 2B).

  As described above, the first compression process is executed only for the frame data to be transmitted first, and the second compression process is executed for the frame data to be transmitted later, so that the data transmission amount as a whole Can be reduced. In other words, the total data size of the compressed frame data to be transmitted can be reduced as compared with the case where frame data is periodically compressed into an I picture and transmitted (see FIGS. 4A and 4B). it can. The frame data to be transmitted first is compressed into an I picture, but the other frame data is compressed into a P picture, so that the average data size can be reduced (see FIG. 2C).

  By the way, since the reproduction of a moving image using a P picture is based on the premise that the immediately preceding frame data is referred to, there is a possibility that the quality of the displayed moving image is deteriorated when the movement of the object becomes intense. Therefore, the processing unit 14 of the imaging apparatus 1 causes the compression unit 12 to execute the above processing and causes the analysis unit 15 to analyze moving image data.

  The analysis unit 15 obtains a parameter by calculating, for example, the dispersion and frequency distribution of color components from the previous compressed frame data and the current compressed frame data, among the compressed frame data, and the parameter and a preset threshold value are calculated. Compare Note that the threshold value is obtained experimentally based on the quality of a moving image that is allowed to be viewed from the human eye by, for example, reproducing the moving image in a state where an object is actually present. Then, when the parameter falls below the threshold, the analysis unit 15 determines that the quality of the moving image has deteriorated and generates a request command. This request command is given to the processing unit 14.

  The processing unit 14 causes the compression unit 12 to execute the first compression processing by interpreting the given request command. The compression unit 12 executes the first compression process to create compressed frame data obtained by compressing the frame data to be transmitted next into an I picture. Therefore, when the request command is generated, compressed frame data compressed into an I picture is transmitted to the display device 2 instead of compressed frame data compressed into a P picture.

  For example, as shown in FIGS. 3A to 3C, it is assumed that a request command is given to the processing unit 14 when the compressed frame data P5 is transmitted to the display device 2. In this case, the next frame data A7 displayed on the display unit 23 is obtained by expanding the compressed frame data I2 obtained by compressing the image into the I picture transmitted from the imaging device 1, so that the quality of the moving image is improved again. As a result, it is possible to prevent a reduction in the quality of the moving image reproduced on the display device 2.

  As described above, only the frame data to be transmitted first is compressed into an I picture (reference picture), and the other frame data is compressed into a P picture (difference picture). Therefore, the average value of the data size is reduced, and The amount of data transmission can be reduced. If the analysis unit 15 analyzes the quality of the moving image and determines that the quality of the moving image has deteriorated, the compressed frame data image-compressed into an I picture (reference picture) is transmitted to the display device 2. Decline can be prevented. Furthermore, since the imaging device 1 can analyze a sudden change in the object, it is not necessary to exchange extra information as compared with the case of analysis using the display device 2, and the video that the user views is distorted. Can be reduced.

  By the way, the analysis unit 15 of the present embodiment obtains parameters by calculating the dispersion and frequency distribution of the color components, that is, analyzes based on the color information of the compressed frame data. For this reason, in the present embodiment, it is possible to detect the disturbance of the video accompanying the color change. On the other hand, the analysis unit 15 may obtain the parameter by calculating the variance of the luminance component and the frequency distribution from the previous compressed frame data and the current compressed frame data among the compressed frame data. That is, the analysis unit 15 may analyze based on the luminance information of the compressed frame data. In this case, similarly to the case of analyzing based on the color information, it is possible to detect the disturbance of the video accompanying the color change. In addition, since the luminance component is a component that tends to feel uncomfortable for humans, it is possible to determine the deterioration of the quality of the moving image at a level close to the human experience.

  Further, the analysis unit 15 may obtain the parameter by calculating the data size of the compressed frame data, that is, may analyze based on the data size of the compressed frame data. In this case, since the deterioration of the quality of the moving image can be determined by a one-dimensional number called a data size, the analysis unit 15 can be configured by software with a small processing load or hardware with a small circuit scale. Therefore, costs can be reduced as compared with the case of analyzing based on color information and luminance information.

  The above-described configuration of the moving image transmission system of the present embodiment is merely an embodiment of the present invention, and is not intended to limit the technical scope of the present invention to the above example, and does not depart from the spirit of the present invention. Can be changed to a degree. For example, in the above example, a P picture is adopted as the differential picture, but a B picture (bidirectional predictive coded image) may be used instead of or together with the P picture.

  In the present embodiment, the processing unit 14 of the imaging device 1 executes a detection process, and when an object is detected by the detection process, transmission of moving image data is started. However, transmission of moving image data may be started when moving image data is obtained from the imaging apparatus 1 regardless of the presence or absence of an object. That is, the timing for starting transmission of moving image data can be appropriately set according to the purpose of use of the moving image transmission system.

  In the present embodiment, the imaging device 1 and the display device 2 constitute the moving image transmission system of the present invention. Here, in the interphone system, since a device with a camera is often used as the slave unit, such a slave unit with a camera and the display device 2 may constitute the moving picture transmission system of the present invention. In this case, the slave unit is a transmission device, and the display device 2 is a reception device.

  In this embodiment, the moving image transmission system of the present invention is applied to an intercom system. However, the moving image transmission system of the present invention can also be applied to a system such as a monitoring system. That is, the moving image transmission system of the present invention is applicable as long as moving image data is transmitted in real time.

1 Imaging device (transmitting device)
DESCRIPTION OF SYMBOLS 10 Input part 12 Compression part 13 Transmission part 15 Analysis part 2 Display apparatus (reception apparatus)
20 receiving unit 22 decompressing unit 23 display unit A1 to A8 frame data I1, I2 I picture (reference picture)
P1 to P7 P picture (difference picture)

Claims (5)

A transmission device and a reception device that transmit and receive moving image data to and from each other, and the transmission device transmits a compressed frame data compressed into a reference picture once and then obtains a difference obtained from a difference between immediately preceding frame data and acquired frame data The compressed frame data compressed into the picture is transmitted, and the compressed frame data is analyzed to obtain a parameter for evaluating the quality of the moving image. When the parameter falls below the threshold, the next frame data is compressed into the reference picture. And transmitting it to the receiving apparatus.   The transmission device includes: an input unit for obtaining moving image data; and a compression unit that compresses moving image data by compressing each of a plurality of frame data in time series included in the moving image data obtained by the input unit. A transmission unit that transmits the compressed frame data compressed by the compression unit to the reception device in time series, and an analysis unit that analyzes the compressed frame data and obtains parameters for evaluating the quality of a moving image. And the receiving device includes a receiving unit that receives the compressed frame data transmitted from the transmitting device, a decompressing unit that decompresses the moving image data compressed by decompressing the compressed frame data, and a decompression unit. A display unit configured to display the moving image data, and the compression unit uses the first frame data of the plurality of frame data as a dependency on other frame data. A first compression process for compressing the reference picture that is irrelevant and can be decompressed independently; and a second compression process that compresses the remaining frame data of the plurality of frame data into the differential picture that includes differences from other frame data. 2. The moving picture transmission system according to claim 1, wherein the first compression processing is executed on the next frame data when the parameter falls below a threshold in the analysis unit. .   The moving image transmission system according to claim 2, wherein the analysis unit performs analysis based on color information of the compressed frame data.   The moving image transmission system according to claim 2, wherein the analysis unit analyzes based on luminance information of the compressed frame data.   The moving image transmission system according to claim 2, wherein the analysis unit analyzes based on a data size of the compressed frame data.

Images