KR101364238B1 - Video transmission and reception system, and method thereof operating in accordance with channel status - Google Patents

Abstract

The present invention relates to an image transmission / reception system and a method for adaptively operating according to a channel state. The present invention relates to an operation mode of an encoding unit and a decoding unit according to a channel state. By operating, recording is possible even if the channel condition is bad.In addition, in the adaptive mode and the disabled mode, the background image and the foreground image are processed separately, and the bit rate is adaptively adjusted according to the channel state to enhance the transmission reliability. The present invention provides a video transmission / reception system and method for robustly operating in a wireless channel environment by setting a buffer in a subsystem so that the foreground image is not lost even in a disabled mode in which the wireless channel is disconnected.
To this end, the present invention provides a method for transmitting an image, the method comprising: transmitting photographed image information photographed under the control of a camera controller to an encoder; Determining a channel state by the channel state determiner; A mode switching step of a mode switching unit performing a mode switching operation between a normal mode, an adaptive mode, and a disabled mode based on the channel state determination result; In the normal mode, as a result of the mode switching, the encoding unit encoding the captured image information, and a transmitter transmitting the encoded captured image information through a wireless channel; When the mode is changed, in the adaptive mode, the encoding unit separates the foreground image and the background image by encoding and storing the foreground image information, and transmitting the encoded foreground image information through a wireless channel; And when the disabled mode is disabled, the encoder separates the foreground image and the background image from the foreground image and encodes the foreground image information. When the image is converted to the normal mode or the adaptive mode, the transmitter transmits the stored foreground image information to the wireless channel. Transmitting through.

Description

Video transmission and reception system adaptive to channel status and its method {Video transmission and reception system, and method according operating in accordance with channel status}

The present invention relates to an image transmission / reception system and a method thereof, and more particularly, to an image transmission / reception system and method for adaptively defining an operation mode according to a channel state so as to transmit, receive, and store photographing information without losing the information. It is about.

In the following embodiment, as an example of the image transmission and reception system and method of the present invention, a CCTV system and method for transmitting and receiving an image captured by a closed circuit TV (CCTV) camera through a wireless channel will be described as an example. This is not limited to this in advance.

In general, CCTV systems used for security and surveillance have been replaced by digital cameras and digital video recorders (DVRs), starting with the initial system using analog cameras and VCRs (Video Cassette Recorders). As development progresses, intelligent CCTVs that track and monitor an object are equipped with motion analysis functions.

Digital video has a huge amount of data, so compression processing is essential in terms of storage and transmission. Until now, a number of standards for image compression have been established, and among them, the Moving Picture Expert Group (MPEG) under the ISO / IEC (International Standard Organization / International Electrotechnical Commission) has adopted MPEG-1, MPEG-2, MPEG-4, etc. The compression standard was established, and the ITU-T (International Telecommunication Union-Telecommunication Standard Sector) proposed the H.26x series of image compression standards.

On the other hand, until now, the CCTV system itself has been operated in a wired manner with its own network as a closed circuit in the communication method between the CCTV camera and the control center / DVR. have. In recent years, as the IEEE 802.11 Wireless Local Area Network (WLAN) standard has been commercialized, IP-based open protocols have been changed, and IP cameras have been widely used.

However, as wireless LANs are used by many users, such as smartphones and wireless Internet, the number of channel users increases, and the wireless channel environment changes significantly according to the propagation path characteristics, resulting in severe changes in quality of service (QoS). There is a problem. In addition, a multipath environment caused by obstacles between trees (AP) and CCTV cameras, trees, and moving objects deteriorates wireless quality.

In particular, as the number of users such as smartphones and wireless Internet has recently increased, the date traffic has increased, and as the contents have become multimedia, the data traffic volume has increased exponentially. As the amount of data traffic increases, the minimum signal-to-noise ratio (SNR) required for decoding is lowered. To overcome this, the traffic rate must be reduced or the traffic volume must be reduced at a high compression rate.

In fact, WLANs such as IEEE802.11n negotiate the transmission rate and SNR relationship by varying the modulation scheme in the physical layer or the media access control (MAC) layer or by applying error control techniques or channel coding techniques when the channel conditions are poor. It keeps the communication state properly. Therefore, if the channel condition is bad, the encoding unit should reduce the bit rate with high compression and maintain a high FER (Frame Error Rate, SNR) .However, source coding and channel coding techniques and channel bandwidth alone, which are channel bandwidth and protocol dependent, are very poor. In the wireless channel environment, the above-described problem cannot be solved, and an error generated during image transmission from a CCTV camera can not be decoded, thereby making it impossible to record an image.

Therefore, the prior art as described above has a problem in that the image transmission is not possible due to the deterioration of the image quality due to the deterioration of the wireless channel state and the temporary disconnection of the wireless channel, it is an object of the present invention to solve this problem.

Accordingly, an object of the present invention is to provide an image transmitting / receiving system and method for adaptively encoding, transmitting, and decoding an image according to a channel state.

That is, the present invention divides the operation mode of the encoding unit and the decoding unit into the normal mode, the adaptive mode, and the disable mode according to the channel state, and defines and operates the respective functions so that recording is possible even in a bad channel state. In the disabled mode, the background image and the foreground image are processed separately so that the bit rate is adaptively adjusted according to the channel state to enhance the transmission reliability. The buffer image is set in the encoder subsystem to enable the foreground image even in the disabled mode where the wireless channel is disconnected. It is an object of the present invention to provide an image transmission / reception system and method that robustly operate in a wireless channel environment by preventing loss of the data.

The objects of the present invention are not limited to the above-mentioned objects, and other objects and advantages of the present invention which are not mentioned can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. It will also be readily apparent that the objects and advantages of the invention may be realized and attained by means of the instrumentalities and combinations particularly pointed out in the appended claims.

A system of the present invention for achieving the above object, the image transmission system, the camera control unit for controlling an external camera; A channel state determination unit for determining a channel state; A mode switching unit for performing mode switching between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the channel state determination unit; An encoding unit for encoding photographing image information from the camera according to a mode switching result in the mode switching unit; A storage unit for storing the captured image information encoded by the encoding unit; And a transmission unit for transmitting the captured image information encoded in the encoding unit or the captured image information stored in the storage unit through a wireless channel, and if the mode switching result in the mode switching unit is a normal mode, the encoding. Encoding the additional photographing image information, and the transmitting unit transmits the encoded captured image information through a wireless channel, and if the mode switching result of the mode switching unit is the adaptive mode, the encoding unit separates the foreground image and the background image. If the foreground image information is encoded and stored, and the transmission unit transmits the encoded foreground image information through a wireless channel, and the result of mode switching in the mode switching unit is disabled mode, the encoding unit separates the foreground image and the background image. Encode and save the foreground image information, then switch to normal or adaptive mode. Side transmits the transfer portion through a wireless channel the stored foreground video information.

In addition, the method of the present invention for achieving the above object, In the image transmission method, the step of transmitting the captured image information captured by the control of the camera control unit to the encoding unit; Determining a channel state by the channel state determiner; A mode switching step of a mode switching unit performing a mode switching operation between a normal mode, an adaptive mode, and a disabled mode based on the channel state determination result; In the normal mode, as a result of the mode switching, the encoding unit encoding the captured image information, and a transmitter transmitting the encoded captured image information through a wireless channel; When the mode is changed, in the adaptive mode, the encoding unit separates the foreground image and the background image by encoding and storing the foreground image information, and transmitting the encoded foreground image information through a wireless channel; And when the disabled mode is disabled, the encoder separates the foreground image and the background image from the foreground image and encodes the foreground image information. When the image is converted to the normal mode or the adaptive mode, the transmitter transmits the stored foreground image information to the wireless channel. Transmitting through.

On the other hand, another system of the present invention for achieving the above object, the image receiving system, a receiving unit for receiving the captured image information or foreground image information via a wireless channel; A background information updating / storing unit for storing and updating background information; A channel state determiner for determining a channel state according to the information from the receiver; A mode switching unit for performing mode switching between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the channel state determination unit; A decoding unit for decoding photographed image information or foreground image information from the receiving unit according to a mode switching result in the mode switching unit; A background information determiner configured to determine background information corresponding to the foreground image decoded by the decoder; An image synthesizer configured to synthesize the background image determined by the background information determiner and the foreground image decoded by the decoder; And an order adjusting unit for sorting and storing the images synthesized by the image synthesizing unit.

In addition, another method of the present invention for achieving the above object, In the image receiving method, the channel state determining unit for determining the channel state; A mode switching step of a mode switching unit performing mode switching between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the channel state determination unit; Decoding the captured image information by the decoding unit to restore and store the image and update the background information in the normal mode as a result of the mode switching; As a result of the mode switching, in the adaptive mode, the decoding unit decodes the foreground image information to restore the foreground image, the background information determiner determines the background information corresponding to the decoded foreground image, and the image synthesis unit determines the background information. Synthesizing the determined background image with the foreground image decoded by the decoding unit, and the order adjusting unit sorting and storing the image synthesized by the image synthesizing unit; And proceeding to the channel state determining step in the disabled mode as a result of the mode switching.

As described above, the present invention has the effect of encoding and storing only the foreground image and decoding and storing the received foreground image even in a situation where the channel condition is not good (adaptation mode), so that important surveillance image information can be recorded without losing the information. have.

In addition, the present invention is provided with a memory in the CCTV camera in case the wireless channel is disconnected (disabled mode), after storing only the foreground image by encoding, and then transmits again when the channel state is possible to transmit the image of the wireless channel environment The effect is that you can record all the shots regardless.

For example, in recent years, smart phones and various multimedia devices are being serviced through a wireless Wi-Fi network, and the urban wireless channel environment in which many users are concentrated has a severe fluctuation due to the surge in wireless data traffic. Wireless channels are frequently disconnected, and channel coding such as an error control technique cannot solve such problems. By defining the adaptive mode and the disabled mode that guarantee the minimum operation in such a wireless channel environment, the bandwidth of the wireless channel can be efficiently used and the surveillance video information captured by the camera can be transmitted without loss even in a poor radio wave environment. .

1 is a view showing a picture format of a GOP (Group of Picture) level used in MPEG,
2 is a diagram illustrating an embodiment of switching between operation modes (normal mode, adaptive mode, disabled mode) according to the present invention;
3 is a configuration diagram of an embodiment of an image transmission system that operates adaptively according to a channel state according to the present invention;
4 is a configuration diagram of an embodiment of an image receiving system that operates adaptively according to a channel state according to the present invention;
5 is a flowchart illustrating an embodiment of an image transmission method that operates adaptively according to a channel state according to the present invention;
6 is a flowchart illustrating an embodiment of an image receiving method that operates adaptively according to a channel state according to the present invention;
7 is a diagram illustrating a method of reconstructing a background image according to the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings, It can be easily carried out. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

And throughout the specification, when a part is referred to as being "connected" to another part, it includes not only "directly connected" but also "electrically connected" with another part in between. Also, when a component is referred to as " comprising "or" comprising ", it does not exclude other components unless specifically stated to the contrary .

First, the technical gist of the present invention will be described in order to help the understanding of the present invention.

In general, image information has a huge amount of information in nature, and thus processes such as compressing, storing, and transmitting the image obtained through the camera without processing the image as it is. Image compression is a lossy compression method that seeks to a high compression rate while eliminating redundancy and taking information loss at a level that satisfies human visual characteristics, and a lossless compression method that compresses at a low compression rate based on entropy. Among the video compression standards, lossy compression techniques are MPEG established by the MPEG group under ISO / IEC and H.26x by ITU-T. The purpose of the first video compression was reflected in MPEG, an image compression standard established for the storage and playback of images. On the other hand, from the ITU-T perspective, a standard for a technology that shows high compression ratio while simultaneously considering source coding and channel coding to cope with transmission delay and loss occurring during transmission while targeting multimedia service and streaming service through broadcasting network or heterogeneous communication network. Was needed. H.264 is a jointly enacted standard, with the two institutions converging in common. The same result, but H.264 is the name used by the ITU, and the MPEG group calls it MPEG-4 Part 10 / AVC (Advanced Video Coding).

In H.264, a frame structure consists of an I-frame (Intra Frame), a P-frame (Predictive Frame), and a B-Frame (Bi-Predictive Inter Frame).

In this case, the I-frame is an independent frame and the most important frame, and a method of compressing only information in its own frame is used, and a lot of bit rates are involved, and a reference frame is a resynchronization point in encoding or decoding. And the first frame must always start with an I-frame.

The P-frame is a frame generated by predicting the previous I-frame and the P-frame. The P-frame is predicted depending on the previous I-frame and the P-frame. It is sensitive, and if an error occurs in the previous I-frame and P-frame, a fault accumulates.

The B-frame is a frame capable of bi-prediction, which refers to both the I and P frames before and after, and has the highest compression rate among the three frames of I-frame, P-frame, and B-frame. It also shows a characteristic sensitive to transmission error.

And since the decoding unit always decodes with I-frame, if an error occurs in the I-frame, image restoration is impossible.

Therefore, in order to solve the problem that transmission error greatly affects image quality when a video image is transmitted through a wireless channel sensitive to noise or changes in the surrounding environment, the present invention provides a normal mode, Encoding and decoding are performed in three modes: adaptive mode and disabled mode. For example, in the normal mode, the image frame (including the background frame) is transmitted and stored at a predetermined period, and in the adaptive mode, the image is separated into the background image and the foreground image, and only the foreground information is transmitted and synthesized with the previously stored background information. In the disable mode, the camera encodes the foreground information and transmits it when the channel state is restored and synthesizes the previously stored background information.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a diagram illustrating a picture format of a GOP (Group of Picture) level used in MPEG, and shows a frame structure of H.264.

Here, the I-frame is an independent frame, and the I-frame is encoded by removing spatial redundancy between adjacent blocks in its own screen without referring to another screen.

For the P-frame and the B-frame, a method of compressing the inter-motion by predicting inter motion and removing temporal redundancy is used. In this case, the P-frame is encoded by predicting the closest I-frame and the P-frame closest to the previous one, and the B-frame is compressed by referring to both the I-frame and the P-frame before and after, so that the compression rate is low. The highest. In addition, since the remaining two frames (P-frame and B-frame) except for I-frames depend on previous frames, errors are propagated and have a great effect on quality, so that decoded images cannot be read frequently.

2 is a diagram illustrating an example of switching between operation modes according to an exemplary embodiment of the present invention, and shows a switching state between three operation modes, a normal mode, an adaptive mode, and a disabled mode.

Although not shown in the drawings, the image transmission system according to the present invention adaptively encodes, transmits, and decodes an image according to channel conditions. Such an image transmission system includes an image transmission system (see FIG. 3 to be described later) that performs a function of adaptively encoding and transmitting an image captured by a CCTV camera according to a channel state, and adaptively receives a received image according to a channel state. And a video receiving system (see FIG. 4 to be described later) that performs a function of decoding and storing and determining background information.

On the other hand, wireless networks generally have high SNR (Signal-to-S) at the expense of transmission speed by varying the modulation schemes or applying error control techniques or channel coding techniques in the physical layer or the MAC (Media Access Control) layer when the channel state is bad. Noise ratio). Therefore, the encoding unit should reduce the amount of information while reducing the image quality at a high compression rate. However, in the case of a wireless channel, if a transmission error occurs in the channel unacceptably, there is a limit in a method of changing a modulation scheme or applying an error control technique or a channel coding technique as described above. In this case, the camera operates in the adaptive mode or the disabled mode, and transmits only the foreground information without transmitting the background information, thereby transmitting the information captured by the camera without losing the information.

As shown in Figure 2, the operation mode according to the present invention, the normal mode operating in a good channel state, the channel state does not guarantee the normal operation, but to ensure the minimum image quality by reducing the transmission speed It is composed of three modes of the adaptive mode operating in the state of, and the disable mode operating in the state in which communication is not possible because the channel state is very poor, and the mode switching operation is adaptively performed according to the channel state.

For example, if the channel condition is good, it operates in H.264 normal mode.If the channel condition is not good, and it is operated enough to transmit the minimum picture quality at low speed of high compression, it is converted to adaptive mode. If it is poor, it goes into the disabled mode.

) 시각특성을 이용한 주관적 평가도 많이 사용하고 있으므로 끊김이 발생되지 않도록 동작되는 신뢰성이 중요시 되는 CCTV 카메라 영상에 있어서 CINR 값은 절대적 기준보다 가변됨이 타당할 것이다. 정상모드는 H.264 표준에서 정한 도 1의 과정으로 동작하는 것을 의미하며, 정상모드로 동작 중 채널상태가 좋지 않아져서 오류 제어 기법과 변조 기법으로 전송속도를 낮추어 저속으로 최소 화질을 전송할 수 있을 정도로 동작할 수 있는 경우(예를 들어 채널상태값이 임계치1보다 크고 임계치2보다 작은 경우) 적응모드로 전환된다. 도 2의 201, 202는 적응모드로 전환되는 경우를 보여주고 있다. 도 2의 205, 206은 전술한 채널평가 파라미터가 전송이 불가능한 경우(예를 들어 채널상태값이 임계치1 이하인 경우)로서, 불능모드로 전환되는 경우를 나타내고 있다. 그리고 도 2의 203, 204는 각각 적응모드, 불능모드에서 정상모드로 전환되는 예를 보여주고 있다.
In more detail, the switching between the operation modes depends on the channel state, and the determination of the channel state is determined as an error of data generated in the transmission / reception wireless channel between the video transmission system on the camera side and the access point (AP). For example, in a known wireless LAN or wireless data communication standard, a modulation coding scheme (MCS) level is changed according to channel state, and a corresponding AMC (Adaptive Modulation & Coding) technique is applied to the physical layer (BPSK to 64QAM). PHY) is changing the coding rate. At the same time, in conjunction with the MAC layer that provides the link adaptation function, the channel capacity is variable according to the channel state. As a result, it is used while varying the received CINR required for demodulation by increasing or decreasing the transmission rate according to the channel state. Accordingly, the required CINR value may be determined by interworking with a protocol provided by the link layer that is different from the minimum CINR value required for recovery according to a radio transmission technique. That is, the error control technique and the flow control technique provided in the link layer protocol formed between the video transmission system on the camera side and the AP may include ACK / NAK, retransmission request, and Forward Error Correction (FEC) technique, and are periodically transmitted from the AP. The channel state can be determined based on a case in which a beacon signal, a frame error rate (SNR), and a header file which are important information are damaged. That is, as an example, the threshold indicating the channel state between the camera and the AP is determined by the measured CINR, measured by TH1 = 0 and TH2 = 10, and the above-mentioned error is encoded and transmitted with the highest quality that can be provided by the camera video transmission system. Is generated and the channel state is good (e.g., when the channel state value is greater than or equal to the threshold 2), the operation is performed in the normal mode. For example, the wireless CCTV used in the present invention assumes a relatively limited environment used in a fixed environment, but is not limited to a fixed type, and the MCS level is adjusted upward after several successful data frames in normal mode. The quality of the image is evaluated a lot by the objective evaluation criteria, MSE and PSNR (for example,

) Since the subjective evaluation using visual characteristics is also used a lot, it is reasonable that the CINR value is more variable than the absolute standard for CCTV camera images where reliability is important to prevent disconnection. Normal mode means to operate according to the process of Fig. 1 defined in the H.264 standard, the channel state is not good during normal mode, the error control technique and the modulation technique can reduce the transmission speed to transmit the minimum image quality at low speed If the degree of operation is possible (for example, when the channel state value is greater than the threshold 1 and less than the threshold 2), the mode is switched to the adaptive mode. 201 and 202 of FIG. 2 show a case where an adaptive mode is switched. 205 and 206 of FIG. 2 show a case in which the aforementioned channel evaluation parameter is impossible to transmit (for example, when the channel state value is equal to or less than the threshold 1), and is switched to the disabled mode. 203 and 204 of FIG. 2 show an example of switching from the adaptive mode and the disable mode to the normal mode, respectively.

3 is an exemplary configuration diagram of an image transmission system 300 that operates adaptively according to a channel state according to the present invention, and is largely comprised of an encoding subsystem 310 and a transmission subsystem 320.

At this time, the encoding subsystem 310 is delivered separately from the channel state or the external AP generated through the protocol control provided by the camera control unit 311, the transmission subsystem 320 for controlling the external CCTV camera 330. A channel state determining unit 312 for determining a channel state according to channel control information such as a beacon signal, and a mode between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the channel state determining unit 312 Encoding for encoding (channel coding, source coding) the captured image information from the external CCTV camera 330 according to the mode switching result of the mode switching unit 313 and the mode switching unit 313 for performing the switching. And a buffering unit 315 for storing the captured image information encoded by the encoding unit 314.

The transmission subsystem 320 transmits the captured image information encoded by the encoding unit 314 or the captured image information stored in the buffering unit 315 to the image receiving system 400 through the WLAN. At this time, the transmission subsystem 320 connects the communication channel with the AP, exchanges captured image information and control information, and supports functions such as error control, speed control, flow control, and sequence matching generated in the communication process.

Here, in addition to the photographed image, the photographed image information may additionally include a photographing time (Time) and PTZ (Pan Tilt) so that the image receiving system 400 may easily set, determine, calculate, and store a background image. Zoom) is transmitted including the information.

A detailed operation method of the image transmission system 300 will be described later with reference to FIG. 5.

4 is an exemplary configuration diagram of an image receiving system 400 that operates adaptively according to a channel state according to the present invention, and includes a receiving subsystem 410 and a decoding subsystem 420.

First, the reception subsystem 410 connects a communication channel with the image transmission system 300 and exchanges photographed image information (or foreground image information) and control information, and performs error control, speed control, and flow control generated in a communication process. In order to support a function such as order matching, a channel control information may be generated and a beacon signal for checking a channel state with the image transmission system 300 may be provided.

And the decoding subsystem 420 is a channel information for determining the channel status according to the channel information or channel control information from the background information update / storage unit 421, the receiving subsystem 410 for storing and updating the background information A mode switching unit 423 for performing mode switching between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the determination unit 422 or the channel state determination unit 422; A decoding unit 424 for decoding the encoded captured image information or the foreground image information from the receiving subsystem 410 according to the mode switching result in the < RTI ID = 0.0 >), and < / RTI > a background corresponding to the decoded foreground image. A background information determiner 425 for determining information, an image synthesizer 426 for synthesizing the background image determined by the background information determiner 425 and the foreground image decoded by the decoder 424, and an image. The image synthesizer 426 includes an order adjusting unit 427 for arranging the order of the synthesized images and storing the sorted images in the memory 428.

Here, the background information updating / storing unit 421 stores initial background information and should update background information according to the situation of the image captured by the CCTV camera 330 to store the initial value of the background. The background information stores the image before the event or the motionless image as the background image. (On the other hand, the moving object is the foreground image.) The boundary of the background image to be stored is the entire range that can be taken. By rotating the camera, the area can be set to not overlap, or it can be scanned and stored to overlap. After the initial value of the background is stored, the background information is updated and stored according to the situation of the target image to maintain the latest situation.

As described above, the present invention constructs and operates a database for setting, updating, or storing background information by constructing a 3D vector map using PTZ as a variable during normal mode operation. That is, in the normal mode operation, the image receiving system 400 separates the surveillance area according to each PTZ value at regular intervals and forms a 3D vector map to provide background information together with camera information such as visual information and PTZ information. A database is set up, stored or updated.

The channel state determination unit 422 determines the channel state according to the channel state or channel control information from the receiving subsystem 410 as described above, or the channel state determination result determined by the image transmission system 300. May be received.

The background information determiner 425 determines a background image by searching a similar background image in a database based on the received time information and the PTZ information, and reconfigures the background information to correspond to each PTZ value.

If the encoded captured image information stored in the disabled mode in which the wireless channel is disconnected is received in the adaptive mode as the wireless channel state improves, the time sequence recorded in the frame is rearranged and the image order is rearranged in chronological order. Database).

A detailed operation method of the image receiving system 400 will be described later in detail with reference to FIG. 6.

5 is a flowchart illustrating an embodiment of an image transmission method that operates adaptively according to a channel state according to the present invention.

First, when the connection between the AP and the video transmission system is established (501), the CCTV camera 330 controls the PTZ under the control of the camera control unit 311 and starts shooting, shooting time (Time), PTZ (Pan Tilt Zoom) ) And the photographed image information including the photographed image is acquired and transmitted to the encoder 314 (502). Here, when the CCTV camera 330 does not have a PTZ function, the PTZ information transfer function may not be performed.

In addition, the channel state determination unit 312 transmits control information including a beacon signal or channel state received from the AP, a link layer protocol between the AP and the video transmission system 300, a state of the buffering unit 315, a modem, and the like. / The channel state is determined (determined) by using information such as the reception level provided by the receiving subsystems 320 and 410 (503), and the mode switching unit 503 determines that the normal mode ( In operation 505, the mode switching operation between the adaptive mode 508 and the disabled mode 512 is performed (504).

As a result of the mode switching 505, in the normal mode (505), the operation mode is defined in H.264, and the captured image information (shooting time, PTZ information, captured image, etc.) received from the CCTV camera 330 is obtained. The encoding unit 314 performs an encoding process 506 and transmits the encoded captured image information from the transmission subsystem 320 to the image receiving system 400.

In this case, even in the normal mode, it is not necessary to operate in the normal mode, and may operate in the adaptive mode for efficient use of the channel band. That is, if the channel state is normal and no event occurs, the video transmission system 300 may continue to operate in the adaptive mode.

As a result of the mode switching 505, in the adaptive mode 508, the encoding unit 314 separates the foreground image and the background image to include only foreground image information (including PTZ-T information (PTZ information, photographing time)). Encode (509). At this time, the PTZ-T information (shooting time, PTZ information) is added to the foreground image and stored in the buffering unit 315 so as to be used as information for determining background information in the image receiving system 400 (510) and the transmission subsystem In operation 320, a process 507 of transmitting to the image receiving system 400 is performed. If a channel error occurs and a retransmission is requested from the AP or a NAK signal is received, the information stored in the buffering unit 315 is continuously stored. If the transmission is successful, the information stored in the buffering unit 315 is deleted.

As a result of the mode switching 505, in the disabled mode (512), the encoding unit 314 separates the foreground image and the background image and encodes only the foreground image information (including PTZ-T information) (513). After storing at 315, the mobile station waits until the wireless channel becomes normal or becomes an adaptive mode, and transmits the stored information to the buffering unit 315 when it is determined to be the normal mode or the adaptive mode. At this time, visual information and PTZ information are saved and transmitted together so that the screen can be rearranged according to time. Further, in the operation of the disabled mode, priority may be given to the foreground of event occurrence such as the appearance of a moving object.

6 is a flowchart illustrating an image receiving method that operates adaptively according to a channel state according to the present invention.

The image receiving system 400 generates and stores the background information initial value during normal operation (normal mode), and updates the background information at regular intervals or according to the situation of the image to be photographed. Background information can be easily constructed when the PTZ function is not used, but for the CCTV camera 330 having the PTZ function, a plurality of background information should be determined and stored according to the PTZ value. It is preferable to set the background information by changing the PTZ as a variable so that the surveillance area is not overlapped.However, the background information can be set separately by overlapping according to the capacity of the database and the purpose of the surveillance camera, and synthesized with similar values according to the received PTZ information. You may. Additionally, moving objects, such as cars entering or leaving a parking lot, are classified as foreground information, but events in the surveillance area for a period of time, such as a car that has been parked for a long time in a parking lot or the state of the parking lot after the car has left the parking lot. If is not detected, it is processed as a background image.

First, when a connection between the video transmission system 300 and the video reception system 400 is established through the AP (601), the channel state determination unit 422 continuously controls the channel state or channel control from the reception subsystem 410. The channel state is determined by monitoring the information (602), the receiving subsystem 410 generates the channel control information and transmits it to the video transmission system 300 (603), and the mode switching unit 423 determines the channel state. In operation 604, the mode switching between the normal mode, the adaptive mode, and the disable mode is performed according to the channel state determination result.

As a result of the mode switching 604, in the normal mode in which the channel state is normal (605), it operates in the mode specified in H.264, and the encoded image captured by the decoding unit 424 from the receiving subsystem 410 is received. The information is decoded in the H.264 mode (606), the image is restored, stored in the memory (database) (607), and the background information is updated (608).

As a result of the mode change 604, in the adaptive mode 609, the decoder 424 decodes the encoded foreground image information received from the receiving subsystem 410 to restore the foreground image (610). In operation 611, the background information determiner 425 determines background information corresponding to the decoded foreground image. That is, the background information determiner 425 searches for a similar background image in the database based on the received PTZ information to determine the background image. The image synthesizer 426 synthesizes the background image determined by the background information determiner 425 and the foreground image decoded by the decoder 424 (613). The order adjusting unit 427 sorts the order of the images synthesized by the image synthesizing unit 426 (613) and stores the order of the images in the memory 428 (614). That is, the order adjusting unit 427 determines the order of the images synthesized by the image synthesizing unit 426 based on the time information and stores the order of the images in the storage unit 428. This is to reorder and store the frames in the order adjusting unit 427 in case the frame order arrives due to the instability of the wireless channel in the disabled mode 615.

As a result of the mode change 604, in the disabled mode 615, the channel state determination process 602 is immediately performed when the wireless channel is disconnected and the reception operation is impossible. At this time, the receiving subsystem 410 continuously proceeds to re-establish the radio channel, but the detailed description thereof will be omitted since it is not a scope of the present invention.

7 is a diagram illustrating a method of reconstructing a background image according to the present invention.

FIG. 7 illustrates an example of reconstructing a background image when the TZ value of the PTZ parameters is fixed and only the P value is changed. For example, a pan (PAN) camera 705 does not overlap the entire surveillance area 710 that can be monitored. The background 1 701, the background 2 702, the background 3 703, and the background 4 704 are stored and operated. In this case, when the pan value of the pan camera 705 is received as a value that shares the background 2 702 and the background 3 703 together, the background information determiner 425 of the image receiving system 400 receives the background. The background image is reconstructed with a part of part 2 and part of background 3, and the image synthesizer 426 synthesizes the foreground image and the reconstructed background image decoded by the decoder 424 to reconstruct the image.

As such, the background image may be composed of one or more screens. In this case, the background image may be set and stored so as not to overlap, and the background image may be set and stored so as to overlap with each other, and the background image may be calculated and synthesized using time information or PTZ information.

On the other hand, the image transmission and reception method according to the present invention as described above is implemented in the form of program instructions that can be executed by various computer means may be recorded on a computer readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions recorded on the medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. The medium may be a transmission medium such as an optical or metal line, a wave guide, or the like, including a carrier wave for transmitting a signal designating a program command, a data structure, or the like. Examples of program instructions include machine language code such as those generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, Various permutations, modifications and variations are possible without departing from the spirit of the invention.

Therefore, the scope of the present invention should not be construed as being limited to the embodiments described, but should be determined by the scope of the appended claims, as well as the appended claims.

300: video transmission system 310: encoding subsystem
311: camera control unit 312: channel state determination unit
313: mode switching unit 314: encoding unit
315: buffering section 320: transmission subsystem
400: video receiving system 410: receiving subsystem
420: decoding subsystem 421: background information update / storage unit
422: channel state determination unit 423: mode switching unit
424: decoding unit 425: background information determining unit
426: image synthesis unit 427: sequence adjustment unit
428: memory

Claims (17)

In a video transmission system,
A camera controller for controlling an external camera;
A channel state determination unit for determining a channel state;
A mode switching unit for performing mode switching between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the channel state determination unit;
An encoding unit for encoding photographing image information from the camera according to a mode switching result in the mode switching unit;
A storage unit for storing the captured image information encoded by the encoding unit; And
A transmitter for transmitting the captured image information encoded by the encoding unit or the captured image information stored in the storage unit through a wireless channel,
When the mode switching result of the mode switching unit is the normal mode, the encoding unit encodes the captured image information, and the transmission unit transmits the encoded captured image information through a wireless channel,
If the mode switching result of the mode switching unit is the adaptive mode, the encoding unit separates the foreground image and the background image, encodes and stores foreground image information, and the transmitter transmits the encoded foreground image information through a wireless channel. ,
If the mode switching result of the mode switching unit is the disabled mode, the encoding unit separates the foreground image and the background image, encodes and stores the foreground image information, and when the mode is switched to the normal mode or the adaptive mode, the transmission unit stores the stored foreground image. An image transmission system for transmitting information through a wireless channel.
The method of claim 1,
The mode switching unit,
If the channel state value is " more than threshold 2 " indicating a good state, the channel state is changed to normal mode.
If the channel state value is greater than the threshold 1 and less than the threshold 2, which indicates that the channel quality is poor and the minimum image quality can be transmitted at low speed, the mode is switched to adaptive mode.
And switching to the disable mode when the "channel state value is below threshold 1" indicating a state in which the channel state is so poor that communication is impossible.
delete The method of claim 1,
The photographed video information,
An image transmission system including a captured image, a captured time (Time), and PTZ (Pan Tilt Zoom) information.
The method of claim 1,
The foreground image information,
An image transmission system including a foreground image, a shooting time (Time), and PTZ (Pan Tilt Zoom) information.
In the video transmission method,
Transmitting photographed image information captured by the camera controller to the encoder;
Determining a channel state by the channel state determiner;
A mode switching step of a mode switching unit performing a mode switching operation between a normal mode, an adaptive mode, and a disabled mode based on the channel state determination result;
In the normal mode, as a result of the mode switching, the encoding unit encoding the captured image information, and a transmitter transmitting the encoded captured image information through a wireless channel;
When the mode is changed, in the adaptive mode, the encoding unit separates the foreground image and the background image by encoding and storing the foreground image information, and transmitting the encoded foreground image information through a wireless channel; And
As a result of the mode switching, the encoding unit separates the foreground image from the background image and stores the foreground image information after encoding the foreground image information. If the image is converted to the normal mode or the adaptive mode, the transmitter transmits the stored foreground image information through a wireless channel. Sending steps
Image transmission method comprising a.
The method according to claim 6,
The mode switching step,
If the channel state value is " more than threshold 2 " indicating a good state, the channel state is changed to normal mode.
If the channel state value is greater than the threshold 1 and less than the threshold 2, which indicates that the channel quality is poor and the minimum image quality can be transmitted at low speed, the mode is switched to adaptive mode.
And switching to the disable mode when the channel state value is not more than the threshold 1 indicating the state in which the channel state is so poor that communication is impossible.
The method according to claim 6 or 7,
The photographed video information,
Including a shooting image, shooting time (Time), and PTZ (Pan Tilt Zoom) information,
The foreground image information,
An image transmission method comprising a foreground image, a shooting time (Time), and PTZ (Pan Tilt Zoom) information.
In a video receiving system,
A receiver for receiving captured image information or foreground image information through a wireless channel;
A background information updating / storing unit for storing and updating background information;
A channel state determiner for determining a channel state according to the information from the receiver;
A mode switching unit for performing mode switching between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the channel state determination unit;
A decoding unit for decoding photographed image information or foreground image information from the receiving unit according to a mode switching result in the mode switching unit;
A background information determiner configured to determine background information corresponding to the foreground image decoded by the decoder;
An image synthesizer configured to synthesize the background image determined by the background information determiner and the foreground image decoded by the decoder; And
An order adjusting unit for sorting and storing the images synthesized by the image synthesizing unit
Image receiving system comprising a.
The method of claim 9,
The mode switching unit,
If the channel state value is " more than threshold 2 " indicating a good state, the channel state is changed to normal mode.
If the channel state value is greater than the threshold 1 and less than the threshold 2, which indicates that the channel quality is poor and the minimum image quality can be transmitted at low speed, the mode is switched to adaptive mode.
And switching to the disabled mode when the "channel state value is below threshold 1" indicating a state in which the channel state is so poor that communication is impossible.
11. The method according to claim 9 or 10,
The photographed video information,
Including a shooting image, shooting time (Time), and PTZ (Pan Tilt Zoom) information,
The foreground image information,
An image receiving system including a foreground image, a shooting time (Time), and PTZ (Pan Tilt Zoom) information.
The method of claim 11,
If the mode switching result in the mode switching unit is the normal mode, the decoding unit decodes the captured image information to restore and store the image,
And if the mode switching result of the mode switching unit is the adaptive mode, the decoding unit to decode the foreground image information to restore the foreground image.
The method of claim 11,
The background information update / storage unit,
An image receiving system for configuring, storing, or updating background information along with visual information and PTZ information by configuring a surveillance area as a 3D vector map according to a PTZ value during a normal mode operation.
The method of claim 11,
The background information determination unit,
The image receiving system of retrieving a background image by searching a background image in a database based on visual information and PTZ information, and reconfiguring the background information to correspond to a PTZ value.
In the video receiving method,
A channel state determination step of determining, by the channel state determination unit, the channel state;
A mode switching step of a mode switching unit performing mode switching between a normal mode, an adaptive mode, and a disabled mode according to a channel state determination result of the channel state determination unit;
Decoding the captured image information by the decoding unit to restore and store the image and update the background information in the normal mode as a result of the mode switching;
As a result of the mode switching, in the adaptive mode, the decoding unit decodes the foreground image information to restore the foreground image, the background information determiner determines the background information corresponding to the decoded foreground image, and the image synthesis unit determines the background information. Synthesizing the determined background image with the foreground image decoded by the decoding unit, and the order adjusting unit sorting and storing the image synthesized by the image synthesizing unit; And
Proceeding to the channel state determining step if it is disabled mode as a result of the mode switching;
And transmitting the image data.
The method of claim 15,
The mode switching step,
If the channel state value is " more than threshold 2 " indicating a good state, the channel state is changed to normal mode.
If the channel state value is greater than the threshold 1 and less than the threshold 2, which indicates that the channel quality is poor and the minimum image quality can be transmitted at low speed, the mode is switched to adaptive mode.
And switching to an incapable mode when the channel state value is less than or equal to the threshold value 1, indicating that the channel state is so poor that communication is impossible.
17. The method according to claim 15 or 16,
The photographed video information,
Including a shooting image, shooting time (Time), and PTZ (Pan Tilt Zoom) information,
The foreground image information,
A method for receiving an image, comprising a foreground image, a time taken for shooting, and Pan Tilt Zoom (PTZ) information.

Images