TWI390989B - Mobile communication apparatus and image adjusting method thereof - Google Patents

Description

Mobile communication device and image adjustment method thereof

The present invention relates to image processing, and in particular, the present invention relates to a mobile communication device having a video image post-processing function and an image adjustment method thereof.

In recent years, due to the advancement of audio-visual communication technology and the trend of digitalization, countries around the world have begun to develop high-quality digital TVs, which has also led to the vigorous development of digital TV-related industries.

Compared with the traditional analog signal, many digital signals cannot be removed during the transmission process. The digital signal does not have this problem, so high-quality signals can be maintained. In addition, digital signals can be compressed to increase the amount of information transmitted. For example, an analog TV channel can only broadcast one program, while a digital TV channel can broadcast three, four or more programs. These are the benefits of digitizing TV signals.

In order to realize the future trend of "mobile digital TV", the European Digital Video Broadcasting Technology Development Organization has adopted the Digital Video Broadcasting-Terrestrial (DVB-T) transmission technology standard originally applied to "fixed digital television". Based on the basic technical requirements of Handheld Digital Video Broadcasting-Handheld (DVB-H), it entered the verification and standardization process in 2004, and conducted experiments on DVB-H transmission technology standards in several regions in Europe. Sexual trials and research projects.

Compared with the original DVB-T transmission technology, DVB-H transmission technology has the advantages of low power consumption, high mobility, easy reception, common platform and network switching service without interruption in order to meet the special needs of handheld devices. . Outlook By integrating mobile communication networks with DVB-H broadcast networks, users can provide more diverse content and interactive services.

As for the high compression video compression technology emphasized by digital TV, the MPEG-4 video compression standard has been dominated in the past. However, as video compression technology continues to evolve, a new H.264 video compression standard has been proposed.

The H.264 video compression standard provides extremely good video quality with minimal video content. In other words, users only need to open a H.264 video file with a relatively small amount of data to see a very clear video picture.

For example, the H.264 video compression standard requires only one-third to one-half of the data transfer rate to provide the same picture quality as the MPEG-2 video compression standard. If the same data transmission rate is used, the resolution of the H.264 video compression standard can reach four times that of the MPEG-4 video compression standard.

In view of the above advantages, the video compression standard adopted by the DVB-H transmission technology is the H.264 video compression standard. With H.264 video compression technology, DVB-H transmission technology can transmit 30 images per second to handheld devices, so that users who watch digital TVs through handheld devices can enjoy the same high quality and fast speed as home viewing. Digital TV shows.

However, when the signal transmitted by DVB-H is unstable, it is likely to cause an error in the H.264 video decoder, resulting in an erroneous image block. At this point, the image post-processing mechanism of the decoded video becomes very important.

The image post-processing methods currently used can be roughly classified into the following two types.

The first method is to directly use the reference image during decoding as the basis for image repair; this method has low computational complexity but causes image in dynamic video. The phenomenon of discontinuity is therefore not practical.

Another approach is to take high-level image processing techniques such as edge detection, contour detection, texture suppression, or proximity edge analysis. The disadvantage of this method is that its computational complexity is too high, and it is not suitable for handheld devices with limited computing power and resources.

Therefore, the main scope of the present invention is to provide a mobile communication device and an image adjustment method thereof to solve the above problems.

The mobile communication device and the image adjustment method thereof provided by the invention can perform appropriate real-time image on the error region in the video stream without increasing the computing burden of the mobile communication device and taking into consideration the image quality of the video stream. make up.

A mobile communication device in accordance with an embodiment of the present invention. The mobile communication device has the function of decoding and playing a video stream. The mobile communication device comprises a determination module, a capture module, a calculation module and a correction module.

Since a video stream usually contains a relatively large number of continuously played images after decoding, it is very likely that some of the erroneous image areas are generated in some of the images because the transmission signal or the decoder is unstable. Therefore, the function of the determining module is to determine whether one of the current video images in the decoded video stream contains an error region.

The capture module is electrically connected to the determination module. If the judgment result of the judgment module is yes, the capture module will retrieve information about the neighboring area of the error area. In fact, the neighborhood information may be a dynamic vector associated with a neighboring region located near the error region.

The computing module is electrically connected to the capturing module, and is configured to calculate speculative information corresponding to one of the error regions according to the neighboring area information. In fact, the speculative information can be one of the speculative dynamic vectors associated with the error region. That is, the computing module can estimate the dynamic vector corresponding to the error region according to the motion vector of the region near the error region.

The correction module is electrically connected to the computing module and configured to correct the error region according to one of the reference video and the speculative information in the decoded video stream. In fact, the reference image may be a previous image in the video stream relative to one of the current images.

In practical applications, the mobile communication device can further include a decoding module. The decoding module is electrically connected to the determining module, and the function is to decode the original video stream to generate the decoded video stream.

Another embodiment of the present invention is an image adjustment method. In fact, the image adjustment method can be applied to a mobile communication device having a function of decoding and playing video streams. First, the method decodes an original video stream to generate a decoded video stream. Then, the method determines whether one of the current video images in the decoded video stream contains an error region. If the result of the determination is yes, the method extracts information about the neighboring area of the error area. In fact, the neighborhood information may be a dynamic vector associated with a neighboring region located near the error region.

Then, the method calculates speculative information corresponding to one of the error regions according to the neighboring area information. In fact, the speculative information can be one of the speculative dynamic vectors associated with the error region. Thereafter, the method corrects the error region according to one of the reference video and the speculative information in the decoded video stream. In fact, the reference image may be a previous image in the video stream relative to one of the current images.

Compared with the prior art, the mobile communication device and the image adjustment method thereof according to the present invention provide an image post-processing mechanism for the decoded video stream, which not only can obtain good image processing effects, but also avoids the previous reference when decoding directly. The image as a basis for image repair is caused by image discontinuity, and because the computational complexity required is not high, the image adjustment method according to the present invention is very suitable for a mobile communication device with relatively limited computing resources.

The advantages and spirit of the present invention will be further understood from the following detailed description of the invention.

The invention provides a mobile communication device and an image adjustment method thereof for performing instantaneous image compensation on an error region in a video stream received by the mobile communication device.

A first embodiment of the present invention is a mobile communication device. In this embodiment, the mobile communication device has the function of decoding and playing a video stream. Please refer to FIG. 1 , which is a functional block diagram of the mobile communication device. As shown in FIG. 1 , the mobile communication device 1 includes a determination module 12 , a capture module 14 , a calculation module 16 , and a correction module 18 . Next, the relationship between the modules included in the mobile communication device 1 and the functions thereof will be described in detail.

In general, in the process of video transmission or video decoding, it is very likely that due to the instability of the signal transmission or the decoder, an erroneous image area appears in some decoded images of the video stream. This causes discontinuous video streaming, which seriously affects the user's feeling of enjoying the video. Therefore, the function of the determining module 12 in the mobile communication device 1 is to determine whether a current image in a decoded video stream contains an error region. For example, in the H.264 video compression standard, by checking the header of a block It is known whether the block is an error block, thereby determining whether the image contains an error area.

In practical applications, the decoded video stream may be a Digital Video Broadcasting-Handheld (DVB-H) video stream. The DVB-H video stream can meet the special requirements of low power consumption, high mobility, easy reception and uninterrupted network switching services required by mobile communication devices through DVB-H transmission technology.

In this embodiment, the capture module 14 is electrically connected to the determination module 12. If the judgment result of the module 12 is YES, that is, the current image in the decoded video stream contains the error area, the capture module 14 will retrieve information about the neighboring area of the error area. In practical applications, the neighbor information may be a dynamic vector associated with a neighbor located near the error region.

The computing module 16 is electrically connected to the capturing module 14 and is configured to calculate speculative information corresponding to one of the error regions based on the neighboring area information. In fact, the speculative information can be one of the speculative dynamic vectors associated with the error region.

In this embodiment, the correction module 18 is electrically connected to the computing module 16 and is configured to correct the error region according to one of the reference video and the speculative information in the decoded video stream. In fact, the reference image may be a previous image in the video stream relative to one of the current images.

Next, an example will be used to describe how the mobile communication device 1 estimates the estimated motion vector of the error region in the video stream by using the neighbor region motion vector near the error region of the current image, and according to the speculative motion vector. And a previous image corrects the error area.

Since the present invention utilizes the signal characteristics of DVB-H and possible error conditions, and restores the current image to a high quality image through error correction. For example, the present invention must utilize the characteristics of the space to obtain information that facilitates error correction by the correlation between the error region and its neighboring region. As shown in FIG. 2, this example only considers the adjacent areas in the four directions of the left side, the upper side, the upper left side, and the upper right side of the error area 26. Since the images in the video have dynamic characteristics, the location of a particular area is likely to be different in different image frames. As shown in FIG. 2, the relative position of the error region 26 in the current image 22 and the corresponding region 27 of the reference image 24 relative to the background pattern 28 is different. Therefore, the present invention employs a method of dynamic vectors.

In addition, in order to improve the performance of H.264 encoding, these adjacent regions can be further cut into different sizes. As shown in FIG. 2, the area above the error area 26 in the current image 22 is cut into upper and lower blocks, wherein the block of the lower diagonal line can be regarded as the adjacent area of the error area 26; the adjacent area of the left side of the error area 26 It is cut into four blocks, wherein the blocks of the upper right and lower right diagonal lines can be regarded as the adjacent areas of the error area 26; as for the two adjacent areas in the upper left and upper right areas of the error area 26, the original size is maintained. The same can be regarded as the adjacent area of the error area 26.

As shown in FIG. 2, in the current image 22, each of the divided blocks and the undivided adjacent regions each correspond to one dynamic vector. In the H.264 system, seven different sizes and shapes are available (16×16, 16×8, 8×16, 8×8, 8×4, 4×8, 4×4). Therefore, a neighboring area can be divided into up to sixteen blocks, so the neighboring area can contain up to sixteen dynamic vectors.

In addition, since each adjacent area has different importance to the wrong area, a new set of dynamic vectors can be obtained by weighting. In this example, assuming that the distance between the adjacent area and the wrong area is inversely proportional to the correlation between the two, the weight can be weighted according to the distance as a weight when calculating the new dynamic vector.

According to the above method, the motion vector mv _{2 of the} region above the error region 26 can be obtained by the following equation: mv ₂ = (mv ₂₁ ^* 1 + mv ₂₂ ^* 2) / 3.

Similarly, the motion vector mv _{4 of the} left region of the error region 26 can be obtained by the following equation: mv ₄ = (mv ₄₁ ^* 1 + mv ₄₂ ^* 1 + mv ₄₃ ^* 2+mv ₄₄ ^* 2) / 6.

After the motion vectors mv ₁ , mv ₂ , mv ₃ and mv ₄ of the adjacent regions in the four directions of the error region 26 are obtained, the median of the four motion vectors is taken, and the reference image 24 is obtained. The estimated dynamic vector mv _{0 of the} corresponding region 27 can be expressed by the following equation: mv ₀ =median(mv ₁ ,mv ₂ ,mv ₃ ,mv ₄ ).

Since the motion vector can represent the moving characteristics of an image region, and the relative position of the image region to the reference image (previous image) in the time domain is known, the estimated motion vector obtained after the dynamic vector estimation can represent the error region. The correlation with the previous image in the time domain, and the error region is corrected by the speculative dynamic vector.

Since the decoder usually decodes the previous or previous images as the basis for reference decoding, the speculative motion vector can be used to locate the corresponding region corresponding to the error region in the current image in the previous image, and The two-dimensional pixel data of the corresponding area is moved to the position of the error area of the current image, thereby improving the quality of the image. Compared with the conventional method, directly moving the two-dimensional pixel data of the same position of the previous image to the position of the wrong area of the current image causes the image to be discontinuous, ignoring the dynamic characteristics of the image, and the image processing according to the present invention The effect can greatly improve this phenomenon and get high quality video images.

As shown in FIG. 3, the mobile communication device 1 can further include a receiving module 10 and a decoding module 20. In fact, the receiving module 10 can include an antenna. In this embodiment, one of the video streams received by the receiving module 10 of the mobile communication device 1 may contain a relatively large number of continuously played images. For example, if the video stream received by the mobile communication device 1 is transmitted through the DVB-H transmission technology, since the DVB-H transmission technology adopts the H.264 video compression standard, it can transmit up to 30 images per second to the mobile communication. Device 1.

In this embodiment, the decoding module 20 is electrically connected to the determining module 12, and the function is to decode the original video stream to generate the decoded video stream. In fact, the decoding module 20 can be an H.264 video stream decoder. Since the H.264 video stream decoder compresses a video stream using the H.264 video compression standard, it can cooperate with the DVB-H transmission technology that also adopts the H.264 video compression standard.

Figure 4 shows an example of the H.264 video stream decoding procedure. In this example, the video data extracted by the H.264 demultiplexer is a one-bit stream, assuming that the binary stream is 000010001110010111101101. The bit stream contains the parameter characteristics and compressed data of each image, and the reconstructed image is obtained through the following H.264 decoding process.

As shown in FIG. 4, first, the H.264 decoder performs a Context Adaptive Variable Length Decoding (CAVLD) step to convert the binary stream into a frequency domain data.

Then, by an inverse quantization step, the H.264 decoder can appropriately amplify the coefficients in the frequency domain data to conform to the encoding technique of the encoder. Thereafter, the frequency domain data can be inversely converted into coefficients of the spatial domain by an integer discrete step of inverse cosine transform (IDCT). The above steps of inverse quantization and integer discrete cosine inverse conversion are all steps of an inverse transform.

Finally, the H.264 decoder cooperates with the spatial domain image prediction technology or the time domain dynamic compensation technology to obtain complete spatial pixel information, that is, the video image seen by the user on the screen of the mobile communication device.

With H.264 video compression technology, the mobile communication device 1 can provide users with excellent video quality by using a relatively small video stream. Therefore, the user only needs to open a small H.264 video file to pass through. The mobile communication device 1 enjoys very clear video content.

A second embodiment of the present invention is an image adjustment method. In this embodiment, the image adjustment method can be applied to a mobile communication device, and the mobile communication device can have the function of decoding and playing a video stream.

Please refer to FIG. 5 , which is a flow chart showing the image adjustment method. As shown in FIG. 5, first, the method performs step S10 to decode an original video stream to generate a decoded video stream. The original video stream and the decoded video stream each include a plurality of images.

In fact, the decoding process performed by the method in step S10 can be achieved by one of the H.264 video stream decoders in the mobile communication device. The decoded video stream can be a handheld digital video broadcast video stream.

Due to the instability of the signal transmission or decoder, it may result in an erroneous image area in some decoded images. Therefore, the method will perform step S11 to determine whether one of the current video images in the decoded video stream contains an error region. For example, in the H.264 video compression standard, whether a block is an error block can be determined by checking a header of a block to determine whether the image contains an error area.

If the result of the determination in step S11 is YES, the method proceeds to step S12 to extract information about the neighboring area of the error area. In effect, the neighboring area information is a dynamic vector associated with one of the adjacent areas of the error area.

Thereafter, the method performs step S13, and calculates speculative information corresponding to one of the error regions according to the neighboring area information. In fact, the speculative information is one of the speculative dynamic vectors associated with the error region.

Finally, the method performs S14, and the error region is corrected according to one of the reference video and the speculative information in the decoded video stream. In effect, the reference image is a previous image of the decoded video stream relative to one of the current images.

Compared with the prior art, the mobile communication device and the image adjustment method thereof according to the present invention provide an image post-processing mechanism for the decoded video stream, which not only can obtain good image processing effects, but also avoids the previous reference when decoding directly. The image as a basis for image repair is caused by image discontinuity, and because the computational complexity required is not high, the image adjustment method according to the present invention is very suitable for a mobile communication device with relatively limited computing resources.

The features and spirit of the present invention will be more apparent from the detailed description of the preferred embodiments. On the contrary, the intention is to cover various modifications and equivalents within the scope of the invention as claimed.

S10~S15‧‧‧ Process steps

1‧‧‧Mobile communication device

10‧‧‧ receiving module

12‧‧‧Judgement module

14‧‧‧Capture module

16‧‧‧Computation Module

18‧‧‧Correction module

20‧‧‧Decoding module

22‧‧‧ Current image

24‧‧‧Reference image

26‧‧‧Wrong area

27‧‧‧Corresponding area

28‧‧‧Background pattern

Mv ₀ ~mv ₄₄ ‧‧‧dynamic vector

1 is a functional block diagram of a mobile communication device in accordance with a first embodiment of the present invention.

Figure 2 shows an example of calculating a speculative dynamic vector.

FIG. 3 is a detailed functional block diagram of the mobile communication device shown in FIG.

Figure 4 shows an example of the H.264 video stream decoding procedure.

FIG. 5 is a flow chart showing an image adjustment method according to a second embodiment of the present invention.

1‧‧‧Mobile communication device

12‧‧‧Judgement module

14‧‧‧Capture module

16‧‧‧Computation Module

18‧‧‧Correction module

Claims (15)

A mobile communication device includes: a determining module, configured to determine whether a current image of a decoded video stream includes an error region, wherein the decoded video stream includes a plurality of images; and a capture module, Electrically connected to the judging module, the capturing module captures information about a neighboring area of the error area; a computing module is electrically connected to the capturing module for calculating a correspondence according to information of the neighboring area The information is estimated in one of the error areas; and a correction module is electrically connected to the calculation module for correcting the error area according to the reference image and the speculation information in the decoded video stream. The mobile communication device according to claim 1, wherein the adjacent area includes an upper left, upper, upper right, and left area of the error area. The mobile communication device of claim 1, wherein the information of the neighboring area is a dynamic vector of the neighboring area. The mobile communication device of claim 1, wherein the speculative information is a speculative dynamic vector associated with the error region. The mobile communication device of claim 1, wherein the reference image is a previous image of the video stream relative to one of the current images. The mobile communication device of claim 1, wherein the decoded video stream is a Digital Video Broadcasting-Handheld (DVB-H) video stream. The mobile communication device of claim 1, further comprising: a decoding module electrically connected to the determining module, configured to decode an original video stream to generate the decoded video stream. The mobile communication device of claim 7, wherein the decoding module is an H.264 video stream decoder. An image adjustment method includes the following steps: (a) determining whether a current image in a decoded video stream includes an error region, wherein the decoded video stream includes a plurality of images; (b) if the determination in step (a) is yes, (c) calculating, based on information of the neighboring region, information corresponding to the error region; and (d) determining, according to the information of the decoded video stream, the reference image and the speculative information Fix the error area. The image adjustment method according to claim 9, wherein the adjacent area includes an upper left, upper, upper right, and left areas of the error area. The image adjustment method of claim 9, wherein the information of the neighboring area is a dynamic vector of the neighboring area. The image adjustment method according to claim 9, wherein the speculation information is one of a speculative motion vector associated with the error region. The image adjustment method of claim 9, wherein the reference image is a previous image of the decoded video stream relative to one of the current images. The image adjustment method of claim 9, wherein the decoded video stream is a handheld digital video broadcast video stream. The image adjustment method of claim 9, further comprising the following steps: (e) decoding the original video stream to generate the decoded video stream.