JP2005051818A - Image processing apparatus and method, and computer readable recording medium recorded with image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus for appropriately processing the reproduction/stop of an image using copyright protection of the image. <P>SOLUTION: Image data encoded using an encoding mode having at least an intra-encoding mode and an inter-encoding mode and security data for protecting said image data are inputted, it is judged on the basis of said security data whether the reproduction of the the encoded image is allowed, the encoding mode of the encoded image data is determined, and the encoded image data are decoded in accordance with the judgement result and the determination result. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and method, and a computer-readable recording medium on which an image processing program is recorded, and more particularly to image processing for protecting the copyright of an image.

  In recent years, standardization of moving picture coding technology has been rapidly progressing. MPEG-1, 2 and H.264 261, H.M. Standards for encoding such as H.263 (these encoding methods are internationally standardized by ISO (International Organization for Standardization)) and ITU (International Telecommunication Union: International Telecommunication Union) have been established. -MPEG-4 international standardization work is in progress as a general-purpose next-generation multimedia coding standard that can be used in many fields such as broadcasting and communication. With the above encoding technology, digitization of images has advanced, and moving image data storage and communication have been rapidly developed.

  For example, in MPEG-4, for example, a moving image scene at a certain moment is considered as an aggregate for each object such as “person”, “car”, “background”, “music”, and “video”, and is adapted to each object. The moving image data is encoded with high efficiency by performing a typical encoding process.

  In a multi-object encoding technique such as MPEG-4, one scene is expressed by a combination of compression-encoded objects. Therefore, it is possible to easily represent another scene by reusing object data from other MPEG-4 data in a certain MPEG-4 data.

  With the spread of the digital encoding standard as described above, the content industry has strongly raised the problem of copyright protection. That is, there is a problem that it is impossible to provide high-quality content with peace of mind to a standard whose copyright is not sufficiently protected.

  This problem is serious in MPEG-4. This is because, as described above, MPEG-4 can be encoded for each object and manipulated / processed. Accordingly, in multi-object coding such as MPEG-4, protection of intellectual property rights in units of objects is a natural requirement from the viewpoint of smooth reuse of each object. Rights protection is also needed.

  Therefore, in order to protect the copyright of a part of the moving image, a method of temporarily stopping the decoding of the moving image so as not to reproduce a part of the moving image can be considered. This is a method in which when considering a portrait right or copyright in a moving image, the decoding of the related part is stopped, and the decoding is resumed after the related part ends.

  However, the following problems occur.

  In a moving picture coding method, coding using inter-frame correlation is generally used. H. mentioned above. 261, H.M. Coding schemes such as H.263, MPEG-1, 2 and even MPEG-4 are well known. In these encoding methods, encoding is basically performed by referring to the previous frame or the previous and subsequent frames in terms of time and performing motion compensation.

  FIG. 261, H.M. The state of reproduction of H.263, etc. is shown. I * represents a frame to be subjected to intra-frame coding (intra coding), and P * represents a frame to be subjected to inter-frame coding (inter coding). In FIG. 14, the time indicates the direction of time flow, the Security indicates a period during which the decoding process is stopped, with a black band, the code indicates the arrangement of screens in the order of encoding processing, and the display indicates the arrangement of screens in the display order.

  Now, it is assumed that decryption is stopped for the purpose of copyright protection (Security) during the period from P4 to P9. The decoding of the moving image stops at P3, and the image is not displayed until decoding is resumed thereafter. Since the writing of the encoded data to the buffer is stopped simultaneously with the stop of the decoding, the encoded data of P4 to P9 is discarded. Therefore, when decoding from P10 is started, P9 that should be referred to by P10 is discarded, so that decoding processing after P10 is not performed normally and until I1 for which intraframe coding is performed is decoded. The images of the frames from P10 to P13 are disturbed or the decoding operation is interrupted.

  In the MPEG-1, 2, 4 encoding method, bidirectional motion compensation is also used in which prediction is performed from the front and back in time.

  This is shown in FIG. B * represents a frame to be encoded in both directions (bidirectional). In FIG. 15, the time indicates the direction of time flow, the Security indicates a period during which the decoding process is stopped, with a black band, the code indicates the arrangement of screens in the order of encoding processing, and the display indicates the arrangement of screens in the display order. Depending on the B frame, the screen encoding process order and the original screen order are different.

  Now, it is assumed that the decoding of the encoded data from B10 to P3 is stopped due to copyright protection. Since the motion compensation is not performed correctly as in FIG. 14, the decoding process is not normally performed on the images of the frames B30, B31, B40, and B41, and the image is disturbed or the decoding operation is interrupted.

  SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide an image processing apparatus and method for suitably processing image reproduction / stop by copyright protection of an image, and a computer-readable recording medium on which an image processing program is recorded in consideration of the above problems. It is said.

  An image processing apparatus according to the present invention includes an input unit that inputs image data encoded using at least an encoding mode having an intra encoding mode and an inter encoding mode, and security data for protecting the image data. Determining means for determining whether reproduction of the encoded image data is permitted based on the security data; encoding mode determining means for determining an encoding mode of the encoded image data; and Decoding means for decoding the encoded image data and outputting the decoded image data, and decoding by the decoding means according to the output of the determination means and the output of the encoding mode determination means Control means for controlling processing, and the control means determines the coding mode when the judgment means permits reproduction from reproduction not permitted. There and controls so as to stop the decoding process to the image data determined as the intra coding mode.

  Further, the image processing apparatus of the present invention inputs at least image data encoded using an encoding mode having an intra encoding mode and an inter encoding mode and security data for protecting the image data. Means for determining whether or not reproduction of the encoded image data is permitted based on the security data; encoding mode determination means for determining an encoding mode of the encoded image data; , Decoding means for decoding the encoded image data, and outputting the decoded image data, and according to the output of the determination means and the output of the encoding mode determination means, Control means for controlling the decoding process, and the control means determines the encoding mode when reproduction is permitted from reproduction not permitted by the determining means. Means and controlling to resume decoding from the image data determined as the intra coding mode.

  Also, the image processing method of the present invention is an input for inputting image data encoded using at least an encoding mode having an intra encoding mode and an inter encoding mode and security data for protecting the image data. A step of determining whether or not reproduction of the encoded image data is permitted based on the security data; an encoding mode determination step of determining an encoding mode of the encoded image data; A decoding step for decoding the encoded image data in accordance with the determination result and the determination result, wherein the decoding step has been permitted from reproduction not permitted by the determination step. In this case, the decoding process is stopped until the image data determined in the encoding mode determination step as the intra encoding mode.

  Also, the image processing method of the present invention is an input for inputting image data encoded using at least an encoding mode having an intra encoding mode and an inter encoding mode and security data for protecting the image data. A step of determining whether or not reproduction of the encoded image data is permitted based on the security data; an encoding mode determination step of determining an encoding mode of the encoded image data; A decoding step for decoding the encoded image data in accordance with the determination result and the determination result, wherein the decoding step has been permitted from reproduction not permitted by the determination step. In this case, the decoding process is restarted from the image data determined in the encoding mode determination step as the intra encoding mode.

  As is apparent from the above description, in the present invention, it is possible to suitably process stop / playback of a moving image by copyright protection of the moving image. As a result, it is possible to prevent the occurrence of an error in the operation of the decoder at the time of decoding and to operate it safely and reliably, and to prevent image disturbance based on inappropriate motion compensation.

<First embodiment>
Hereinafter, a first embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing the configuration of an image processing apparatus as a first embodiment of the present invention.

  In FIG. 1, reference numeral 1001 denotes a storage device that stores moving image data. The moving image data is composed of encoded data of each moving image and security encoded data obtained by encoding information for protecting the copyright. In this embodiment, the security encoded data is transmitted before moving image data is transmitted. However, the present invention is not limited to this method, and the security encoded data may be time-division multiplexed on the moving image data and transmitted.

  In this embodiment, a case will be described in which moving image encoded data is encoded by the MPEG-1 encoding method. The MPEG-1 encoding method is described in detail in ISO / IEC 11172-2, and thus the description thereof is omitted. Further, in order to facilitate the description, it is assumed that encoding is performed in units of frames, and it consists of I frames for performing intra-frame encoding and P frames using inter-frame correlation. Note that this moving image data may be any encoding method using intra-frame encoding frames and inter-frame correlation. Of course, field-based encoding may be used.

  FIG. 2 shows the structure of security encoded data.

  In FIG. 2, 2001 is a CodeLength code representing the code length of security-encoded data, 2002 is an IPcode code representing information necessary for copyright authentication, 2003 is a Security StartTime code indicating the time to start protecting copyrights, etc., 2004 Is a SecurityEndTime code indicating the time for releasing protection of copyright and the like. In addition, the code | symbol of 2002-2004 will continue the combination, if the moving image part which wants to apply security is divided | segmented and there are multiple.

  Returning to the description of FIG. 1, reference numeral 1002 denotes a communication circuit, which includes a LAN, a public line, a wireless line, a broadcast wave, and the like. A communication interface 1003 receives moving image data from the communication circuit 1002.

  A security decoding unit 1004 decodes the security encoded data described above, and instructs the stop and restart of the playback of the moving image compared with the authentication data. 1005 decodes the encoded moving image data and plays the moving image A video decoding unit 1006 is a monitor for displaying the reproduced moving image.

  A processing operation of moving image data in the apparatus configured as described above will be described below.

  An operator (not shown) first selects moving image data to be reproduced from the storage device 1001. The moving image encoded data and the security encoded data of the selected moving image data are read, and the moving image encoded data is input to the video decoding unit 1005 and the security encoded data is input to the security decoding unit 1004.

  FIG. 3 is a detailed block diagram of the security decryption unit 1004.

  In FIG. 3, 151 is an input terminal for inputting security encoded data, 152 is a security decoder that decodes the security encoded data and obtains authentication information and information related to copyright protection time, and 153 analyzes the authentication information. A security authenticator that determines whether to perform decoding or prohibition, 154 is a timer that generates a time for correctly reproducing a moving image, and 155 controls the video decoding unit 1005 according to the determination of the security authenticator 153. A control signal generator 156 for generating a signal to be performed is an output terminal for outputting the control signal generated by the control signal generator 155 to the video decoding unit 1005.

  Decryption operation of the security decryption unit 1004 configured as described above will be described below.

  The security encoded data input from the input terminal 151 is decoded by the security decoder 152, and the CodeLength code is decoded from the code shown in FIG. 2 to grasp the entire amount of the encoded data and to protect the copyright. For the portion, the IPcode code indicating information necessary for copyright authentication, the SecurityStartTime code indicating the time for starting copyright protection, and the SecurityEndTime code indicating the time for releasing the copyright protection are decoded to acquire the respective information.

  Information necessary for authentication is input to the security authenticator 153. Further, when there is no instruction from the security authenticator 153, the control signal generator 155 generates a control signal from the output terminal 156 to the video decoding unit 1005 so that the video decoding unit 1005 performs the decoding process of the moving image data.

  The security authenticator 153 compares with authentication information that has been acquired in advance or entered by the operator, and determines whether or not to permit decryption. When permitting the decoding process, the control signal generator 155 is informed of the permission of decoding. At this time, if the video decoding unit 1005 has already generated a control signal so as to decode moving image data, the control signal is maintained as it is.

  On the other hand, when the decryption process is prohibited, the security authenticator 153 outputs a signal prohibiting the decryption process to the control signal generator 155, and the control signal generator 155 performs the protection of the corresponding copyright input from the security decoder 152. The information about the time to perform (information obtained by decoding the Security StartTime code) and the time input from the timer 154 are compared, and when the time when the copyright protection starts becomes equal to the time, a signal for stopping the decoding of the moving image is generated. Output from terminal 156.

  In addition, the information regarding the time for performing copyright protection (information obtained by decoding the SecurityEndTime code) is compared with the time inputted from the timer 154, and when the time when the copyright protection is finished becomes equal to the time, A signal for resuming the decoding of the image is output from the output terminal 156.

  The video decoding unit 1005 receives encoded moving image data from the storage device 1001 and a control signal from the security decoding unit 1004.

  FIG. 4 is a detailed block diagram of the video decoding unit 1005.

  In FIG. 4, 101 is an image data input terminal for inputting moving image encoded data from the storage device 1001, and 102 is a control signal input terminal for inputting a control signal from the security decoding unit 1004.

  Reference numeral 103 is a buffer for storing input moving image encoded data, 104 is for reading MPEG-1 encoded data from the buffer 103, header information indicating the content of the code, motion compensation code for motion compensation, DCT conversion It is a separator that separates data in which coefficients are encoded.

  A header decoder 105 decodes the separated header information, and acquires information by decoding various start codes, image sizes, and encoding modes of each frame.

  106 is a motion compensation decoder that decodes the motion compensation code, decodes the encoding mode and motion vector of each macroblock, and acquires information necessary for motion compensation, and 107 is a texture decoder that decodes the code. , Quantization coefficient, quantization result of DCT transform coefficient, and the like.

  Reference numeral 108 denotes an inverse quantization / IDCT device which inputs quantization coefficients, quantization results of DCT transform coefficients, etc., performs inverse quantization and inverse DCT transform, and obtains pixel data or error data by motion compensation.

  109 is an adder for adding the motion compensation result and the error data, 110 is a reference frame memory for storing a reference frame necessary for motion compensation, 111 is a motion of the contents of the reference frame memory 110 according to the result of the motion vector decoder 106. It is a motion compensator that performs compensation.

  Reference numeral 113 denotes a display frame memory that temporarily stores the decoded frame image for display. Reference numeral 112 denotes a frame mode determiner that receives a frame encoding mode from the header decoder 105 and determines whether the frame is an I frame or a P frame. is there. The frame mode determination unit 112 controls the reference frame memory 110, the display frame memory 113, and the like. Reference numeral 114 denotes an output terminal for outputting moving image data to the monitor 1006.

  Decoding processing in the video decoding unit 1005 configured as described above will be described below.

  When the control signal from the control signal input terminal 102 indicates that decoding is possible, the buffer 103 inputs moving image encoded data from the image data input terminal 101 and accumulates the information. When the control signal from the control signal input terminal 102 instructs to prohibit the decoding, the buffer 103 does not store the moving image encoded data and discards the data.

  First, the decoding operation when the control signal indicates that decoding is possible will be described.

  The separator 104 receives the encoded data from the buffer 103, and separates the header information indicating the content of the code, the motion compensation code for performing motion compensation, and the data encoded with the DCT transform coefficient. The header decoder 105 acquires information by decoding various start codes, image sizes, and encoding modes of each frame from the header information separated by the separator 104.

  The motion compensation decoder 106 decodes the motion compensation code separated by the separator 104, decodes the coding mode, motion vector, etc. of each macroblock, obtains information necessary for motion compensation, If this information is output to the motion compensator 111, motion compensation is performed.

  The texture decoder 107 decodes the encoded data separated by the separator 104 to obtain quantization coefficients, quantization results of DCT transform coefficients, and the like. The decoded quantized data is input to the inverse quantization / IDCT unit 108 to obtain pixel data or error data by motion compensation.

  The error data is added to output data from the motion compensator 111 in an adder 109. The motion compensator 111 performs a motion compensation from the reference frame memory 110 according to the motion vector obtained from the value 0 if it is a macroblock that performs intraframe coding from the motion compensation decoder 106, and performs reference data Is output to the adder 109.

  The pixel value obtained by the adder 109 is stored in the reference frame memory 110 and the display frame memory 113. The image data stored in the display frame memory 113 is output to the monitor 1006 via the output terminal 114 and displayed.

  Next, the decoding operation when the control signal instructs decoding prohibition from decoding permission will be described.

  The buffer 103 stops the code input, and stops the operations of the header decoder 105, the motion compensation decoder 106, the motion compensator 111, and the texture decoder 107. Further, updating of the reference frame memory 110 and the display frame memory 113 is prohibited. As a result, the image data of the last decoded frame is output from the output terminal 114. For this reason, the monitor 1006 continues to display the last frame permitted to be decoded.

  Next, the decoding operation when the control signal instructs decoding permission from decoding prohibition and instructs decoding restart will be described.

  The buffer 103 resumes the code input and operates the header decoder 105. The header decoder 105 detects and determines the next start code. When the start code of the frame is recognized, the encoding mode of the frame is input to the frame mode determination unit 112.

  If the frame coding mode is P frame, the frame mode decision unit 112 does not resume the operations of the motion compensation decoder 106, motion compensator 111, and texture decoder 107, and the reference frame memory 110 and display frame memory 113 Maintain a state where updating is prohibited.

  If the frame coding mode is I frame, the frame mode determination unit 112 restarts the operations of the motion compensation decoder 106, motion compensator 111, and texture decoder 107, and the reference frame memory 110 and the display frame memory 113. Resume updates for. Thereafter, the operation of each unit is continued until the control signal is prohibited from being decoded. Thereby, the display on the monitor 1006 is also updated, and the display of the moving image is resumed.

  When decoding is resumed by a series of selection operations as described above, it is easy to prevent the frame image from being disturbed or the decoding operation from being interrupted by performing processing from the frame that has been subjected to intraframe coding. can do.

  For example, as shown in FIG. 5, even when decoding is prohibited due to copyright protection (Security) during the encoding frame period from P4 to P9, from the time when Security is applied, the frame immediately before the Security is applied. Since the display of the frame P3 continues and the decoding process is resumed from the next frame I1 or later, there is no interruption in the image or image distortion, and the operator is not shown an unpleasant image.

  In the above embodiment, during the period when decoding is prohibited, the decoded image is output immediately before the prohibition. The predetermined image is stored in advance in a memory or the like, and the predetermined image is output. It may be.

  In this embodiment, the memory configuration and the like can be changed as appropriate. Further, by performing the same operation on moving image data input from the communication circuit 1002 via the communication interface 1003, the same effect can be obtained for communication data.

  Further, an output terminal 199 as shown in FIG. 6 is provided, and information on the header decoder 105, for example, information on the decoding time of the frame (for example, decoding result of time_code code of group of picture, decoding result of temporal_reference code of picture) is provided. By outputting to the outside, it is also possible to easily reconstruct the synchronization with the voice or the like when decoding is resumed.

<Second embodiment>
FIG. 7 is a block diagram showing the configuration of the video decoding unit 1005 as the second embodiment of the present invention. In addition, the same number is attached | subjected about the component similar to a 1st Example, The detailed description is abbreviate | omitted.

  In FIG. 7, reference numeral 201 denotes a motion compensator capable of performing motion compensation in both directions, which is motion compensation of a B frame, in addition to motion compensation of a P frame, and 202 and 203 store reference frames necessary for motion compensation. The reference frame memory functions in the same manner as the reference frame memory 110.

  Reference numeral 204 denotes a display frame memory which operates in the same manner as the display frame memory 113, but the input destination is selected in addition to the adder 109 and the reference frame memories 202 and 203.

  Decoding processing in the video decoding unit 1005 configured as described above will be described below.

  First, the decoding operation when the control signal indicates that decoding is possible will be described.

  The separator 104 receives the encoded data from the buffer 103 and separates it into information. The code is distributed to the header decoder 105, motion compensation decoder 106, and texture decoder 107.

  The motion compensator 106 decodes the encoding mode, motion vector, etc. of each macroblock, acquires information necessary for motion compensation, and inputs these information to the motion compensator 201 for P and B frames, Compensate. If it is a P frame, the motion compensator 201 performs motion compensation with reference to the content of the frame updated last from the reference frames 202 and 203. If it is a B frame, the reference frames 202 and 203 are referred to, and motion compensation is performed with the content of the last updated frame as the forward prediction reference frame and the other frame as the backward prediction reference frame.

  The texture decoder 107 and the inverse quantization / IDCT unit 108 obtain pixel data or error data by motion compensation, and the adder 109 adds the error data and the result of the motion compensator 201.

  If the pixel value (decoded image) obtained by the adder 109 is an I frame or a P frame, a reference frame that stores the contents of a frame decoded in the past in the reference frame memory 202 or 203. Store in memory. When the B frame continues, the decoded image of the I frame or the P frame is not stored in the display frame memory 204, and after the decoding of the B frame and the output to the display memory are finished, the decoding of the I frame or the P frame is performed. Transfer from the reference frame memory 202 or 203 storing the image to the display frame memory 113 is performed.

  If the pixel value (decoded image) obtained by the adder 109 is a B frame, writing to the reference frame memories 202 and 203 is not performed, but only writing to the display frame memory 204 is performed.

  The image data stored in the display frame memory 204 is output to the monitor 1006 via the output terminal 114 and displayed.

  Next, the decoding operation when the control signal instructs decoding prohibition from decoding permission will be described.

  The buffer 103 stops the code input, and stops the operations of the header decoder 105, the motion compensation decoder 106, the motion compensator 201, and the texture decoder 107. Further, updating of the reference frame memory 202, the reference frame memory 203, and the display frame memory 204 is prohibited.

  If the result of the frame mode decision unit 112 is after decoding of the I frame, the update is prohibited as it is. If the P frame has been decoded, the contents of the reference frame memory that has been decoded and written are read from the reference frame memory 202 or 203 and transferred to the display frame memory 204, and all updates are prohibited. If it is after the decoding of the B frame, the content of the reference frame memory that was last updated is read from the reference frame memory 202 or 203 and transferred to the display frame memory 204, and all updates are prohibited.

  As a result, the image data of the frame decoded last in time is output from the output terminal 114. For this reason, the monitor 1006 continues to display the last frame permitted to be decoded.

  Next, the decoding operation when the control signal instructs decoding permission from decoding prohibition and instructs decoding restart will be described.

  The buffer 103 resumes the code input and operates the header decoder 105. The header decoder 105 detects and determines the next start code. When the start code of the frame is recognized, the encoding mode of the frame is input to the frame mode determination unit 112.

  If the frame coding mode is P or B frame, the frame mode decision unit 112 does not resume the operations of the motion compensation decoder 106, motion compensator 201, and texture decoder 107, and the reference frame memories 202 and 203 are displayed. The state where the update of the frame memory 204 is prohibited is maintained.

  If the frame coding mode is I frame, the frame mode determination unit 112 restarts the operations of the motion compensation decoder 106, motion compensator 201, and texture decoder 107, and the reference frame memories 202 and 203 and the display frame memory 204 Resume updates for. The encoded data of the B frame following the I frame is not decoded. Since the control signal is permitted for the subsequent I frame or the P frame and thereafter, each part is changed until normal decoding and updating are prohibited. The decoding operation is continued. Thereby, the display on the monitor 1006 is also updated, and the display of the moving image is resumed.

  When decoding is resumed by such a series of selection operations, even if a moving image includes a B frame by performing processing from a frame that has been subjected to intraframe coding, the frame image is disturbed or the decoding operation is interrupted. Can be easily prevented.

  For example, as shown in FIG. 8, even when the decoding frame period from B10 to P3 is prohibited due to copyright protection (Security), the screen before the prohibited period is not normally decoded due to the prohibited period. From the first frame to the first I1 frame that appears after the prohibition period, the display of the P1 frame that was normally decoded immediately before the security is applied continues, and the decoding process is restarted from the I1 frame onward. And no unpleasant image is shown to the operator. An undistorted image can be presented to the operator.

  In the above embodiment, during the period when decoding is prohibited, the decoded image is output immediately before the prohibition, but the predetermined image is stored in advance in a memory or the like and the predetermined image is output. It may be.

<Third embodiment>
FIG. 9 is a block diagram showing a configuration of an image processing apparatus as a third embodiment of the present invention.

  In FIG. 9, reference numeral 300 denotes a central processing unit (CPU) that controls the entire apparatus and performs various processes, and 301 denotes an operating system (OS), software, and a necessary storage area necessary for the operation. Memory to provide.

  302 is a bus for connecting various devices to exchange data and control signals, 303 is a storage device for storing moving image data, 304 is a communication circuit, and is composed of a LAN, public line, wireless line, broadcast radio wave, and the like. Yes.

  A communication interface 305 receives moving image data from the communication circuit 304, a display frame memory 306 that stores frame data of moving images to be displayed, and a monitor 307 that displays reproduced moving images.

  Here, the use and storage status of the memory 301 are shown in FIG.

  The memory 301 controls the entire apparatus, OS for operating various types of software, Systems decoder software for decoding encoded data obtained by encoding information for synchronization control of audio and moving images, protection of copyrights, etc. Security decoder software for decoding encoded data to be performed, moving image decoder software for decoding moving images, and audio decoder software for decoding audio and the like are stored. In addition, there is an image area for storing the decoded image for reference during display and motion compensation, and a working area for storing various calculation parameters.

  In such a configuration, prior to processing, the Systems decoder is operated to separately monitor the synchronization of moving images and sounds. Moving image data including security encoded data, moving image encoded data, and audio encoded data is input from the storage device 303 or the communication interface 305 and stored in the work area of the memory 301. Although the description will be continued assuming that the moving image encoded data is encoded by MPEG-1, any encoding method that performs motion compensation may be used.

  FIG. 11 shows the structure of security encoded data used in the third embodiment.

  The security encoded data used in the third embodiment is A for identifying copyright protection for audio or copyright for moving images, in addition to the codes in FIG. 2 shown in the first embodiment. A / V flag 2005 is added.

  Next, the decoding operation by the CPU 300 will be described according to the flowchart shown in FIG.

  First, it is determined in step S01 whether or not a stream has been input. If a stream has been input, the process proceeds to step S02. If no stream has been input, the flow ends.

  In step S02, the security decoder software is operated to determine whether or not decoding is permitted for the moving image data. If the decryption result of the security decoder is permitted, the process proceeds to step S03, and if the decryption result of the security decoder is prohibited, the process proceeds to step S05.

  In step S03, the moving image decoder software is operated to decode the moving image in accordance with the MPEG-1 decoding method, the decoded image is stored in the image area of the memory 301, and the process proceeds to step S04 to display the image data in the image area. The data is sequentially sent to the display frame memory 306, displayed on the monitor 307, and the process returns to step S01.

  In step S05, the moving picture decoder software is stopped. However, since security can be individually set for voice and moving images, in order to ensure synchronization, readout of encoded data while decoding is stopped is performed synchronously, but is not stored in the working area of the memory 301, It is thrown away.

  Next, in step S06, it is determined whether or not decoding restart is instructed from the security decoder. If restart is not instructed, the process returns to step S05, and if restart is instructed, the process proceeds to step S07.

  In step S07, when a restart of decoding is instructed from the security decoder (step S06), the moving picture decoder software is operated from the stopped state, but first, up to the PTYPE code representing the frame mode of each frame is decoded.

  In step S08, it is determined whether PTYPE is an I frame. If it is not an I frame, the process returns to step S07 to decode the next PTYPE code. If it is an I frame, the process proceeds to step S09.

  In step S09, decoding after PTYPE is resumed, the moving image is decoded in accordance with the MPEG-1 decoding method, the decoded image is stored in the image area of the memory 301, and the process proceeds to step S10.

  In step S10, the image data of the image area is sent to the display frame memory 306 in the display order, displayed on the monitor 307, and the process returns to step S01.

  With such a series of selection operations, part or all of the apparatus can be realized by software. In addition, it was shown that security for copyright protection can be performed individually for audio and moving images.

<Other examples>
The present invention may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, a reader, a printer, etc.) or an apparatus composed of a single device (for example, a digital television, a digital VTR).

  In addition, a software program code for realizing the functions of the embodiment is provided in a computer in an apparatus or a system connected to the various devices so as to operate the various devices so as to realize the functions of the above-described embodiments. What is implemented by operating the various devices according to a program stored in a computer (CPU or MPU) of the system or apparatus supplied is also included in the scope of the present invention.

  In this case, the software program code itself realizes the functions of the above-described embodiments, and the program code itself and means for supplying the program code to the computer, for example, the program code are stored. The storage medium constitutes the present invention.

  As a storage medium for storing the program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, by executing the program code supplied by the computer, not only the functions of the above-described embodiments are realized, but also the OS (operating system) in which the program code is running on the computer, or other application software, etc. Needless to say, the program code is also included in the embodiment of the present invention even when the functions of the above-described embodiment are realized in cooperation with the embodiment.

  Further, the supplied program code is stored in the memory provided in the function expansion board of the computer or the function expansion unit connected to the computer, and then the CPU provided in the function expansion board or function storage unit based on the instruction of the program code However, it is needless to say that the present invention also includes a case where the function of the above-described embodiment is realized by performing part or all of the actual processing.

  For MPEG-4 multi-objects, as shown in FIG. 13, an Objectcode code 2006 for identifying an object is added, security is decoded for each object, a plurality of video decoding units are provided, and each video decoding unit By performing decoding and display from the I-VOP of the object, it is possible to similarly ensure the security of decoding and prevent deterioration of image quality.

  In addition, although the start of decoding is performed in units of frames, the present invention is not limited to this. Of course, units of fields or a plurality of macroblocks may be used.

  Also, the format of the security encoded data, the authentication method, the method of specifying the portion for protecting the copyright, etc. are not limited to the above embodiment, and other formats, authentication methods, etc. may be used.

  Moreover, in the said Example, although the period which applies Security was decided previously, it is not restricted to it. For example, after the user inputs a password for canceling the security and authenticating the password, the security may be cancelled, and the timing for canceling the security may be determined by the user.

1 is a block diagram showing a configuration of an image processing apparatus as a first embodiment of the present invention. It is a figure for demonstrating the security coding data in 1st Example of this invention. It is a figure which shows the structure of the security decoding part 1004 in 1st Example of this invention. It is a figure which shows the structure of the video decoding part 1005 in 1st Example of this invention. It is a figure for demonstrating the mode of the decoding in 1st Example of this invention. It is a figure which shows another structure of the video decoding part 1005 in 1st Example of this invention. It is a figure which shows the structure of the video decoding part 1005 in the 2nd Example of this invention. It is a figure for demonstrating the mode of the decoding in the 2nd Example of this invention. It is a block diagram which shows the structure of the image processing apparatus as 3rd Example of this invention. It is a figure which shows the content of the memory 301 as a 3rd Example of this invention. Security encoded data according to the third embodiment of the present invention will be described. It is a flowchart which shows the image processing operation as 3rd Example of this invention. It is a figure for demonstrating the security coding data corresponding to the MPEG-4 multi-object in the Example of this invention. It is a figure for demonstrating the mode of the conventional decoding. It is a figure for demonstrating the mode of the conventional decoding.

Claims (9)

Input means for inputting image data encoded using an encoding mode having at least an intra encoding mode and an inter encoding mode, and security data for protecting the image data;
Determining means for determining whether reproduction of the encoded image data is permitted based on the security data;
Encoding mode determining means for determining an encoding mode of the encoded image data;
Decoding means for decoding the encoded image data and outputting the decoded image data;
Control means for controlling the decoding process of the decoding means according to the output of the judging means and the output of the encoding mode determining means,
The control unit controls the decoding process to stop until the image data determined to be the intra encoding mode by the encoding mode determination unit when the determination unit determines that the reproduction is permitted but the reproduction is permitted. An image processing apparatus. Input means for inputting image data encoded using an encoding mode having at least an intra encoding mode and an inter encoding mode, and security data for protecting the image data;
Determining means for determining whether reproduction of the encoded image data is permitted based on the security data;
Encoding mode determining means for determining an encoding mode of the encoded image data;
Decoding means for decoding the encoded image data and outputting the decoded image data;
Control means for controlling the decoding process of the decoding means according to the output of the judging means and the output of the encoding mode determining means,
The control unit performs control so that the decoding process is restarted from the image data determined by the encoding mode determination unit to be the intra encoding mode when the determination unit determines that reproduction is permitted from reproduction not permitted. An image processing apparatus.   The image processing apparatus according to claim 1, wherein the inter coding mode includes at least one of a forward prediction coding mode and a bidirectional prediction coding mode.   4. The image processing apparatus according to claim 1, wherein the image data is moving image data.   The image processing apparatus according to claim 1, wherein the security data includes at least a code for authenticating a copyright.   6. The encoded image data includes a code for identifying the encoding mode, and the encoding mode determination unit determines the encoding mode based on the code. Image processing apparatus. An input step of inputting image data encoded using an encoding mode having at least an intra encoding mode and an inter encoding mode and security data for protecting the image data;
A determination step of determining whether or not reproduction of the encoded image data is permitted based on the security data;
An encoding mode determination step for determining an encoding mode of the encoded image data;
A decoding step of decoding the encoded image data in accordance with the determination result and the determination result;
The decoding step stops the decoding process up to the image data determined as the intra encoding mode by the encoding mode determination step when reproduction is permitted from reproduction non-permitted in the determination step. Processing method. An input step of inputting image data encoded using an encoding mode having at least an intra encoding mode and an inter encoding mode, and security data for protecting the image data;
A determination step of determining whether or not reproduction of the encoded image data is permitted based on the security data;
An encoding mode determination step for determining an encoding mode of the encoded image data;
A decoding step of decoding the encoded image data in accordance with the determination result and the determination result;
The decoding step restarts the decoding process from the image data determined to be the intra encoding mode by the encoding mode determination step when the reproduction is permitted from the reproduction not permitted by the determination step. Processing method.   A computer-readable storage medium storing a computer program for executing the image processing method according to claim 7 or 8.

Images