JP2020182011A - Image processing method and image processing device - Google Patents

Abstract

To suppress the execution of unnecessary decoding in an image processing device.SOLUTION: In an image processing method executed by an image processing device, second encoded data is generated by executing a decryption process on first encrypted data generated by encrypting first encoded data including the encoded image data and first data, and it is determined whether second coded data includes the first data, and when the second coded data includes the first data, the second coded data is decoded, and when the second coded data does not include the first data, the second encoded data is discarded without being decoded.SELECTED DRAWING: Figure 5

Description

The present invention relates to an image processing method and an image processing apparatus.

In Patent Document 1, an image that generates plaintext encoded image data by executing a decoding process on the encrypted encoded image data and decodes the plaintext encoded image data. The processing equipment is described.

Japanese Unexamined Patent Publication No. 2004-158936

In an image processing apparatus as described in Patent Document 1, when a defect occurs in the decoding process, information including noise is generated instead of the plaintext encoded image data, and the information including noise is decoded. Therefore, useless decoding is executed.

The image processing method according to the present invention is an image processing method executed by an image processing apparatus, and is a first encryption generated by encryption of first encoded data including encoded image data and first data. The second coded data is generated by executing the decoding process on the encrypted data, it is determined whether or not the second coded data includes the first data, and the second coded data is the second coded data. When one data is included, the second coded data is decoded, and when the second coded data does not include the first data, the second coded data is discarded without being decoded.

The image processing apparatus according to the present invention performs a decryption process on the first encrypted data generated by encrypting the first encoded data including the encoded image data and the first data. A generation unit that generates two coded data, a decoding unit, a determination unit that determines whether or not the second coded data includes the first data, and the second coded data generate the first data. If it is included, the decoding unit decodes the second encoded data, and if the second encoded data does not include the first data, the decoding unit discards the second encoded data without decoding it. Includes a decoding control unit.

It is a figure which shows an example of the system 1000 which includes an image supply device 1 and a projector 2. It is a figure which shows the outline of the communication of the image frame D1 in the system 1000. It is a figure which shows an example of the image supply device 1. It is a figure which shows an example of the projector 2. It is a flowchart for demonstrating the processing operation of the encrypted image stream.

A: First Embodiment A1: Outline of the system 1000 FIG. 1 is a diagram showing an example of the system 1000 including the image supply device 1 and the projector 2.

The image supply device 1 is, for example, a PC (Personal Computer). The image supply device 1 is not limited to a PC, and may be, for example, a tablet terminal, a smartphone, a video playback device, a DVD (Digital Versatile Disc) player, a Blu-ray disc player, a hard disk recorder, a TV tuner device, or a video game machine.

The image supply device 1 wirelessly transmits distribution data including moving image data indicating a moving image and audio data indicating a sound to the projector 2. The image supply device 1 may transmit the distribution data to the projector 2 by wire. The distribution data is, for example, data representing a movie. The distribution data is not limited to data representing a movie. For example, the distribution data may be data representing a television program. Delivery data can also be referred to as content.

The projector 2 is arranged, for example, on a desk, table or floor. The projector 2 may be hung from the ceiling. The projector 2 receives the distribution data from the image supply device 1. The projector 2 displays the moving image on the projection surface 3 by projecting the moving image based on the moving image data included in the distribution data on the projection surface 3. The projector 2 is an example of a display device. The display device is not limited to the projector 2, and may be a display, for example, an FPD (Flat Panel Display). The FPD is, for example, a liquid crystal display, a plasma display or an organic EL (Electro Luminescence) display. Further, the projector 2 outputs a sound based on the audio data included in the distribution data from the speaker 212.

In the system 1000, Miracast is used as a method of wireless communication of image data. Miracast is a registered trademark. In Miracast, HDCP (High bandwidth Digital Content Protection) content protection technology is used, and an MPEG2-TS (Moving Picture Experts Group 2-Transport Stream) container is used.

A2: Outline of transmission of image frame D1 in system 1000 FIG. 2 is a diagram showing an outline of communication of image frame D1 in system 1000. The image frame D1 is an example of image data.

The image supply device 1 uses the image frame D1 constituting the image stream as H. The encoded image frame D2 is generated by encoding based on 264. H. 264 is a standard for video compression. H. 264 is H. Also called 264 / AVC (Advanced Video Coding). The coded image frame D2 is an example of coded image data. Compression is an example of encoding or encoding. In the example shown in FIG. 2, it is assumed that an I frame, which will be described later, is generated as the encoded image frame D2. The encoded image frame is not limited to the I frame, and may be a P frame or a B frame described later.

The image supply device 1 includes H. Based on 264, the first coded packet D3 is generated by arranging the start code SC indicating the position of the coded image frame D2 in front of the coded image frame D2. The start code SC is an example of the first data and the second data. The start code SC is hint information for detecting the encoded image frame D2. The start code can also be referred to as synchronization information or a sync word. The first coded packet D3 is an example of the first coded data. In the first coded packet D3, the start code SC is located at the beginning of the header D31 and the encoded image frame D2 is located at the payload D32. The header D31 also includes information for decoding, specifically decompressing, the encoded image frame D2.

The image supply device 1 generates the first encrypted data D4 by encrypting the first coded packet D3 based on HDCP.
Specifically, the image supply device 1 generates the first coded packet D3 by using the encryption key generated based on the count number of the image frames to be encrypted and the encryption key 51a. The image supply device 1 generates a first encrypted PES (Packetized Elementary Stream) packet D5 in which the first encrypted data D4 is mounted on the payload D52. In the header D51 of the first encrypted PES packet D5, the encryption key 51a used for encryption and the decryption flag 51b indicating whether or not the decryption process for the payload D52 is necessary are located. When the decryption process for the payload D52 is required, the decryption flag 51b is set. When the decryption process for the payload D52 is not required, the decryption flag 51b is not set.

The image supply device 1 wirelessly transmits an encrypted image stream including the first encrypted PES packet D5 to the projector 2 by a transport stream (TS).

When the projector 2 receives the encrypted image stream transmitted by the transport stream, the projector 2 extracts the first encrypted PES packet D5 from the encrypted image stream.

When the decryption flag 51b of the header D51 of the first encrypted PES packet D5 is set, the projector 2 uses the encryption key 51a of the header D51 to execute a decryption process on the payload D52 to perform a second decryption process. Generate the coded packet D6.

Here, when the decoding process for the payload D52 is normally executed, the start code SC is located in the header D61 and the encoded image frame D2 is located in the payload D62 in the second coded packet D6.

On the other hand, if there is a problem in the decryption process for the payload D52, for example, if the encryption key 51a is changed due to the influence of noise during wireless transmission, or if a TS packet loss occurs, the second code In the encrypted packet D6, the start code SC is not located in the header D61, and the encoded image frame D2 is not located in the payload D62.

Therefore, when the start code SC is present in the header D61, the projector 2 generates the image frame D1 by decoding the second coded packet D6, specifically, the encoded image frame D2 of the payload D62. ..
Further, when the start code SC does not exist in the header D61, the projector 2 discards the second coded packet D6, specifically, the encoded image frame D2 of the payload D62 without decoding.

A3: An example of the image supply device 1. FIG. 3 is a diagram showing an example of the image supply device 1. The image supply device 1 includes an image data output unit 101, an image encoder 102, an image encoder 103, an audio data output unit 104, an audio encoder 105, an audio encoder 106, a multiplexer 107, and a first communication unit 108. And, including.

The image data output unit 101 is composed of, for example, a circuit. The image data output unit 101 outputs a plurality of image frames D1 indicating a moving image. The plurality of image frames D1 indicate, for example, a moving image of a movie.

The image encoder 102 is composed of, for example, a circuit. The image encoder 102 sets each of the plurality of image frames D1 to H. A plurality of encoded image frames D2 are generated by compressing in the 264 format. The format of video compression is H. Not limited to 264. For example, as a video compression format, H. 265 may be used. H. 265 is H. It is also called 265 / HEVC (High Efficiency Video Coding). The image encoder 102 arranges the start code SC in front of the start position of the coded image frame D2 for each coded image frame D2. H. In 264, "0x001" is used as the start code. H. In 265, "0x0001" is used as the start code.

Each of the plurality of encoded image frames D2 is either an I frame, a P frame, or a B frame.

The I frame is an intra coded frame. An I-frame is a frame generated by encoding an image frame by intraframe compression. The image frame that is the basis of the I frame is an example of the first image frame. An I frame is a frame that can be decoded independently without referring to other frames.

The P frame is a Predicted Frame. The P frame is a frame generated by encoding an image frame by inter-frame compression. The image frame based on the P frame is an example of the second image frame. A P-frame is a frame generated by encoding in an image stream by inter-frame compression using an image frame that exists before the original image frame of the P-frame. The P frame is an example of image-encoded data.

The B frame is a Bidirectional Predicted Frame. The B frame is a frame encoded by inter-frame compression. The B frame is an image frame that exists in the image stream before the original image frame of the B frame and an image that exists in the image stream after the original image frame of the B frame. A frame and a frame encoded by inter-frame compression using.

The start code SC and the I frame indicating the start position of the I frame are included in the first coded packet D3. The start code SC indicating the start position of the I frame is included at the beginning of the header D31 of the first coded packet D3, and the I frame is included in the payload D32 of the first coded packet D3.

The start code SC and the P frame indicating the start position of the P frame are included in the third coded packet. The start code SC indicating the start position of the P frame is included at the beginning of the header of the third coded packet, and the P frame is included in the payload of the third coded packet. The third coded packet is an example of the third coded data.

The start code SC and the B frame indicating the start position of the B frame are included in the fifth coded packet. The start code SC indicating the start position of the B frame is included at the beginning of the header of the fifth coded packet, and the B frame is included in the payload of the fifth coded packet.

The plurality of coded image frames D2 and the plurality of start codes SC form a first coded image stream. The first coded image stream includes, for example, a first coded packet, a third coded packet, and a fifth coded packet.

The image crypter 103 is composed of, for example, a circuit. The image crypter 103 creates an encrypted image stream by encrypting the first encoded image stream. The encrypted image stream includes, for example, a first encrypted PES packet, a third encrypted PES packet, and a fifth encrypted PES packet.

The image encrypter 103 generates the first encrypted data D4 by executing the encryption using the encryption key for the first encoded packet D3 including the I frame. The image encrypter 103 generates a first encrypted PES packet D5 that mounts the first encrypted data D4 on the payload D52. The image encrypter 103 mounts the encryption key 51a on the header D51 of the first encrypted PES packet D5. The header D51 of the first encrypted PES packet D5 is provided with the decryption flag 51b.

The image encrypter 103 generates the third encrypted data by executing the encryption using the encryption key on the third encoded packet including the P frame. The third encrypted data is an example of the second encrypted data. The image encrypter 103 generates a third encrypted PES packet that carries the third encrypted data in the payload. The image encrypter 103 mounts an encryption key in the header of the third encrypted PES packet. A decryption flag is provided in the header of the third encrypted PES packet.

The image encrypter 103 generates the fifth encrypted data by executing the encryption using the encryption key for the fifth encrypted packet including the B frame. The image encrypter 103 generates a fifth encrypted PES packet that carries the fifth encrypted data in the payload. The image encrypter 103 mounts an encryption key in the header of the fifth encrypted PES packet. A decryption flag is provided in the header of the fifth encrypted PES packet.

The audio data output unit 104 is composed of, for example, a circuit. The audio data output unit 104 outputs an audio stream including audio data indicating audio. The audio data indicates, for example, the audio of a movie.

The voice encoder 105 is composed of, for example, a circuit. The audio encoder 105 produces a encoded audio stream by compressing the audio stream. Hereinafter, the encoded audio stream will be referred to as a “first encoded audio stream”. The first coded voice stream includes the first coded voice packet. The payload of the first coded voice packet is loaded with coded voice data generated by encoding the voice data.

The voice crypter 106 is composed of, for example, a circuit. The voice crypter 106 generates an encrypted voice stream by encrypting the first coded voice stream. Hereinafter, the encrypted audio stream will be referred to as an "encrypted audio stream". The voice encoder 106 generates encrypted voice data by performing encryption using an encryption key on the first coded voice packet. The voice crypter 106 generates a voice PES packet with the encrypted voice data loaded in the payload. The voice crypter 106 mounts an encryption key in the header of the voice PES packet.

The multiplexer 107 is composed of, for example, a circuit. The multiplexer 107 generates a multiplexed stream by multiplexing the encrypted image stream and the encrypted audio stream.

The first communication unit 108 is composed of, for example, a circuit. The first communication unit 108 wirelessly transmits the multiplexed stream as a transport stream to the projector 2. The multiplexed stream is an example of distribution data.

A4: Example of projector 2 FIG. 4 is a diagram showing an example of projector 2. The projector 2 includes a second communication unit 201, a demultiplexer 202, an image decryptor 203, an image decoder 204, an image processing unit 205, a frame memory 206, a light valve drive unit 207, a projection unit 208, and the like. It includes a voice crypter 209, a voice decoder 210, a voice processing unit 211, a speaker 212, a storage unit 213, and a processing unit 214. The projection unit 208 includes a light source 21, a red liquid crystal light valve 22R, a green liquid crystal light valve 22G, a blue liquid crystal light valve 22B, and a projection optical system 23. The second communication unit 201, the demultiplexer 202, the image decryptor 203, the image decoder 204, the storage unit 213, and the processing unit 214 are included in the image processing device 100.

Hereinafter, when it is not necessary to distinguish the red liquid crystal light bulb 22R, the green liquid crystal light bulb 22G, and the blue liquid crystal light bulb 22B from each other, these are referred to as "liquid crystal light bulb 22".

Further, when it is not necessary to distinguish the first encrypted PES packet D5, the third encrypted PES packet, the fifth encrypted PES packet, and the voice PES packet from each other, these are referred to as "PES packets".

The second communication unit 201 is composed of, for example, a circuit. The second communication unit 201 receives the multiplexed stream wirelessly transmitted by the transport stream from the image supply device 1.

The demultiplexer 202 is composed of, for example, a circuit. The demultiplexer 202 is also referred to as a Demuxer. The demultiplexer 202 generates an encrypted image stream and an encrypted audio stream from the multiplexed stream transmitted by the transport stream. The demultiplexer 202 provides the encrypted image stream to the image decryptor 203. The demultiplexer 202 provides an encrypted voice stream to the voice decryptor 209.

The image decrypter 203 is composed of, for example, a circuit. The image decryptor 203 generates a second coded image stream by executing a decryption process on the encrypted image stream. The image decrypter 203 is an example of a generation unit. For example, the image decrypter 203 operates as follows.

The image decryptor 203 encrypts the first encrypted data mounted on the payload D52 of the first encrypted PES packet D5 in the encrypted image stream in the header D51 of the first encrypted PES packet D5. The second coded packet D6 is generated by executing the decryption process using the key. If the decoding process is executed normally, there is an I frame in the payload D62 of the second encoded packet D6. The second coded packet is an example of the second coded data. The encryption key is an example of decryption information for decrypting encrypted data.

The image decryptor 203 decrypts the third encrypted data mounted on the payload of the third encrypted PES packet in the encrypted image stream by using the encryption key existing in the header of the third encrypted PES packet. By executing the process, the fourth coded packet is generated. If the decoding process is executed normally, there is a P frame in the payload of the fourth coded packet. The fourth coded packet is an example of the fourth coded data.

The image decryptor 203 decrypts the fifth encrypted data contained in the payload of the fifth encrypted PES packet in the encrypted image stream by using the encryption key existing in the header of the fifth encrypted PES packet. By executing the process, the sixth coded packet is generated. If the decoding process is executed normally, there is a B frame in the payload of the sixth coded packet.

Each of the second coded packet D6, the fourth coded packet, and the sixth coded packet is an element of the second coded image stream. The second coded image stream is another example of the second coded data.

The image decoder 204 is composed of, for example, a circuit. The image decoder 204 is an example of a decoding unit. When the image decoder 204 receives the second coded packet D6 in the second coded image stream, the payload D62 of the second coded packet D6 is based on the start code SC existing in the header of the second coded packet D6. Identify the I-frame of. The image decoder 204 decodes the I frame.

When the image decoder 204 receives the fourth coded packet in the second coded image stream, the image decoder 204 sets the P frame of the payload of the fourth coded packet based on the start code SC existing in the header of the fourth coded packet. Identify. The image decoder 204 decodes the P frame by using the image frame used when generating the P frame, for example, the image frame that is the source of the I frame.

When the image decoder 204 receives the 6th coded packet included in the 2nd coded image stream, the image decoder 204 sets the B frame of the payload of the 6th coded packet based on the start code existing in the header of the 6th coded packet. Identify. The image decoder 204 decodes the B frame using the image frame used when generating the B frame.

The image processing unit 205 is composed of circuits such as a single or a plurality of image processors. The image processing unit 205 receives, for example, an image frame from the image decoder 204.

The image processing unit 205 expands the image frame into the frame memory 206. The frame memory 206 is configured by, for example, a storage device such as a RAM (Random Access Memory). The image processing unit 205 generates an image signal by performing image processing on the image frame expanded in the frame memory 206.

The image processing executed by the image processing unit 205 includes, for example, a resolution conversion process. In the resolution conversion process, the image processing unit 205 converts the resolution of the image frame into, for example, the resolution of the liquid crystal light valve 22. In addition to the resolution conversion process, the image processing unit 205 shows other image processing, for example, a geometric correction process for correcting trapezoidal distortion of an image projected by the projection unit 208, and an image frame provided by the image supply device 1. An OSD process for superimposing an OSD (On Screen Display) image on an image may be executed.

The light bulb drive unit 207 is composed of, for example, a circuit such as a driver. The light bulb drive unit 207 drives the liquid crystal light bulb 22 based on the image signal provided by the image processing unit 205.

The projection unit 208 displays an image on the projection surface 3 by projecting an image based on the image signal on the projection surface 3. The projection surface 3 is an example of a display surface.

The light source 21 is, for example, an LED. The light source 21 is not limited to the LED, and may be, for example, a xenon lamp, an ultrahigh pressure mercury lamp, or a laser light source. The light emitted from the light source 21 is reduced in brightness distribution by an integrator optical system (not shown), and then separated into red, green, and blue color light components, which are the three primary colors of light, by a color separation optical system (not shown). Will be done. The red color light component is incident on the red liquid crystal light bulb 22R. The green color light component is incident on the green liquid crystal light bulb 22G. The blue color light component is incident on the blue liquid crystal light bulb 22B.

The liquid crystal light bulb 22 is composed of a liquid crystal panel or the like in which liquid crystal exists between a pair of transparent substrates. The liquid crystal light bulb 22 has a rectangular pixel region 22a including a plurality of pixels 22p located in a matrix. In the liquid crystal light bulb 22, a drive voltage is applied to the liquid crystal for each pixel 22p. When the light bulb drive unit 207 applies a drive voltage based on the image signal to each pixel 22p, each pixel 22p is set to a light transmittance based on the drive voltage. The light emitted from the light source 21 is modulated by passing through the pixel region 22a, and an image based on the image signal is formed for each colored light. The liquid crystal light bulb 22 is an example of an optical modulation device.

An image of each color is synthesized for each pixel 22p by a color synthesis optical system (not shown) to generate a color image. The color image is projected via the projection optical system 23.

The voice decrypter 209 is composed of, for example, a circuit. The voice decryptor 209 generates a second encoded voice stream by performing a decryption process on the encrypted voice stream. Specifically, the voice decryptor 209 decrypts the encrypted voice data contained in the payload of the voice PES packet included in the encrypted voice stream by using the encryption key existing in the header of the voice PES packet. Generates a second coded voice packet by executing. The second coded voice packet is an element of the second coded voice stream.

The audio decoder 210 is composed of, for example, a circuit. The voice decoder 210 generates voice data by decoding the encoded voice data of the payload of the second coded voice packet contained in the second coded voice stream.

The voice processing unit 211 generates a voice signal based on the voice data. For example, the voice processing unit 211 generates a voice signal by processing voice data based on a volume instruction. The speaker 212 outputs a sound based on an audio signal.

The storage unit 213 is a recording medium that can be read by the processing unit 214. The storage unit 213 includes, for example, a non-volatile memory and a volatile memory. Examples of the non-volatile memory include ROM (Read Only Memory), EPROM (Erasable Programmable Read Only Memory), and EEPROM (Electrically Erasable Programmable Read Only Memory). Examples of the volatile memory include RAM.

The storage unit 213 stores the control program executed by the processing unit 214 and various data used by the processing unit 214.

The processing unit 214 is composed of, for example, a single processor or a plurality of processors. As an example, the processing unit 214 is composed of a single CPU (Central Processing Unit) or a plurality of CPUs (Central Processing Units). A part or all of the functions of the processing unit 214 may be configured by circuits such as DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), PLD (Programmable Logic Device), and FPGA (Field Programmable Gate Array). .. The processing unit 214 executes various processes in parallel or sequentially.

The processing unit 214 functions as the determination unit 214a and the decoding control unit 214b by reading and executing the control program from the storage unit 213.

The determination unit 214a determines whether or not the second coded image stream, which is the decoding result of the image decryptor 203, includes the start code SC. For example, when receiving the second coded packet D6 in the second coded image stream, the determination unit 214a determines whether or not the start code SC exists in the second coded packet D6.
When receiving the fourth coded packet in the second coded image stream, the determination unit 214a determines whether or not the start code SC exists in the fourth coded packet.
When receiving the sixth coded packet in the second coded image stream, the determination unit 214a determines whether or not the start code SC exists in the sixth coded packet.

When the decoding result of the image decryptor 203 includes the start code SC, it is assumed that the decoding process by the image decryptor 203 is normally executed.
On the other hand, if the decoding result of the image decryptor 203 does not include the start code SC, it is assumed that the decoding process by the image decryptor 203 is not normally executed.
For example, if the encryption key used for the decryption process is changed due to the influence of noise during wireless transmission from the image supply device 1 to the projector 2, the encoding included in the second coded image stream. The start code SC does not exist at the beginning of the header of the packet, for example, the second coded packet.
Further, if the first encrypted image stream is changed due to the influence of noise during wireless transmission from the image supply device 1 to the projector 2, even if the encryption key is correct, the second encryption is performed. There is a high possibility that the start code SC does not exist at the beginning of the header of the coded packet included in the image stream, for example, the second coded packet.
Further, if the image decryptor 203 is affected by noise while the image decryptor 203 is executing the decryption process, even if the encryption key is correct, the coded packet included in the second coded image stream, for example, There is a high possibility that the start code SC does not exist at the beginning of the header of the second coded packet.

The determination unit 214a determines whether or not the decoding process by the image decryptor 203 is normally executed by determining whether or not the decoding result of the image decryptor 203 includes the start code SC.

The decoding control unit 214b provides the image decoder 204 with the second coded packet D6 when the second coded packet D6 in the second coded image stream includes the start code SC. The decoding control unit 214b provides the image decoder 204 with the second coded packet D6, so that the image decoder 204 decodes the I frame mounted on the payload of the second coded packet.

The decoding control unit 214b does not provide the second coded packet D6 to the image decoder 204 when the second coded packet D6 does not include the start code SC. By not providing the second coded packet D6 to the image decoder 204, the decoding control unit 214b does not cause the image decoder 204 to decode the I frame mounted on the payload of the second coded packet. In this case, the decoding control unit 214b discards the second encoded packet D6.

When the fourth coded packet in the second coded image stream includes the start code SC, the decoding control unit 214b provides the image decoder 204 with the fourth coded packet to provide the image decoder 204 with the fourth code. The P frame mounted on the payload of the packet is decoded.
If the image frame necessary for decoding the P frame does not exist, the image decoder 204 discards the P frame without decoding it. For example, if the image frame used when the P frame is generated does not exist, for example, if the encoded image frame such as the I frame that encodes the image frame is not decoded, the image The decoder 204 discards the P frame without decoding it.
When the fourth coded packet does not include the start code SC, the decoding control unit 214b does not cause the image decoder 204 to decode the payload of the fourth coded packet by not providing the fourth coded packet to the image decoder 204. In this case, the decoding control unit 214b discards the fourth coded packet.

When the sixth coded packet in the second coded image stream includes the start code SC, the decoding control unit 214b provides the image decoder 204 with the sixth coded packet to provide the image decoder 204 with the sixth code. The B frame mounted on the payload of the packet is decoded.
When the sixth coded packet does not include the start code SC, the decoding control unit 214b does not cause the image decoder 204 to decode the payload of the sixth coded packet by not providing the sixth coded packet to the image decoder 204. In this case, the decoding control unit 214b discards the sixth coded packet.

A5: Example of operation of projector 2 Next, the operation of projector 2 will be described. In the following, for simplification of the description, it is assumed that the image supply device 1 wirelessly transmits the multiplexed stream to the projector 2 as a transport stream. Further, it is assumed that the frame rate of the moving image displayed based on the encrypted image stream included in the multiplexed stream is 30 fps (frame per second). The frame rate is not limited to 30 fps and can be changed as appropriate.

The second communication unit 201 receives the multiplexed stream. The demultiplexer 202 generates an encrypted image stream and an encrypted audio stream from the multiplexed stream received by the second communication unit 201. The demultiplexer 202 provides the encrypted image stream to the image crypter 203 and the encrypted audio stream to the audio crypter 209.

FIG. 5 is a flowchart for explaining the processing operation of the encrypted image stream. In the following, a case where the first encrypted PES packet D5 arrives at the image decryptor 203 will be described as an example.

In step S101, the image decryptor 203 determines whether or not the decryption flag 51b located in the header 51 of the first encrypted PES packet D5 is set.

When the decryption flag 51b is set, in step S102, the image decryptor 203 performs a second decryption process on the first encrypted data D4 of the payload D52 by using the encryption key 51a of the header 51. Generate the coded packet D6. The image decryptor 203 outputs the second coded packet D6 to the determination unit 214a as a part of the second coded image stream.

Subsequently, in step S103, the determination unit 214a initializes the internal timer of the determination unit 214a, and then operates the internal timer. The internal timer may be a timer independent of the determination unit 214a.
Here, since the frame rate of the moving image displayed based on the encrypted image stream is 30 fps, if the decoding process is performed normally, the start code SC in the second encrypted image stream is 1/30 second. Appears once before the passage.
Therefore, the determination unit 214a sets 1/30 second as the timeout time of the internal timer. The timeout time of the internal timer is not limited to 1/30 second, and may be set to a time equal to or greater than the reciprocal of the frame rate of the moving image.

Subsequently, in step S104, the determination unit 214a determines whether or not the start code SC exists in the portion of the second coded packet D6 that has reached the determination unit 214a.

If the start code SC is present, in step S105, the decoding control unit 214b starts providing the second coded packet D6 to the image decoder 204. When the image decoder 204 receives the second coded packet D6, the image decoder 204 identifies the I frame mounted on the payload D62 with reference to the start code SC described in the header D61. Subsequently, the image decoder 204 starts decoding the I frame.

If the start code SC does not exist in step S104, the determination unit 214a determines in step S106 whether or not a timeout has occurred in the internal timer.

When the time-out has not occurred, the determination unit 214a in step S107 sets the second coded image stream newly provided by the image decryptor 203 as a new search target for the start code SC. Subsequently, the process returns to step S104.

When a timeout occurs in step S106, the decoding control unit 214b determines in step S108 that the decoding process has failed, and sends the second coded image stream, specifically, the second coded packet D6 to the image decoder 204. Discard without providing. Therefore, it is possible to suppress the display of a distorted image. In this case, the projection unit 208 maintains the projected image without updating it.

If the decryption flag 51b is not set in step S101, the process proceeds to step S105.
Further, when each of the third encrypted PES packet and the fifth encrypted PES packet arrives at the image decryptor 203, the same operation as when the first encrypted PES packet arrives at the image decryptor 203 is executed.

The voice decryptor 209 secondly encodes the encrypted voice data of the payload of the voice PES packet in the encrypted voice stream by performing a decryption process using the encryption key existing in the header of the voice PES packet. Generate a voice packet. Subsequently, the voice decryptor 209 provides the voice decoder 210 with a second coded voice packet.

The voice decoder 210 generates voice data by decoding the coded voice data of the payload of the second coded voice packet.

The voice processing unit 211 generates a voice signal based on the voice data generated by the voice decoder 210. The speaker 212 outputs a sound based on an audio signal.

A6: Summary of the First Embodiment The image processing method and the image processing apparatus according to the above-described embodiment include the following aspects.

The image decryptor 203 generates the second coded packet D6 by executing the decryption process on the payload D52 of the first encrypted PES packet D5 included in the first encrypted image stream. The determination unit 214a determines whether or not the second coded packet D6 includes the start code SC. When the second coded packet D6 includes the start code SC, the decoding control unit 214b provides the image decoder 204 with the code in the second coded packet D6 by providing the second coded packet D6 to the image decoder 204. The converted image frame D2 is decoded. When the second coded packet D6 does not include the start code SC, the decoding control unit 214b does not provide the second coded packet D6 to the image decoder 204, that is, the second coded packet D6 is provided to the image decoder 204. The second coded packet D6 is discarded without decoding the coded image frame D2 included in.

If the second coded packet D6 does not include the start code SC, there is a possibility that a problem has occurred in the decoding process. According to this aspect, when the second coded packet D6 does not include the start code SC, the data in the payload D62 of the second coded packet D6 is not decoded by the image decoder 204. Therefore, it is possible to suppress the execution of unnecessary decoding of decoding the data that failed to be decoded. Further, when the image corresponding to the decoding result is displayed, it is possible to suppress the display of the image corresponding to the decoding result of the data that failed to be decoded.

The start code SC, which is an example of the first data, is information indicating the position of the coded image frame D2 in the first coded packet D3. Specifically, in the first coded packet D3, the coded image frame D2 is located after the start code SC. Therefore, the start code SC, which is information for identifying the encoded image frame D2, can also be used for determining the success or failure of the decoding process. Therefore, the amount of data can be reduced as compared with the configuration in which dedicated data for determining the success or failure of the decoding process is used as the first data. Although the amount of data is larger than that of the configuration using the start code SC as the first data, it is possible to suppress unnecessary decoding even if the dedicated data for determining the success or failure of the decoding process is adopted. ..

A start code is added to a portion of the second coded image stream for one frame of the image frame D1, for example, a portion generated by the image decryptor 203 in 1/30 second when the frame rate of the image frame D1 is 30 fps. When the SC is present, the determination unit 214a determines that the second coded image stream includes the start code SC. When the start code SC does not exist in the portion of the second coded image stream for one frame of the image frame, the determination unit 214a determines that the second coded image stream does not include the start code SC. Therefore, the portion of the second coded image stream required for determining the presence or absence of the start code SC can be set according to the image frame.

The image decryptor 203 executes a decryption process on the first encrypted image stream that has been received by the second communication unit 201, for example, the first encrypted data D4 included in the received first encrypted image stream. According to this aspect, when the first encrypted data D4 is changed due to the influence of noise or the like during the communication of the first encrypted image stream, the decryption result of the first encrypted data D4 is decrypted, which is useless decoding. Can be suppressed from being executed.

The image decryptor 203 executes a decryption process on the first encrypted data D4 by the second communication unit 201 using the received encryption key 51a. According to this aspect, when the encryption key 51a is changed due to the influence of noise or the like during the communication of the encryption key 51a, a useless decryption of decoding the result of the decryption using the encryption key 51a is executed. Can be suppressed.

B: Modification example The modification of the embodiment illustrated above will be illustrated below. Two or more embodiments arbitrarily selected from the following examples may be appropriately merged to the extent that they do not contradict each other.

B1: First variant example In the first embodiment, Miracast was used as the communication technology, but the communication technology is not limited to Miracast, and the encrypted image data is encrypted and transmitted, and the encrypted image data is encrypted. Any technique may be used as long as it is a technique for receiving the image data of the above and decoding and decoding the image data.

B2. Second Modified Example In the first embodiment and the first modified example, the start code is H.I. 264 or H. Not limited to the start code used in 265, H. 264 and H. A start code used in a compression method different from any of 265 may be used. Further, the first data and the second data are not limited to the start code, and other codes added based on the standard in the image stream may be used. An example of another code is a code that indicates an attribute of an image frame. In addition, a code added to the image stream may be used for trick play such as fast forward and fast rewind.

B3. Third Modified Example In the first embodiment and the first modified example to the second modified example, the decoding control unit 214b generates a P frame in a situation where the fourth coded packet for mounting the P frame on the payload includes a start code. When the encoded frame of the frame used in, for example, the I frame is decoded, the image decoder 204 may decode the P frame using the decoding result of the I frame.
The decoding control unit 214b determines that the coded frame of the frame used for generating the P frame, for example, the I frame, is not decoded in the situation where the fourth coded packet carrying the P frame in the payload contains the start code. The P frame may be discarded without causing the image decoder 204 to decode the P frame.
According to the third modification, it is possible to suppress decoding of another image frame by using an image frame based on the decoding result in which the decoding process is not normally executed, and it is possible to improve the robustness of the image.

B4. Fourth Modified Example In the first embodiment and the first modified examples to the third modified example, the liquid crystal light valve 22 was used as an example of the light modulation device, but the light modulation device is not limited to the liquid crystal light valve and can be appropriately changed. is there. For example, the optical modulation device may be configured by using three reflective liquid crystal panels. Further, the optical modulation device may be configured such as a method using one liquid crystal panel, a method using three digital mirror devices (DMD), a method using one digital mirror device, and the like. When only one liquid crystal panel or DMD is used as the optical modulation device, members corresponding to the color separation optical system and the color synthesis optical system are unnecessary. In addition to the liquid crystal panel and DMD, a configuration capable of modulating the light emitted by the light source 21 can be adopted as the optical modulation device.

B5. Fifth Modification Example When an FPD is used as a display device instead of the projector 2 in the first embodiment and the first modification to the fourth modification, the FPD is mounted on an electronic device such as a tablet terminal or a smartphone, for example. It may be an FPD that is used, or an FPD that is used in an electronic blackboard or an electronic conference system.

1 ... Image supply device, 2 ... Projector, 100 ... Image processing device, 201 ... Second communication unit, 202 ... Demultiplexer, 203 ... Image decryptor, 203 ... Image crypter, 204 ... Image decoder, 205 ... Image processing unit, 206 ... frame memory, 207 ... light valve drive unit, 208 ... projection unit, 213 ... storage unit, 214 ... processing unit, 214a ... determination unit, 214b ... decoding control unit.

Claims (10)

An image processing method executed by an image processing device.
The second coded data is generated by executing a decryption process on the first encrypted data generated by encrypting the first coded data including the coded image data and the first data.
It is determined whether or not the second coded data includes the first data, and
When the second coded data includes the first data, the second coded data is decoded, and when the second coded data does not include the first data, the second coded data is decoded. Discard without
Image processing method. The encoded image data has an encoded image frame and has an encoded image frame.
The first data is information indicating the position of the coded image frame in the first coded data.
The image processing method according to claim 1. In the first coded data, the coded image frame is located after the first data.
The image processing method according to claim 2. The encoded image data is generated by encoding an image frame.
When the first data exists in a portion of the second coded data for one frame of the image frame, it is determined that the second coded data includes the first data.
When the first data does not exist in the portion of the second coded data for one frame of the image frame, it is determined that the second coded data does not include the first data.
The image processing method according to any one of claims 1 to 3. Upon receiving the encrypted data,
The decryption process is executed on the received encrypted data.
The image processing method according to any one of claims 1 to 4. Receive the decryption information for decrypting the encrypted data,
Using the decryption information, the decryption process is executed on the encrypted data.
The image processing method according to any one of claims 1 to 5. The encoded image data is generated by encoding the first image frame.
The second encryption generated by encrypting the third coded data including the image coded data generated by encoding the second image frame using the first image frame and the second data. The fourth coded data is generated by executing the decoding process on the data.
When the second coded data is decoded in a situation where the fourth coded data includes the second data, the fourth coded data is decoded using the decoding result of the second coded data. ,
When the second coded data is discarded without being decoded in a situation where the fourth coded data includes the second data, the fourth coded data is discarded without being decoded.
The image processing method according to any one of claims 1 to 6. A generator that generates the second coded data by executing a decryption process on the first encrypted data generated by encrypting the first coded data including the coded image data and the first data. When,
Decryptor and
A determination unit for determining whether or not the second coded data includes the first data,
When the second coded data includes the first data, the decoding unit is made to decode the second coded data, and when the second coded data does not include the first data, the decoding unit has the decoding unit. A decoding control unit that discards the second encoded data without decoding it,
Image processing equipment including. The first data is a start code included in the header of the second encoded data.
The image processing method according to claim 1. The first coded data is included in the first coded image stream and is encoded according to a moving image compression standard.
The image processing method according to claim 1.

Images