KR20050115805A - Device and method for identifying photographer of image in terminal using camera - Google Patents

Abstract

In a mobile terminal having a camera, the video data captured by the camera in the camera shooting mode is coded using a predetermined coding scheme, the header information of the coded video data stream is generated, inserted into the data stream, and the photographer information is generated as a watermark. Then, a watermark is inserted into the header and the coded data stream, and the watermarked data stream is stored.

Description

DEVICE AND METHOD FOR IDENTIFYING PHOTOGRAPHER OF IMAGE IN TERMINAL USING CAMERA}

The present invention relates to an image processing apparatus and method for a terminal to which a camera is attached, and more particularly, to an apparatus and method for inserting user identification information into a captured image.

In general, an image processing apparatus includes a camera for photographing an image and a display unit for displaying an image signal photographed from the camera. The camera may use a CCD or a CMOS sensor, and the display unit may use an LCD. With the miniaturization of the camera device, the device for capturing the image is increasingly miniaturized. And miniaturized digital cameras are being widely used. In addition, such a camera device is a trend that is mounted on the portable terminal. The portable terminal may capture a video screen and display the moving picture and the still picture, and transmit the captured video screen to the base station.

Miniaturized digital cameras as described above are easy to carry and shoot, and thus, their use is increasing. In addition, since the photographing using the compact camera device is made electronically, no sound is produced when photographing. In this case, you can take a picture secretly without the consent of the other party using a mobile terminal may affect personal privacy. Therefore, it is currently prescribed to forcibly generate a shooting sound when shooting a digital camera device. That is, when taking a picture by using a camera mounted on the portable terminal, a "click" sound is forcibly generated. This is to inform that the picture is taken around when the picture is taken using the mobile terminal.

However, even when a shooting sound is generated during the shooting as described above, when the surrounding environment is a noisy place, the person taking the picture may not recognize this. In addition, when the user using the compact camera device removes the generation of the photographing sound, the other party cannot detect that he or she is taking a picture.

In addition, as the Internet and the like have been developed, it is possible to infringe privacy without uploading a photo to be secretly taken as described above. Therefore, when taking a picture using a camera, the above problems can be solved by inserting information of the camera taking the picture and the user into the picture image.

Accordingly, an object of the present invention is to provide an apparatus and method for inserting information of a terminal into a captured image when taking a picture in a terminal having a camera.

Another object of the present invention is to insert the information of the terminal into the photographed image when taking a picture in a mobile terminal having a camera, the terminal receiving the photo image can extract the information inserted in the image to identify the photographer And providing a method.

It is still another object of the present invention to provide an apparatus and method for compressing and encoding an image taken when a picture is taken by a mobile terminal having a camera, and inserting information of a photographer into the compressed and encoded image stream.

Still another object of the present invention is to compress and encode a photographed image when taking a picture in a mobile terminal having a camera and insert the photographer's information into a compressed and encoded image stream, and the receiver extracts the photographer's information from the received image and then compresses it. The present invention provides a device and method for decoding and displaying the extracted image and checking the extracted photographer information when necessary.

It is still another object of the present invention to provide an apparatus and method for compressing and encoding an image taken when photographing a still image in a mobile terminal having a camera and inserting photographer information into an image stream of the still image.

It is still another object of the present invention to provide an apparatus and method for compressing and encoding an image taken when a picture is taken in a mobile terminal having a camera into a composite video of still images and inserting photographer information into an image stream of the synthesized video. Is in.

It is still another object of the present invention to provide an apparatus and method for compressing and encoding an image taken when a picture is taken by a video in a portable terminal having a camera and inserting photographer information into an image stream of the video.

DETAILED DESCRIPTION A detailed description of preferred embodiments of the present invention will now be described with reference to the accompanying drawings. It should be noted that the same components in the figures represent the same numerals wherever possible.

In the following description, specific details such as the format of a video signal, a transport packet format, a video compression scheme, etc. are shown to provide a more general understanding of the present invention. It will be apparent to one of ordinary skill in the art that the present invention may be readily practiced without these specific details and also by their modifications.

Hereinafter, the method will be described based on a method of inserting information of a photographer into a data format of an image to be encoded. However, information of the photographer may be inserted into the captured image. In addition, in the embodiment of the present invention will be described on the assumption that the camera is a portable terminal, the same may be applied to a general digital camera. In addition, in the embodiment of the present invention, a mobile terminal may use a terminal unique number (ESN) and user telephone numbers as the photographer information. And in the case of a camera may be a device unique number of the camera. Also, in addition to the information, a photographing time and a photographing place may be added.

1 is a view showing the configuration of a mobile terminal having a camera according to an embodiment of the present invention, which can be a configuration of a mobile telephone.

Referring to FIG. 1, the RF communication unit 121 performs a wireless communication function of the portable telephone. The RF communication unit 121 includes an RF transmitter for upconverting and amplifying a frequency of a transmitted signal, and an RF receiver for low noise amplifying and downconverting a received signal. The data processor 120 includes a transmitter for encoding and modulating the transmitted signal and a receiver for demodulating and decoding the received signal. That is, the data processor 120 may include a modem and a codec CODDEC. Here, the modem performs a function of modulating and demodulating signals transmitted and received, and various modems such as CDMA, GSM, UMTS, and the like may be used. The codec includes a data codec for processing transmission and reception data, and an audio codec for processing audio signals transmitted and received.

The key input unit 123 includes keys for inputting numeric and character information and function keys for setting various functions. The communication interface unit 125 is connected to an external device through a communication cable, and performs a serial communication function with the external device under the control of the controller.

The memory 130 may include a program memory, a data memory, and moving picture memories that store image signals generated and received according to an exemplary embodiment of the present invention. The program memory may store programs for controlling a general operation of the portable telephone and programs for controlling a path of an image signal applied to a display unit according to an exemplary embodiment of the present invention. In addition, the data memory temporarily stores data generated during the execution of the programs.

The controller 110 controls the overall operation of the portable telephone. In addition, the controller 110 may include the data processor 20. In addition, the controller 110 controls operations of capturing, storing, reproducing, transmitting, and receiving the video signal according to a mode command set by the key input unit 123 according to an exemplary embodiment of the present invention. In addition, the controller 110 performs a function of displaying user data on the display unit 160. The user data may include information displayed on a mobile terminal, user data for displaying a current time, reception sensitivity, battery levels, and user data displayed by a user's setting.

The camera module 140 includes a camera sensor for converting an optical signal sensed during image capturing into an electrical signal, and a signal processor for converting an analog image signal photographed from the camera sensor into digital data. Herein, it is assumed that the camera sensor is a CCD sensor, and the signal processor may be implemented by a digital signal processor (DSP). In addition, the camera sensor and the signal processor may be integrally implemented or may be separately implemented.

The image processor 150 performs a function of generating screen data for displaying an image signal output from the camera unit 140. The image processor 150 processes the image signal output from the camera unit 140 and outputs the frame image data according to the characteristics and size of the display unit 160. In addition, the image processor 150 includes an image codec 250, and the image codec 250 performs a function of coding an image signal in a set manner or decoding the coded frame image data into an original image signal. The image processor 150 performs a function of capturing and reproducing an image signal according to a mode set under the control of the controller 110. The image codec 250 may be a codec of a still image (hereinafter referred to as a jpeg), a composite video (hereinafter referred to as a motion jpeg), and a video (hereinafter referred to as a moving picture expert group (mpeg)).

The display unit 160 displays the frame image signal output from the image processor 50 on a screen, and displays user data output from the controller 110. In addition, the display unit 160 displays a video signal reproduced under the control of the controller 110. The display unit 160 may use an LCD. In this case, the display unit 160 may include an LCD controller, a memory capable of storing image data, and an LCD display device. In this case, when the LCD is implemented using a touch screen method, the LCD may operate as an input unit.

A GPS receiver 170 receives GPS information from a GPS satellite and transmits the GPS information to the controller 110. In this case, the GPS information may be information on which the mobile phone is currently located. According to an embodiment of the present invention, when the current image is captured, the GPS receiver may receive location information (location) and time information.

Referring to FIG. 1, when the cellular phone uses Code Division Multiple Access (CDMA) channels, the data processor 120 may be a CDMA modem. In addition, the data processor 120 includes an audio codec for coding and decoding an audio signal in a video signal according to an embodiment of the present invention as described above.

The mobile phone according to an embodiment of the present invention may perform an operation of displaying or transmitting a picture of a person or the surrounding environment on a video screen. First, the camera unit 140 may be mounted on a portable telephone or connected to a predetermined position outside. That is, the camera unit 140 may be an external camera or an internal camera. The camera unit 140 may include a sensor for capturing an image and a signal processor for converting an image signal photographed from the sensor into digital data. The sensor may be a CMOS sensor or a CCD (Charge Coupled Device) sensor. The image captured by the camera unit 140 is converted into an electrical signal by an internal sensor, and then the signal processor converts the image signal into digital image data. The converted digital image signal and the synchronization signal are output to the image processor 50. The synchronization signal may be a horizontal synchronization signal (Hsync) and a vertical synchronization signal (Vsync).

2 is a diagram illustrating an example of a configuration of the image processor 150 according to an embodiment of the present invention. The image processor 150 performs a function of coding and decoding an image signal input from the camera unit 140 while simultaneously interfacing an image signal between the camera unit 140 and the display unit 160. As described above, the image processor 150 includes an image codec 250 for coding and decoding an image signal. In an embodiment of the present invention, the image codec 250 may be a JPEG, a motion JPEG, an MPEG, or an H.264 codec. .

Referring to FIG. 2, the camera interface 211 performs a function of interfacing image data output from the camera 140. Here, it is assumed that the image data output from the camera 140 is image data in YUV format, and the display unit 160 displays image data in RGB format. In the embodiment of the present invention, it is assumed that the image data output from the camera 140 is YUV format, and the display unit 160 displays the image data in RGB format.

The scaler 213 scales the image signal photographed by the camera 140 to a size that can be displayed on the display unit 160 according to the control signal output from the controller 10. The scaler 213 reduces and crops the image signal pixels output from the camera 140 to the number of pixels of the image signal that can be displayed on the display unit 160. However, when the display unit 160 can display an image data having a size larger than that of the image data output from the camera 140, the scaler 213 enlarges the image signal output from the camera 140 under the control of the controller 10. Can be designed to be displayed. The enlarged display method may be selected by displaying the number of pixels that can be displayed on the display unit 140 from the image signal output from the camera 140.

The color converter 215 converts the YUV data output from the scaler 213 into RGB and outputs the converted RGB. When the image data photographed by the camera 140 is generated in the RGB format, or when the display unit 160 can display the image data in the YUV format, the configuration of the color converter 215 may be omitted.

The LCD interface 217 interfaces with the image data on the display unit 160. The LCD interface unit 217 may have a buffer therein to perform a function of buffering image data interfaced with the display unit 160 in the LCD interface 217.

The image codec 250 may perform a function of image coding or reconstructing the coded image data under the control of the image processing controller 200. The image codec 250 inputs a camera image signal output from the color converter 215 or an image signal displayed on the display unit 160 under the control of the image processing controller 200, and performs image coding using a set coding scheme. Coded image data is generated by inserting identification information. The image codec 250 extracts the identification information of the photographer from the image signal coded under the control of the image processing controller 200, decodes the coded image data, and outputs the decoded image data to the scaler 213 or the LCD interface 217. The image processing controller 200 may directly process key data input from the key input unit 123 during the image processing. According to an exemplary embodiment of the present invention, the controller 110 checks the key input generated by the key input unit 123, and transmits the key input related to the image processing to the image processing controller 200 when the key data related to the image processing is performed. It will be described assuming that the command is performed.

The image processing controller 200 controls the overall operation of the image processor 150. That is, the image processing controller 200 controls the display unit 160 under the control of the controller 110 to control the overall operation of the display of the image data photographed from the camera 140. In addition, the image processing controller 200 controls the image codec 250 to insert the identification information of the photographer into the image data or to extract the identification information of the photographer inserted into the image data. The method of inserting the photographer's identification information into the image data may include watermarking the photographer's identification information on the image data or watermarking the photographer's identification information on the stream of the image data. In the embodiment of the present invention, it will be described on the assumption that watermarking identification information of a photographer is included in the stream of the image data. In addition, the embodiment of the present invention will be described assuming an example of watermarking the photographer information on the coded video data stream after video coding the video data. However, after the photographer information is inserted before coding the image data, a method of coding the image data into which the photographer information is inserted may be used.

The image memory 230 stores or outputs image data under the control of the image processing controller 200. The image data stored in the image memory 230 may be the captured image data, and may also be the image data received from the mobile terminal.

The selector 219 selects data output from the image processor 150 or data output from the controller 110 based on the path control signal output from the controller 110 and outputs the data to the display 160.

Referring to FIG. 2, in the image capturing mode, the controller 110 passes control of image processing to the image processing controller 200. Then, the image processing controller 200 displays the user data transmitted from the controller 110 on the display unit 160, and displays the image data captured by the camera unit 140 on the display unit 160 and the image signal captured according to a user's command. Control the operation of storing the memory 130 in the memory 130. That is, the controller 10 controls an operation related to communication and controls the operation of the display unit 160 only when performing a communication function. In the standby state that does not perform the communication function, it is assumed that the overall operation control of the display unit 160 is performed by the image processing controller 200. That is, when the operation of the camera unit 140 is selected, the overall operation of the mobile terminal is controlled by the image processing controller 200.

Referring to the operation of transmitting the image data acquired by the camera 140 to the display unit 160, the image processor 150 processes the image signal captured by the camera 140 and displays it on the display unit 160. In this case, the scaler 213 of the image processor 150 may remove a portion of the pixel increment or select a portion of the pixels to display the number of pixels of the image data output from the camera 140 on the display unit 160. Can be. In this case, the captured video signal is not stored in the memory 230 and is displayed as a preview screen on the display unit 160. In the operation of displaying the image signal on the preview screen as described above, the image coding operation does not occur, and thus the operation of inserting the information of the photographer into the image signal to be photographed is also omitted.

In addition, in the process of displaying the video signal captured by the camera 140 on the video screen, the user may store the displayed video signal. That is, when a function related to image processing is selected on the function key or menu of the image processing of the key input unit 123, the user may store the image signal photographed by the camera unit 140. At this time, when a shooting command is generated, the image processing controller 200 controls the image codec 250 to code an image signal input through the camera interface unit 211 and inserts photographer information into the stream of the coded image data. After watermarking the photographer information, the information is stored in the image memory 230. In addition, when playing the stored image data or the received image data, the image processing controller 200 controls the image codec 250 to extract the photographer information watermarked on the image data. The image codec decodes the coded image data and outputs the decoded image data to the scaler 213 for display on the display unit 160. The image processing controller 200 may display the extracted photographer information on the display unit 160 through the selector 219.

3 is a diagram illustrating a configuration of the image codec 250. The image data is coded in a predetermined manner, and the photographer information is inserted into the coded image data as a watermark according to an embodiment of the present invention. The structure is shown. Although FIG. 3 illustrates the structure of the coder, the structure of the decoder operates in the reverse order of the coder and has a structure similar to that of the coder.

Referring to FIG. 3, the encoder 311 codes input image data according to a set method. In this case, the coding scheme is coded in a manner set according to the shape of the image data. That is, the image codec 250 may have different coding methods according to still images or moving images, and different coding methods may be used for still images or moving images. In the embodiment of the present invention, it is assumed that a JPEG coder is used for a still image and an MPEG4 or H.264 coder is used for a moving image. It is also possible to use a motion JPEG (motion JPEG) coder that processes a still image continuously as a moving image and also processes a composite moving image signal of audio and text in the moving image. The encoder 311 inputs raw image data to be photographed, and performs a function of compressing and encoding the image data by using a set coding scheme. The encoder 311 may be one of a JPEG coder, a motion JPEG coder, an MPEG4 coder, or an H.264 coder as described above. A codec adaptive header inserter 313 inserts a header into the coded image data stream. The header information may be header information according to a corresponding coding scheme, and the header information is inserted at a predetermined position of a coded video data stream.

The mixer 317 generates a watermark by mixing the mobile terminal information and the random sequence. The mixer 317 may use an exclusive OR gate. In this case, the portable terminal information may be photographer information, and may be an electronic serial number (ESN) and a mobile identification number (MIN). Herein, the ESN is a unique number of the mobile terminal, and is a code assigned independently to each mobile terminal. The MIN may be a phone number of the mobile terminal as information of the mobile terminal. In addition to the information of the mobile terminal as described above, information such as the time (Time) and the location where the image was taken may be included. In this case, the information about the photographing location may be obtained through the GPS receiver 170. If there is no GPS reception, information of a base station serving wireless communication of the current mobile terminal may be used. The random sequence may be a spreading code (PN sequence) used in a mobile terminal, and may also use a promised random sequence that can be commonly used in all mobile terminals. In addition, the random sequence may not be used. In this case, the mixer 317 may be omitted. In the mixer 317, the photographer information, which is mixed with a specific random sequence, is used as watermark information of the image data.

A watermark embedder 315 inserts the watermark output from the mixer 317 into the header and the coded image data stream. That is, the watermark inserter 315 inserts the watermark at a predetermined position of a video data stream composed of the header and coded video data. In this case, the watermark inserter 315 may watermark the photographer information on the image data by using the method of inserting the photographer information at a predetermined position of the coded image data stream. That is, when storing or transmitting the coded data, the coded image data is generated in a predetermined standard. Accordingly, the watermark inserter 315 inserts the photographer information into a predetermined position of the coded image data stream, and the position where the photographer information is inserted should be a predetermined position.

In general, there is a reserved area that is not used in the format of the coded video data stream. Therefore, the specific area of the spare area may be used as the watermarking area of the photographer information. An embodiment of the present invention will be described taking an example of using a specific position as a watermarking region in a stream of image data coded as described above. However, in addition to the above watermarking method, a method of directly watermarking the photographer information on the image data may also be used. The image data stream output from the watermark inserter 315 may be composed of a watermark (photographer information), header information, and a coded image data stream.

The output unit 319 inserts a start and end code into the stream output from the watermark inserter 315 and outputs a stream of coded image data. The start code is a sign indicating the start of a coded video data stream, and the end code is a sign indicating the end of a coded video data stream. Accordingly, the data stream output from the output unit 319 is composed of a start code, a watermark, a header, coded image data, and an end code. Here, the watermark inserter 315 may insert the photographer information after the start code, the next position of the header information, or the front end of the end code.

The image codec 250 having the above configuration generates image streams coded in different formats according to coding schemes. In this case, the image codec 250 may perform coding of still image data, and may also perform coding of video data. In the embodiment of the present invention, a video coding operation of JPEG, Motion JPEG, MPEG4, and H.264 methods will be described. Here, the JPEG coding method is a method of coding still images, the motion JPEG coding method is a method of synthesizing still images and coding them in a video form, and the MPEG4 and H.264 coding methods are methods for coding video data. An embodiment of the present invention according to each coding scheme as described above will be described with reference to FIGS. 4 to 8. 4 is a diagram illustrating a structure of a JPEG stream in which photographer information is watermarked according to an embodiment of the present invention, and FIG. 5 is a diagram of a structure of a JPEG 200 stream in which photographer information is watermarked according to an embodiment of the present invention. 6 is a diagram illustrating a structure of a motion JPEG stream in which photographer information is watermarked according to an exemplary embodiment of the present invention, and FIG. 7 is a diagram in which photographer information is watermarked according to an exemplary embodiment of the present invention. FIG. 8 is a diagram showing the structure of an MPEG4 stream, and FIG. 8 is a diagram showing the structure of an H.264 stream watermarked with photographer information according to an embodiment of the present invention.

First, referring to FIG. 4, the image codec 250 may be a JPEG image codec. The image data captured by the camera 140 is applied to the encoder 315 of the image codec 311. Then, the encoder 311 encodes the input image data by JPEG coding to generate ECS ₀ -ECS _last of 411 of FIG. 4. In this case, ECS ₀ -ECS _last of the 411 becomes JPEG coded image data. Thereafter, the JPEG coded image data is applied to the header inserter 313 to add header information as shown in 411. The header information may include a JFIF marker, a length, a JFIF string, a table / misc, a frame header, and a scan header. Here, the table / misc information has a structure as shown in 413 of FIG. 4, a frame header has a structure as shown in 415 of FIG. 4, and a scan header has a structure as shown in 417 of FIG. 4. .

Thereafter, the watermark inserter 315 inserts the photographer information in front of the header information as shown in 411 of FIG. In this case, the photographer information includes a marker for identifying the watermark, a length of a wordmark, and a watermark of X bytes. Here, the x-byte watermark is the photographer information, and may be ESN, MIN, photographing time and location, or some information thereof. Here, the ESN and the MIN are unique information of the mobile terminal and may be information for identifying the photographer.

After inserting the photographer information into the JPEG stream, the output unit 319 inserts the start code SOI and the end code to complete and output the JPEG coded image data stream. Then, the image processing controller 200 may store the JPEG stream, such as 411 of FIG. 4, output from the output unit 319 in the memory 230, and may also be transmitted to the outside by the controller 110 of the terminal.

Secondly, referring to FIG. 4, the image codec 250 may be a JPEG image codec. The image data captured by the camera 140 is applied to the encoder 315 of the image codec 311. Then, the encoder 311 encodes the input image data by JPEG coding to generate a bit stream of 451 of FIG. 5. Here, the bit stream of 451 becomes image data coded by JPEG 2000. Thereafter, the JPEG coded image data is applied to the header inserter 313 to add header information as shown in 451. The header information includes a main header marker segment, a plurality of tile part headers, and start of data (SOD). Therefore, as shown in 451 of FIG. 5, it can be seen that the JPEG2000 image data structure includes a main header, a plurality of tile part headers, SODs, and bitstreams. Here, the main header information has a structure as shown in 453 of FIG. 5, and a tile part header has a structure as shown in 455 of FIG. 4.

In an embodiment of the present invention, the information of the photographer is watermarked in a reserved region between the SOC and the main header. To this end, the watermark inserter 315 connected to the rear end of the header inserter 313 inserts photographer information in front of the main header information as shown in 451 of FIG. In this case, the photographer information includes a marker for identifying the watermark, a length of a wordmark, and a watermark of X bytes. Here, the x-byte watermark is the photographer information, and may be ESN, MIN, photographing time and location, or some information thereof. Here, the ESN and the MIN are unique information of the mobile terminal and may be information for identifying the photographer.

After inserting the photographer information into the JPEG stream, the output unit 319 inserts a start code SOC (start of codestream) at the start of the stream and an end code EOC (end of codestream) at the end of the stream in JPEG 2000 format. The coded video data stream is completed and output. Then, the image processing controller 200 may store the JPEG 2000 stream as shown in 451 of FIG. 4 output from the output unit 319 in the memory 230, and may also be transmitted to the outside by the controller 110 of the terminal.

Third, referring to FIG. 6, the video codec 250 may be a motion JPEG video codec. In this case, the motion JPEG image is composed of JPEG images and reconstructed into a video form. The motion JPEG image may also synthesize audio and / or text data. Therefore, the motion JPEG may be a composite video signal. In this case, as shown in 511 of FIG. 6, the synthesized moving image data has a structure in which text data is positioned, and then audio data having a frame size and image data are interlaced in units of frame sections. Then, a motion JPEG header is inserted in front of the composite video data as described above. Here, the motion JEPG header has a structure as shown in 513 of FIG. 6. The motion JPEG header includes a data structure constituting the motion JPEG as shown in 515 and header information consisting of a frame header and a scan header as shown in 517 of FIG. 6. The table / misc information has the same structure as 519 of FIG. 6, the frame header has the same structure as 521 of FIG. 6, and the scan header has the same structure as 523 of FIG. 6. .

The watermark inserter 315 inserts the photographer information into the motion JPEG header area as shown in 513 of FIG. In this case, the photographer information includes a marker for identifying the watermark, a length of a wordmark, and a watermark of X bytes. Here, the x-byte watermark is the photographer information, and may be ESN, MIN, photographing time and location, or some information thereof. Here, the ESN and the MIN are unique information of the mobile terminal and may be information for identifying the photographer.

After inserting the photographer information into the motion JPEG stream, as shown in 513 of FIG. 6, a start code SOI (start of image) is inserted in the front part of the motion JPEG header area, and an end code (end) in the end part of the motion JPEG header. of image) to complete the motion JPEG stream. Then, the image processing controller 200 may store the JPEG stream, such as 411 of FIG. 4, output from the output unit 319 in the memory 230, and may also be transmitted to the outside by the controller 110 of the terminal.

Thirdly, referring to FIG. 7, the video codec 250 may be an MPEG4 video codec. The MPEG4 stream has a structure as shown in 611 of FIG. In the MPEG4 stream having the structure as described above, a start code (Visual_object_sequence_start_code) is a code indicating the start of an MPEG4 stream, and a profile and level identifier (Profile_and_level_indication) are identifiers indicating attributes and sizes, and a watermark start code (Watermark_start_code) and water. The watermark indicates the start of the watermark and watermark information, and a plurality of visual objects are included, and an end code (Visual_object_sequence_end_code) is inserted at a position where the visual object ends.

Each visual object is composed of visual object information and a video object as shown in 613 of FIG. 7. The video object includes video object information and a plurality of video object layers (video_object_layers) as shown in 615 of FIG. 7. Each video object layer is composed of video object information and a video object group (group_of_videoobjectplane) as shown in 617, and the video object group has a configuration as shown in FIG.

7 illustrates an example in which photographer information, that is, a watermark, is placed in front of the visual object in the MPEG4 data stream. However, the watermark may be located after the start code or before the end code in the MPEG4 stream such as 611 of FIG. 7.

Fourth, referring to FIG. 8, the video codec 250 may be an H.264 video codec. The H.264 stream has a structure as shown in FIG. In the H.264 stream having the structure as shown in FIG. 8, a start code is a code indicating the start of the H.264 stream, and a NAL header (NAL header, NAL: Network Adaptation Layer) is 1 bit of forbidden_zero_bit, 2 It consists of Nal_ref_idc of bits and Nal_unit_type of 5 bits. The 5 bits of Nal_ref_idc are set to 0-12, respectively, as shown in FIG. 8, 13-23 are reserved areas, and 24-31 are unspecified areas. In an embodiment of the present invention, 13 of Nal_ref_idc is set as a watermark NAL header. Thus, the watermark NAL header becomes a watermark header. A watermark of X bytes is inserted after the watermark NAL header. Therefore, a water NAL header and a payload watermark in the H.264 stream are inserted into the H.264 stream. In FIG. 8, NAL headers and payload information following the watermark become existing structures of the H.264 stream.

When storing or transmitting image data photographed by the image codec 250 as described above, the image data captured by the image codec 250 is coded. At this time, when coding image data in the image codec 250, the information of the photographer is watermarked on the captured image data. To this end, the image codec 250 codes the photographed data first using a predetermined coding scheme, generates and inserts header information of a coded data stream into the coded data stream, and waters the photographer information into a stream including the header and data. It generates and inserts a mark, and inserts and stores or transmits a start code of a stream composed of the watermark, header, and data.

As described above, photographer information is inserted into a coded video data stream in the video codec. In this case, when the photographer information is used as a watermark, when the image data is decoded, the watermark may be analyzed to obtain information of the mobile terminal that photographed the photo, thus tracking the photographer. In this case, the decoding of the image data having the watermark is performed in the reverse order of the coding. 9 is a diagram illustrating a decoder configuration of an image codec 250 that decodes the coded data.

Referring to FIG. 9, when coded image data is received, the input unit 351 extracts a watermark, a header, and a stream of coded data by separating a start code and an end code from the received coded image data stream. . The watermark extractor 353 extracts a watermark (ie, photographer information) inserted at a predetermined position from the data stream including the watermark, the header, and the coded data. At this time, the watermark can be known by the watermark header information and extracts the watermark according to the watermark header information. The watermark extracted by the watermark extractor 353 is mixed with a random sequence in the mixer 359 to generate original photographer information. Here, the intermixer 359 may be an exclusive ora gaita, and the random sequence uses the same sequence as the random sequence used in FIG. 3. Therefore, the photographer information output from the mixer 359 is the photographer information inserted during coding, and may be ESN, MIN, time, and / or location information. The photographer information may be applied to the image processing controller 200 and displayed on the display unit 160.

In addition, the header and the stream of the coded data output from the watermark extractor 353 are applied to a header extractor 355, the header extractor 355 extracts the header information, and the decoder 357 is coded output from the header extractor 355. The data stream is decoded and generated as a raw image. Therefore, it can be seen that the decoder configuration of the image codec 250 operates in the reverse order of the operation of the coder.

In the embodiment of the present invention, it is assumed that the operation of watermarking the photographer information on the image data to be photographed is performed in the image codec. However, in the method of watermarking the photographer information on the image data, the information of the terminal may be inserted as a watermark when the camera 140 captures the image. In addition, in the embodiment of the present invention, when the photographer information is inserted into the image data watermark, the insertion of the photographer information as a payload at a predetermined position of the data stream is described as an example. However, in addition to the above method, a method of directly watermarking the photographer information on the captured image data may be used.

As described above, the terminal device having the camera inserts the information of the photographer into the watermark into the image data to be accurately captured, so that the photographing source can be accurately tracked. Currently, terminals attached to cameras are increasing and miniaturized, so that many people can be photographed as subjects in an undesired state, and the captured images can be quickly disseminated in the air through the Internet. However, the image taken as described above can not track the photographer. Therefore, when the photographer's information is inserted into the video taken during the photo shoot, the camera can be tracked during illegal distribution.

1 is a diagram showing the configuration of a portable terminal according to an embodiment of the present invention.

FIG. 2 is a diagram illustrating a configuration of an image processing unit in FIG. 1.

FIG. 3 is a diagram illustrating a configuration of an image codec in FIG. 2. FIG.

4 is a diagram illustrating a structure of a JPEG stream in which photographer identification information is inserted according to an embodiment of the present invention.

5 is a diagram illustrating a JPEG2000 stream structure in which photographer identification information is inserted according to an embodiment of the present invention.

FIG. 6 is a diagram illustrating a structure of a motion JPEG stream into which photographer identification information is inserted according to an embodiment of the present invention. FIG.

7 is a diagram showing the structure of an MPEG4 stream in which photographer identification information is inserted according to an embodiment of the present invention.

8 is a diagram illustrating a structure of an H.264 stream into which photographer identification information is inserted according to an embodiment of the present invention.

FIG. 9 is a diagram illustrating a configuration of an image codec in FIG. 2 and illustrates a configuration of extracting identification information of a photographer from image data including identification information of the photographer according to an exemplary embodiment of the present invention.

Claims (16)

In a device having a camera, An image codec for coding image data captured by the camera; A watermark insertion unit located in the image codec and inserting photographer information into the coded image data stream as a watermark; And a memory configured to store the coded video data stream having the watermark embedded therein. The apparatus of claim 1, further comprising a communication unit configured to transmit coded image data stored in the memory. The method of claim 2, wherein the communication unit And a communication interface unit connected to an external communication device through a communication cable to provide a communication interface function with the external communication device. The method of claim 2, wherein the communication unit, The apparatus characterized in that the RF communication unit for performing an RF communication with the base station. The apparatus as claimed in claim 1, wherein the photographer information is a unique number of a terminal. In the video data processing apparatus of a mobile terminal, Camera, An image processor configured to process image data photographed by the camera and to apply the photographed image data to the image codec in a recording mode; An encoder for encoding the image data output from the camera by a predetermined coding scheme, a header inserter for inserting header information into the coded image data stream, and a photographer to generate and insert watermarker information into the header information and the image data stream. The image codec comprising a watermark inserting unit, A memory for storing the coded video data stream having the watermark embedded therein in a recording mode; And a communication unit for transmitting coded image data selected from coded image data stored in the memory in a communication mode. In the device for displaying video data, Communication unit for receiving the coded image data in the communication mode, A watermark extractor for extracting a watermark inserted at a predetermined position from the coded image data stream output from the communication unit and generating the information of the photographer, and a header for extracting header information in the image data stream output from the watermark extractor. An image codec comprising an extractor and a decoder which decodes the coded image data stream output from the header extractor and restores the original image data; An image processor which processes image data output from the image codec; And the display unit comprises image data output from the image processing unit. The method of claim 7, wherein the communication unit, The apparatus characterized in that the RF communication unit for performing an RF communication with the base station. The apparatus as claimed in claim 8, wherein the photographer information is a unique number of a terminal. In the video data processing apparatus of a mobile terminal, Camera, An image processor configured to process an image coded and the number of images to be output from the camera and the image codec; A display unit which displays image data of the camera under the control of the image processor; A communication unit which transmits the coded image data in the image codec and outputs the received coded image data to the image codec in a communication mode; An encoder for encoding the image data output from the camera by a predetermined coding scheme, a header inserter for inserting header information into the coded image data stream, and a photographer to generate and insert watermarker information into the header information and the image data stream. A watermark extracting unit for outputting to the communication unit a watermark extracting unit for extracting a watermark inserted at a set position from a coded image data stream output from the communication unit and generating photographic information, and at the watermark extracting unit And a header extractor for extracting header information from the output image data stream, and a decoder for decoding the coded image data stream output from the header extractor, restoring the original image data, and outputting the original image data to the image processor. The device characterized in that consisting of a codec. In the video data processing method of a mobile terminal having a camera, Coding the image data photographed by the camera in a camera shooting mode using a set coding scheme; Generating header information of the coded video data stream and inserting the header information into the stream; Generating photographer information as a watermark, embedding the watermark in the header and the coded data stream; And storing the data stream having the watermark embedded therein. 12. The method as claimed in claim 11, wherein the generating of the watermark is performed by mixing photographer information into a predetermined random sequence. The method as claimed in claim 12, wherein the photographer information is ESN information of a mobile terminal. The method as claimed in claim 13, wherein a subscriber's telephone number is further provided in the photographer information. The method as claimed in claim 14, wherein the photographer information further includes the photographed location information. In the video data processing method of a mobile terminal having a camera, Extracting a watermark inserted at a predetermined position of the coded data stream in the image reproduction mode, detecting the photographer information from the extracted watermark; Extracting header information from the data stream from which the watermark is extracted; And decoding and displaying the data stream from which the header information is extracted using a predetermined decoding scheme.