KR20110083095A - Image processing apparatus for creating and playing image linked with multimedia contents and method for controlling the apparatus - Google Patents

Abstract

PURPOSE: A digital image processing apparatus and control method thereof are provided to generate an image which is connected with multimedia content such as a text file or a voice file. CONSTITUTION: A storing unit(14) stores an image file and multimedia content. A display unit displays an image. A user input unit(60) generates an input signal by receiving an input operation of a user. A content connecting unit(61) sets up multimedia content as a connected multimedia area. The content connecting unit generates link data which connects multimedia content.

Description

Image processing apparatus for creating and playing image linked with multimedia contents and method for controlling the apparatus}

Embodiments of the present invention relate to a digital image processing apparatus and a control method thereof, and more particularly, to a digital image processing apparatus capable of connecting and playing multimedia content to an image and a control method thereof.

With a digital image processing device such as a digital camera, you can save images created by taking images of the subject, play them back at any time, and transfer the images to other devices such as a computer or portable media player for viewing. Can be.

In recent years, with the spread of digital cameras and personal computers, even ordinary users with no professional skills or experiences can manipulate the captured images to edit them to their liking.

Also, due to the development of the Internet, the use of software for connecting multimedia contents to a part of an image is becoming more common. For example, software such as Shockwave flash ™ or WordProcess with hypertext can be used to link video, text or audio files to images taken with a digital camera. However, these tasks can be performed using software with special functions after transferring images to an external device such as a computer, and are not readily available to users who are not familiar with the software.

Since the playback function provided in the conventional digital image processing apparatus performs only a limited function of displaying only an image, it may not satisfy various needs of a user who wants to link various multimedia contents to an image.

SUMMARY OF THE INVENTION An object of the present invention is to provide a digital image processing apparatus having a function of generating or playing an image linked with multimedia content and a control method thereof.

Another object of the present invention is to provide a digital image processing apparatus capable of generating an image to which multimedia contents such as a moving image, a still image, a text or an audio file are connected, and a control method thereof.

Another object of the present invention is to provide a digital image processing apparatus and a control method thereof capable of playing an image and simultaneously playing multimedia contents connected to the image.

A digital image processing apparatus according to an aspect of the present invention includes an image file representing an image generated by a photographing unit, a storage unit for storing one or more multimedia contents, a display unit for displaying an image, and a user input operation. A user input unit for generating an input signal and a content connection unit for setting a partial region of an image as a multimedia region to which multimedia contents are connected based on the input signal of the user input unit, and generating link data connecting the multimedia region and the multimedia contents. .

In the present invention, the image file may include an image area having data representing an image and a header area having data representing an attribute of the image, wherein the link data includes a file name of the multimedia content and is to be stored in the header area. Can be.

In the present invention, the digital image processing apparatus may further include a content execution unit that executes the multimedia content connected to the multimedia area when the multimedia area is selected from the image displayed by the display unit based on an input signal of the user input unit.

In the present invention, the content execution unit may cause the guide information for guiding the multimedia content connected to the multimedia area to be displayed on the display unit together with the image.

In the present invention, the multimedia content may include any one file selected from the group consisting of a text file, an audio file, a video file, and an image file.

In the present invention, the digital image processing apparatus may further include a face recognizing unit for recognizing a face of a person included in the image and designating the face as a face area, and the content connection unit may include a guide cover for designating the face area as a multimedia area. Can be displayed on the display.

In the present invention, the content connection unit may separately generate a multimedia connection list including the link data.

A control method of a digital image processing apparatus according to an aspect of the present invention includes a display step of displaying an image, a reception step of receiving a user's input operation, and a partial region of the image based on a user's input operation. An area setting step of setting a multimedia area to which content is to be connected, and a content connection step of generating link data connecting the multimedia area and the multimedia content based on a user's input operation.

In the present invention, the control method may further include a playback step of receiving a user's input operation of selecting the multimedia area to which the multimedia content is connected and executing the multimedia content connected to the multimedia area.

The digital image processing apparatus and control method thereof according to the present invention as described above may generate an image connected to the multimedia content and play the multimedia content connected to the image.

1 is a block diagram schematically illustrating components of a digital image processing apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating steps of a control method of the digital image processing apparatus of FIG. 1.
3 is an operation example illustrating an operation of designating a multimedia area by the digital image processing apparatus of FIG. 1.
4 is a diagram illustrating an operation of connecting multimedia content by the digital image processing apparatus of FIG. 1.
FIG. 5 illustrates an operation of inputting a guide message for guiding multimedia content by the digital image processing apparatus of FIG. 1.
6 is a conceptual diagram illustrating an example of link data generated by the digital image processing apparatus of FIG. 1.
FIG. 7 is a conceptual diagram illustrating another example of link data generated by the digital image processing apparatus of FIG. 1.
8 is a flowchart illustrating steps of a control method of a digital image processing apparatus according to another embodiment of the present invention.
FIG. 9 is an example of an operation showing the execution of the image processing method of FIG. 8.

Hereinafter, the configuration and operation of the digital image processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating components of a digital image processing apparatus according to an embodiment of the present invention.

The digital image processing apparatus according to the embodiment illustrated in FIG. 1 includes a storage unit 14 storing image files and multimedia contents, a display unit 51 displaying images, and a user's input operation to generate an input signal. And a user input unit 60 and a content connection unit 61 for connecting the image and the multimedia content.

The digital image processing apparatus of FIG. 1 is an example implemented in the form of a digital camera. However, the present invention is not limited thereto and may be implemented by various types of electronic devices such as a digital camcorder or a portable multimedia player.

The digital image processing apparatus may include a photographing unit 3 for capturing an image. The imaging unit 3 includes an imaging device 20 for capturing an image of a subject and converting the image into an electrical signal, an image converter 41 for converting an electrical signal from the imaging device 20 to image data, and an imaging device 20. Imaging control unit 49 for controlling the operation of the " The image of the subject is incident on the imaging device 20 through the optical system 10 disposed in front of the imaging device 20.

The optical system 10 disposed in front of the image pickup device 20 includes a plurality of lenses 12 to perform the function of imaging external image light on the image pickup surface of the image pickup device 20. The lenses 12 are arranged to be variable in distance from each other. When the distance between the lenses 12 is changed, the zoom magnification and the focus of the optical system 10 may be adjusted.

The lenses 12 are driven by the zoom driver 11 having a driving means such as a zoom motor, so that positions thereof with respect to each other may be varied. The lenses 12 may include a zoom lens that enlarges or reduces the size of the subject and a focus lens that adjusts the focus of the subject. The zoom driver 11 is operated by receiving a control signal from the driving circuit unit 42 of the controller 40. Accordingly, the zoom driver 11 may drive the optical system 10 such that the optical system 10 has any one of a plurality of magnifications.

The imaging device 20 includes a photoelectric conversion device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and converts image light incident through the optical system 10 into an electrical signal. do. The imaging device 20 is driven by a control signal applied from the imaging control unit 49.

The image conversion unit 41 converts the electrical signal of the image pickup device 20 into image data that can be subjected to image processing or stored in a storage medium.

The process of converting the electrical signal of the image pickup device 20 by the image conversion unit 41 may include, for example, driving noise of the image pickup device 20 included in the electrical signal using a correlated double sampling circuit (CDS circuit). Reducing, adjusting the gain of the signal after noise reduction by an automatic gain control circuit (AGC circuit), converting an analog signal to a digital signal using an A / D converter, and pixel defects for the digital signal. It may include detailed steps such as signal processing such as correction, gain correction, white balance correction, gamma correction, and the like. The correlated double sampling circuit (CDS circuit), automatic gain control circuit (AGC circuit), A / D converter, and the like described herein may be configured as independent circuits.

The image conversion unit 41 converts the electrical signal of the image pickup device 20 into RGB data, and then converts the RGB data into raw data in the form of a YUV signal including a luminance (Y) signal and a color difference (UV) signal. can do. Hereinafter, the image data generated by the photographing unit 3 to represent an image of a subject is referred to as an 'image file'.

The controller 40 is electrically connected to the image pickup device 20, the zoom driver 11, the touch screen 50, the user input unit 60, the storage unit 14, the sound generator 71, and the like, respectively. In order to control the operation of the element, it performs functions such as exchanging control signals with these components, processing data, and the like.

The control unit 40 includes an image conversion unit 41, a driving circuit unit 42, a touch screen control unit 44, a storage control unit 43 that controls data storage for the storage unit 14, and an imaging control unit ( 49, a user interface 46, a face recognition unit 48, a content connection unit 61, a content execution unit 62, a sound control unit 47 and the like.

The controller 40 may be implemented as a microchip or a circuit board including the microchip, and each component included in the controller 40 may be implemented by software or circuits included in the controller 40. . The storage control unit 43 controls data recording to the storage unit 14 and reading of recorded data or setting information.

The storage unit 14 may include a volatile internal memory. The volatile built-in memory may be made of, for example, a semiconductor memory element such as SDRAM (synchronous DRAM). The storage unit 14 may perform a buffer memory function for temporarily storing the raw data generated by the image conversion unit 41 and a work memory function used for the data processing operation of the image conversion unit 41.

Alternatively, the storage unit 14 may include a nonvolatile external memory. The nonvolatile external memory may be, for example, a memory stick, flash memory such as SD / MMC, storage such as HDD or optical storage such as DVD or CD. The storage unit 14 may store an image file compressed and converted into a JPEG file, a TIFF file, or the like, and may store other multimedia contents. The multimedia contents stored in the storage unit 14 may include any one file of a text file, an audio file, a video file, and an image file.

The digital image processing apparatus includes a touch screen 50 having a display unit 51 for displaying an image represented by an image file stored in the storage unit 14 and an input unit 52 for selecting a portion of an image displayed on the display unit 51. It may be provided. The touch screen 50 performs a function of displaying an image captured by the imaging device 20 and a function of sensing a touch on a surface and generating a signal corresponding to the detected position.

The touch screen 50 is an input device that replaces a keyboard and a mouse. Since the touch screen 50 can be directly touched with a hand or a pen to perform a desired task, an intuitive task can be performed under a graphical user interface (GUI) environment. It is a device that lets you.

As the display unit 51 of the touch screen 50, a display device such as a liquid crystal display (LCD) or an organic light emitting diode (OLED) may be used. The display unit 51 may display an image indicated by the image file stored in the storage unit 14.

The input unit 52 may be installed on the surface of the display unit 51 to sense a touch of the surface, and is an example of a user input unit provided in the image processing apparatus of the present invention. The input unit 52 may use various types of technologies such as resistive sensing means, capacitive sensing means, surface acoustic wave use sensing means, infrared (IR) use sensing means, and optical use sensing means.

By touching the input unit 52 of the touch screen 50, the user may select and execute a menu from a portion of the image displayed on the display unit 51, or designate a multimedia area to connect the multimedia content to the displayed image.

The digital image processing apparatus may include a user input unit 60 installed separately from the touch screen 50. The user input unit 60 may be implemented in other forms, for example, a menu button or a dial. When the images are displayed on the display unit 51, the user can operate the user input unit 60 to set a multimedia area and to specify a file of multimedia content to be connected. The user input unit 60 recognizes a user's manipulation, generates an input signal, and applies the generated input signal to the user interface 46.

The content connection unit 61 sets a part of the image as a multimedia area to which the multimedia content is connected based on an input signal of the user input unit 60, and generates link data connecting the multimedia area and the multimedia content.

The content execution unit 62 determines that the multimedia area is selected from the image displayed by the display unit 51 based on the input signal of the user input unit 60, and executes the multimedia content. The content execution unit 62 may also perform a function of displaying guide information for guiding the contents of the multimedia content on the display unit 51 together with the image.

In FIG. 1, the content connection unit 61 and the content execution unit 62 are represented as independent components, but the present invention is not limited to such a configuration, and the content connection unit 61 and the content execution unit 62 are provided to the control unit 40. It may be implemented by software included in a circuit included or a semiconductor device.

The face recognition unit 48 detects face regions corresponding to faces of the person in the image and designates a face region including the detected face.

Various known methods can be used to detect the face of a person. For example, face recognition technology using geometric features of the face can be used. Face recognition using geometric features is a technique for recognizing faces of individual individuals using geometric factors such as the location, size and distance between facial feature points such as eyes, nose and mouth.

The content connection unit 61 may execute a function of displaying a guide cover for guiding a face area designated by the face recognition unit 48 as a multimedia area on the display unit 51.

2 is a flowchart illustrating steps of a control method of the digital image processing apparatus of FIG. 1.

The control method of the digital image processing apparatus according to the embodiment shown in FIG. 2 includes a display step (S110) of displaying an image, a reception step (S101) of receiving a user input operation, and an area setting step of setting a multimedia area. (S102) and content linking steps (S103 to S106) for generating link data for linking the multimedia content.

When the function of connecting the multimedia is executed (S100) and the display step (S110) of displaying an image is executed, the process enters the receiving step (S101) of waiting for receiving a user input. In the display step S110 of displaying an image, a user designates an image file stored in the storage unit 14 to display the image on the display unit 51. The user may select an image file to which the multimedia content is connected through the display step (S110).

When the user's input operation is received in the reception step S101, an area setting step S102 of selecting a partial area of the image and setting it as a multimedia area based on the input operation is executed. The multimedia area corresponds to an area to which multimedia content is connected.

 In the content linking steps S103 to S106, steps of generating link data linking the multimedia area and the multimedia content based on a user's input operation are executed. First, a step (S103) of selecting a type of multimedia to be connected to the set multimedia area is performed. According to the selected multimedia format, a step (S104) of selecting a file of multimedia content previously stored in the storage unit 14, or performing a voice recording or recording a video (S105) is executed.

After selecting the multimedia content, a step (S106) of inputting a guide message guiding the content of the multimedia content may be executed. Thereafter, step S107 of generating link data connecting the multimedia area of the image and the multimedia content is executed.

Since a plurality of multimedia areas can be set in one image, and multimedia content can be connected to each multimedia area, the step S108 of checking the intention of adding a multimedia area can be performed to repeat the steps of S101 to S107. You can exit.

3 is an operation example illustrating an operation of designating a multimedia area by the digital image processing apparatus of FIG. 1.

Since the image 100 is displayed on the touch screen 50, the user operates the touch screen 50 using a finger or the touch pen 110 or by operating the user input unit 60 implemented as a menu button. You can set the multimedia area.

When the image 100 is displayed on the touch screen 50, the guide signs 101, 102, 103 and the guide message 104 guide the areas including the faces recognized by the face recognition unit 48 to the multimedia area. Can be displayed. When the user selects one of the areas displayed on the guide signs 101, 102, and 103, the selected area is set as the multimedia area.

The present invention is not limited to the multimedia area setting method, and instead of selecting a part of areas including the face recognized by the face recognition unit 48 as the multimedia area, the user selects an arbitrary area of the image 100. You can also set the multimedia area by specifying.

4 is a diagram illustrating an operation of connecting multimedia content by the digital image processing apparatus of FIG. 1.

When the operation of selecting the multimedia area is completed through the operation shown in FIG. 3, the step of selecting a file of the multimedia content to be connected to the multimedia area may be performed as shown in FIG. 4. The touch screen 50 displays a guide message 201, a type selection area 202 for selecting a type of multimedia content, and an area 203 for specifying a file of multimedia content.

The multimedia content may select any one of a text file, an audio file, a video file, and an image file stored in the storage unit 14 of the digital image processing apparatus. In the type selection area 202, the user selects one of the menus of the picture, video, music, and text file of the first area 207, thereby storing an image file, a video file, or an audio file stored in the storage unit 14 in advance. You can link text files to the multimedia area.

In addition, the user may select one of the voice recording and the recording menu of the second area 208 in the type selection area 202 to record a voice to be connected to the multimedia area or to record a video.

FIG. 5 illustrates an operation of inputting a guide message for guiding multimedia content by the digital image processing apparatus of FIG. 1.

After designating the multimedia content to be connected to the multimedia area, a guide message for guiding the connected multimedia content may be input. The touch screen 50 displays a message 301 for guiding input of a guide message and an execution menu 302. When the user manipulates the keypad unit 304 using the touch pen 110, a guide message is input to the input window 303. The guidance message input by such a method may be provided to the user before executing the multimedia content connected to the multimedia area.

6 is a conceptual diagram illustrating an example of link data generated by the digital image processing apparatus of FIG. 1.

6 shows the data structure of the image file 400 "ABCD0001.JPG". The image file may include an image area 420 having data representing an image and a header area 410 having data representing an attribute of an image. The link data 401 may be stored in the header area 410.

The link data 401 includes area designation data 411 indicating the sequence number of the multimedia area, position data 412 indicating the location of the multimedia area, type data 413 indicating the type of the connected multimedia content, and connected multimedia content. It may include a file name 414 indicating a file of the, and a guide message 415 for guiding multimedia content.

The header area 410 may be, for example, a maker note area of Exif. Exif (Exchangeable image file format) is one of the storage formats for image files taken by digital cameras or digital camcorders. Exif is an image format standardized by JEIDA (Japan Electronic Industry Promotion Association). It is based on the general-purpose image formats TIFF and JPEG, and adds unique information and operating rules for digital cameras.

FIG. 7 is a conceptual diagram illustrating another example of link data generated by the digital image processing apparatus of FIG. 1.

The digital image processing apparatus includes a DCF list 500 having a file name 501 of an image file. An image captured by the digital image processing apparatus is recorded as an image file in a format such as JPG or TIF, and the file name follows the DCF (design rule for camera file system) standard.

DCF is an abbreviation of "Design rule for Camera File system" and is a file format proposed by the Japan Electronics and Information Technology Industries Association (JITA) as a standard. It describes how to store and use digital camera data. It is explicit. The advantage of the DCF file format is that pictures taken with digital cameras that conform to the DCF file format can be played on other digital camera devices.

An image processing apparatus conforming to the DCF scheme has a DCIM (Digital Camera Image) directory at a top level under a root directory. In the DCIM directory, a plurality of files having a total of eight digit file names combining four characters and four digit serial numbers that can be arbitrarily designated by a user can be recorded.

The content connection unit 61 of the digital image processing apparatus generates a multimedia connection list 600 including the image file, link data connecting the multimedia area and the multimedia content, separately from the DCF list 500.

The multimedia connection list 600 includes area designation data 602 indicating the sequence number of the multimedia area, location data 603 indicating the location of the multimedia area, type data 604 indicating the type of the connected multimedia content, and connected multimedia. It may include a file name 605 indicating a file of content, a guide message 606 for guiding multimedia content, and a file name 601 of a linked image file.

When the content execution unit 62 of the digital image processing apparatus executes the multimedia content connected to the multimedia area of the image file, the multimedia connection list 600 having this format may be referred to.

8 is a flowchart illustrating steps of a method for controlling a digital image processing apparatus according to another embodiment of the present invention, and FIG. 9 is an example of an operation showing how the image processing method of FIG. 8 is executed.

The control method of the digital image processing apparatus shown in FIG. 2 may further include a reproduction step as shown in FIG. 8. The playback step shown in FIG. 8 includes: receiving a user input while displaying an image (S200), selecting a multimedia area (S210), displaying guide information for guiding multimedia content (S220), and The method may include receiving an input of a user executing multimedia content (S230), and playing back the connected multimedia content (S240).

When the image 100 is displayed on the touch screen 50, box-shaped guide signs 701, 702, and 703 that surround the multimedia areas of the image 100 are displayed. Also, file names 701a, 702a, and 703a of multimedia content connected to the respective multimedia areas may be displayed below the guide signs 701, 702, and 703 as guide information.

When the user selects one of the multimedia areas of the image 100 using the touch pen 110, guide messages 705a and 705b may be displayed as guide information of the multimedia content connected to the selected multimedia area. In addition, the content execution unit 62 of FIG. 1 may execute multimedia content connected to the selected multimedia area.

The apparatus according to the present invention includes a processor, a memory for storing and executing program data, a permanent storage such as a disk drive, a communication port for communicating with an external device, a user interface such as a touch panel, a key, a button, and the like. Device and the like. Methods implemented by software modules or algorithms may be stored on a computer readable recording medium as computer readable codes or program instructions executable on the processor. Here, the computer-readable recording medium may be a magnetic storage medium such as a read-only memory (ROM), a random-access memory (RAM), a floppy disk, a hard disk, ), And a DVD (Digital Versatile Disc). The computer readable recording medium can be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. The medium is readable by a computer, stored in the memory, and can be executed by the processor.

All documents including publications, patent applications, patents, etc. cited in the present invention can be incorporated into the present invention in the same manner as each cited document individually and concretely, .

     Particular implementations described in the present invention are examples and do not limit the scope of the present invention in any way. For brevity of description, descriptions of conventional electronic configurations, control systems, software, and other functional aspects of the systems may be omitted. Also, the connections or connecting members of the lines between the components shown in the figures are illustrative of functional connections and / or physical or circuit connections, which may be replaced or additionally provided by a variety of functional connections, physical Connection, or circuit connections. In addition, unless specifically mentioned, such as "essential", "important" may not be a necessary component for the application of the present invention.

In the specification (particularly in the claims) of the present invention, the use of the term “above” and the similar indicating term may be used in the singular and the plural. In addition, in the present invention, when a range is described, it includes the invention to which the individual values belonging to the above range are applied (unless there is contradiction thereto), and each individual value constituting the above range is described in the detailed description of the invention The same. Finally, the steps may be performed in any suitable order, unless explicitly stated or contrary to the description of the steps constituting the method according to the invention. The present invention is not necessarily limited to the description sequence of the above steps. The use of all examples or exemplary language (e.g., etc.) in this invention is for the purpose of describing the invention in detail and is not to be construed as a limitation on the scope of the invention, It is not. Also, one of ordinary skill in the art appreciates that various modifications, combinations and changes can be made depending on design conditions and variables within the scope of the appended claims or equivalents thereof.

3: recording unit 62: content execution unit
10: optical system 71: sound generator
11: zoom drive 100: image
12: lenses 110: touch pen
14: storage unit 202: type selection area
20: imaging device 203: file designation area
40: controller 207: first region
41: image conversion unit 208: second area
42: drive circuit portion 301: message
43: storage control unit 302: execution menu
44: touch screen controller 303: input window
46: user interface 304: keypad section
47: sound control unit 400: image file
48: face recognition unit 401: link data
49: imaging controller 410: header area
50: touch screen 411, 602: area designation data
51: display unit 412, 603: position data
52: inputs 413, 604: type data
60: user input unit 420: image area
61: content connection part 500: DCF list
600: multimedia connection list
101, 102, 103, 701, 702, 703: Guidance signs
104, 201, 415, 606, 705a, 705b: Information message
414, 501, 601, 605, 701a, 702a, 703a: file name

Claims (11)

An image file representing an image generated by the photographing unit and a storage unit storing one or more multimedia contents;
A display unit which displays the image;
A user input unit configured to receive an input operation of a user and generate an input signal; And
And a content connection unit configured to set a partial area of the image as a multimedia area to which the multimedia content is connected based on the input signal of the user input unit, and generate link data connecting the multimedia area and the multimedia content. Image processing device. The method of claim 1,
The image file includes an image region having data representing the image, and a header region having data representing attributes of the image, wherein the link data includes a file name of the multimedia content and is stored in the header region, Digital image processing device. The method of claim 1,
And a content execution unit configured to execute the multimedia content connected to the multimedia area when the multimedia area is selected from the image displayed by the display unit based on the input signal of the user input unit. The method of claim 3,
And the content execution unit causes guide information for guiding the multimedia content connected to the multimedia area to be displayed on the display unit together with the image. The method of claim 4, wherein
The multimedia contents may include any one file selected from the group consisting of a text file, an audio file, a video file, and an image file. The method of claim 4, wherein
The apparatus may further include a face recognition unit configured to recognize a face of a person included in the image and designate it as a face area, and wherein the content connection unit may display a guide on the display unit to designate the face area as the multimedia area. Image processing device. The method according to claim 1 or 2,
The content connection unit separately generates a multimedia connection list including the link data. A display step of displaying an image;
A receiving step of receiving a user input operation;
An area setting step of selecting a partial area of the image based on an input operation of the user and setting the area as a multimedia area to which multimedia content is connected; And
And a content linking step of generating link data linking the multimedia area and the multimedia content based on an input operation of the user. The method of claim 8,
The image file representing the image has a header area having data representing attributes of the image, and in the content linking step, the link data includes a file name of the multimedia content and is stored in the header area. Control method of the device. The method of claim 8,
And receiving a user's input operation of selecting the multimedia area to which the multimedia content is connected, and executing the multimedia content connected to the multimedia area. The method of claim 8,
The content linking step separately generates a multimedia connection list including the link data.

Images