JP2012124828A - Image processing device, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of composing a previously digital-watermarked image without damaging effectiveness of the digital watermark.SOLUTION: An imaging device as the image processing device extracts the digital watermark from a first image which is digital-watermarked, holds the extracted digital watermark, generates a composite image of a second image which is not digital-watermarked and the first image, and when the composite image generated by generation means becomes an image from which the digital watermark cannot be extracted, embeds the held digital watermark.

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program that can synthesize an image in which a digital watermark is embedded and another image.

  There is a digital watermark (Watermark) technology in which other information that is not perceived by human beings is embedded in digital image / sound data, and if necessary, only those who have valid qualifications and rights can extract the embedded information. Thereby, it is possible to enhance the evidence ability of the image or protect the copyright. As the digital watermark technique, there is a method of “invisible data embedding” as described in Patent Document 1 in which embedded data is not perceived by humans. In addition, by actively embedding copyright information and the like as described in Patent Document 2 in the original image in a perceptible state, it is expected to have the effect of discouraging unauthorized use of the image by a third party. There is a method called “data embedding”.

Japanese Patent Laid-Open No. 10-290359 JP-A-8-241403

  On the other hand, there is a photographing mode called multiple exposure imaging that involves combining multiple images. As one of the multiple exposure imaging methods, there is a method in which an image captured in advance is designated and the captured image is sequentially combined with this image. Conventionally, it has not been assumed that a digital watermark is used for a composite image obtained by such multiple exposure imaging.

  In view of the above problems, an object of the present invention is to provide an image processing apparatus, an image processing method, and a program capable of using a digital watermark for a composite image obtained by multiple exposure imaging.

  In order to achieve the above object, the image processing apparatus according to claim 1 includes an extraction unit that extracts the digital watermark from an image in which the digital watermark is embedded, and an extraction unit that extracts the first watermark from which the digital watermark is embedded. Holding means for holding the extracted digital watermark; generating means for combining the first image and at least one image different from the first image to generate a combined image; and combining generated by the generating means And an embedding unit that embeds the digital watermark held by the holding unit in the synthesized image when the image becomes an image that cannot be extracted by the extracting unit.

  According to the present invention, a digital watermark can be used for a composite image obtained by multiple exposure imaging.

1 is a diagram illustrating a schematic configuration of an imaging apparatus as an image processing apparatus according to an embodiment of the present invention. 2A and 2B are diagrams illustrating a multiple exposure imaging mode setting screen displayed on the image display unit in FIG. 1, in which FIG. 1A illustrates an imaging mode switching screen, FIG. 1B illustrates a multiple exposure imaging number screen, and FIG. The multiple mode selection screen in which the composite mode is selected is shown, and (D) shows the multiple mode selection screen in which the selected composite mode is selected. FIG. 2 is a flowchart showing a procedure of image processing executed by the system control circuit in FIG. 1, and shows a procedure of image processing in an imaging composition mode. FIG. FIG. 2 is a flowchart showing a procedure of image processing executed by the system control circuit in FIG. 1, and shows a procedure of image processing in a selective synthesis mode. FIG. FIG. 2 is a flowchart showing a procedure of image processing executed by the system control circuit in FIG. 1 and shows a procedure of image processing when a digital watermark is embedded in a selective synthesis mode. It is a figure which shows an example of the embedded information extracted. It is a figure which shows an example of a visible watermark, (A) shows the image containing a visible watermark, (B) shows the embedding information of the invisible watermark contained in an image. FIG. 2 is a flowchart showing a procedure of image processing executed by the system control circuit in FIG. 1, and shows a procedure of image processing when a visible watermark is embedded in a selective synthesis mode. 2 is a flowchart illustrating a procedure of captured image recording processing for recording all captured images executed by the system control circuit in FIG. 1.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  In the present embodiment, a process for making the digital watermark effective in the multiple exposure imaging mode involving the synthesis of a plurality of images is shown. As described above, the main methods of multiple exposure imaging are the case of sequentially synthesizing captured images, and designating images that have been captured in advance, and sequentially synthesizing the images captured for this image There is a case to do. Consider a case where a digital watermark technique is used for a composite image obtained by such multiple exposure imaging.

  In the former case, by embedding a digital watermark after synthesizing all the images, it is possible to enhance the image evidence capability or protect the copyright for the recorded images.

  Even in the latter case, if a digital watermark is not embedded in a pre-captured image, the recorded image is obtained by performing the same processing as the case of sequentially synthesizing the captured image. In contrast, it is possible to enhance the image evidence capability or to protect the copyright.

  However, in the case of an image in which digital watermark processing is performed on an image that has been captured in advance, there is a possibility that embedded digital watermark information cannot be normally extracted by the synthesis processing. However, if the digital watermark embedding process is performed again after the synthesizing process, there is a concern about the deterioration of the image.

  Further, when the embedded digital watermark is a visible watermark, the visible watermark itself becomes difficult to see by performing the synthesis process. As described above, when an image that has been subjected to digital watermark processing in advance is combined with another image, there is a problem that the effectiveness of the digital watermark is impaired.

  Therefore, in this embodiment, in the case of an image in which digital watermark processing has been performed on an image captured in advance, the digital watermark information is extracted, and the composite image loses the digital watermark information due to the synthesis However, processing for embedding the digital watermark into the composite image is performed.

  FIG. 1 is a diagram showing a schematic configuration of an imaging apparatus as an image processing apparatus according to an embodiment of the present invention.

  In the present embodiment, a single-lens reflex digital camera with interchangeable lenses will be described as an example of an imaging device. Note that this embodiment can also be applied to a digital video camera or the like in which lenses can be exchanged as an imaging device.

  The imaging apparatus 1000 includes a camera body 100 and an interchangeable lens type lens unit 300.

  The lens unit 300 includes an imaging lens 310, an aperture 312, a lens mount 306, an interface 320, a connector 322, an aperture controller 340, a focus controller 342, a zoom controller 344, and a lens system control circuit 350.

  The imaging lens 310 includes a plurality of lenses. The lens mount 306 mechanically couples the lens unit 300 to the camera body 100. The lens mount 306 includes various functions for electrically connecting the lens unit 300 to the camera body 100. The interface 320 is an interface for connecting the lens unit 300 to the camera body 100 in the lens mount 306. The connector 322 is a connector that electrically connects the lens unit 300 to the camera body 100. The connector 322 has a function of transmitting control signals, status signals, data signals, and the like between the camera body 100 and the lens unit 300 and supplying currents of various voltages. Further, the connector 322 may be configured to perform communication using not only electrical communication but also optical communication, voice communication, and the like.

  The aperture control unit 340 controls the aperture 312 based on the photometric information from the photometry control unit 46 in cooperation with the shutter control unit 40 that controls the shutter 12 of the camera body 100 described later. The focus control unit 342 controls focusing of the imaging lens 310. The zoom control unit 344 controls zooming of the imaging lens 310.

  The lens system control circuit 350 controls the entire lens unit 300. The lens system control circuit 350 includes a memory that records operation constants, variables, programs, and the like. Further, the lens system control circuit 350 is a non-volatile memory that holds identification information such as a number unique to the lens unit 300, management information, function information such as an open aperture value, minimum aperture value, and focal length, and current and past set values. The memory is also provided.

  Next, the configuration of the camera body 100 will be described. The camera body 100 includes a lens mount 106, mirrors 130 and 132, an optical viewfinder 104, a shutter 12, an image sensor 14, an A / D converter 16, a D / A converter 26, a timing generation circuit 18, an image processing circuit 20, and a memory. A control circuit 22, an image display unit 28, an image display memory 24, an image composition circuit 31, a memory 30, and a compression / decompression circuit 32 are provided.

  The lens mount 106 mechanically couples the camera body 100 and the lens unit 300. The mirrors 130 and 132 guide the light beam incident on the imaging lens 310 to the optical viewfinder 104 by a single-lens reflex system. The mirror 130 may be either a quick return mirror or a half mirror.

  The optical viewfinder 104 can display, as an optical image, light rays that are incident on the imaging lens 310 guided through the diaphragm 312, the lens mounts 306 and 106, and the mirrors 130 and 132 by a single-lens reflex system. Thereby, it is possible to perform imaging using only the optical viewfinder without using the electronic viewfinder function of the image display unit 28. Further, in the optical viewfinder 104, some functions of the notification unit 54, for example, in-focus state, camera shake warning, flash charging, shutter speed, aperture value, exposure correction, and the like are displayed.

  The shutter 12 is a focal plane type. The image sensor 14 is composed of a CCD, a CMOS sensor, or the like, and is an image sensor that photoelectrically converts a subject image. An optical element such as an optical low-pass filter is disposed in front of the image sensor 14. The light beam incident on the imaging lens 310 is guided by the single-lens reflex system through the aperture 312, the lens mounts 306 and 106, the mirror 130, and the shutter 12, which are light amount limiting means, and is formed on the image sensor 14 as an optical image. .

  The A / D converter 16 converts an analog signal (output signal) output from the image sensor 14 into a digital signal. The D / A converter 26 converts the image into an analog signal so that the display image written in the image display memory 24 is displayed by the image display unit 28.

  The timing generation circuit 18 supplies a clock signal and a control signal to the image sensor 14, the A / D converter 16, and the D / A converter 26, respectively, and is controlled by the memory control circuit 22 and the system control circuit 50.

  The image processing circuit 20 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 16 or the data from the memory control circuit 22. Further, the image processing circuit 20 performs a predetermined calculation process using the image output from the A / D converter 16 as necessary. TTL (through-the-lens) type autofocus (AF) processing and automatic exposure (AE) for the system control circuit 50 to control the shutter control unit 40 and the focus adjustment unit 42 based on the obtained calculation result. Processing and flash pre-emission (EF) processing can be performed. Further, the image processing circuit 20 performs predetermined arithmetic processing using the image output from the A / D converter 16, and also performs TTL auto white balance (AWB) processing based on the obtained arithmetic result. Yes.

  In the example shown in FIG. 3 in the present embodiment, the focus adjusting unit 42 and the photometric control unit 46 are provided exclusively. Therefore, the AF adjustment process, the AE process, and the EF process are performed using the focus adjustment unit 42 and the photometry control unit 46, and the AF process, the AE process, and the EF process using the image processing circuit 20 are not performed. It does not matter. Further, AF processing, AE processing, and EF processing are performed using the focus adjustment unit 42 and the photometry control unit 46, and further, AF processing, AE processing, and EF processing using the image processing circuit 20 are performed. It is good also as a structure.

  The memory control circuit 22 controls the A / D converter 16, the timing generation circuit 18, the image processing circuit 20, the image display memory 24, the D / A converter 26, the memory 30, the compression / decompression circuit 32, and the digital watermark circuit 33. . An image output from the A / D converter 16 is written into the image display memory 24 or the memory 30 through the image processing circuit 20, the memory control circuit 22, or only through the memory control circuit 22.

  The image display unit 28 is a TFT type LCD or the like, and the display image written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26. An electronic viewfinder (EVF) function can be realized by sequentially displaying captured images using the image display unit 28. Further, the image display unit 28 can arbitrarily turn on / off the display according to an instruction from the system control circuit 50, and when the display is turned off, the power consumption of the camera body 100 can be significantly reduced. it can.

  The image composition circuit 31 is a circuit that performs composition processing on a plurality of images. When the multiple exposure imaging mode is set by a normal imaging mode / multiple exposure imaging mode switching button included in the operation unit 70 described later, the image composition circuit 31 performs composition processing of the captured image recorded in the memory 30. .

  The digital watermark circuit 33 performs a digital watermark embedding process and a digital watermark extraction process. The digital watermark circuit 33 is a circuit that reads an image recorded in the memory 30, key information and embedding information generated by the system control circuit 50, and embeds a digital watermark at a predetermined position of the image. In addition, the digital watermark is extracted by reading an image in which the digital watermark is embedded and key information. Further, the digital watermark circuit 33 can extract a visible digital watermark and an invisible digital watermark, and can embed a visible digital watermark and an invisible digital watermark. In the following description, when it is simply expressed as a digital watermark, it indicates an invisible digital watermark, and when distinguishing between visible and invisible, it is expressed as a visible digital watermark and an invisible digital watermark, respectively.

  The memory 30 is a memory for recording still images or moving images, and has a recording capacity sufficient to record a predetermined number of still images or a predetermined amount of moving images. This makes it possible to write a large amount of images to the memory 30 at high speed even in continuous shooting or panoramic imaging in which a plurality of still images are continuously captured. In multiple exposure imaging, it is possible to hold a plurality of images to be combined. Further, when capturing a moving image, it is used as a frame buffer for images that are continuously written at a predetermined rate. Further, the memory 30 can be used as a work area for the system control circuit 50.

  The compression / decompression circuit 32 is a circuit that compresses / decompresses an image using a known compression method. The compression / decompression circuit 32 reads an image recorded in the memory 30 to perform compression processing or decompression processing, and writes the processed data to the memory 30 again. The compression / decompression circuit 32 also has a function of compressing and encoding a moving image into a predetermined format, or expanding a moving image signal from predetermined compression-encoded data.

  The camera body 100 further includes a shutter control unit 40, a focus adjustment unit 42, a photometry control unit 46, and a flash 48.

  The shutter control unit 40 controls the shutter 12 in cooperation with the aperture control unit 340 that controls the aperture 312 based on the photometry information from the photometry control unit 46. The focus adjustment unit 42 is a focus adjustment unit for performing AF (autofocus) processing. The focus adjustment unit 42 causes a single-lens reflex system to make a light beam incident on the imaging lens 310 in the lens unit 300 incident through a diaphragm 312, lens mounts 306 and 106, a mirror 130, and a focus adjustment sub-mirror (not shown). The in-focus state of the image formed as an optical image is measured. The AF control may be performed using the measurement result by the focus adjustment unit 42 and the calculation result obtained by calculating the image from the A / D converter 16 by the image processing circuit 20.

  The photometry control unit 46 performs AE (automatic exposure) processing. The photometry control unit 46 causes the light beam incident on the imaging lens 310 in the lens unit 300 to be incident on the single lens reflex system via the aperture 312, the lens mounts 306 and 106, the mirror 130, and a photometry sub mirror (not shown). Thus, the exposure state of the image formed as an optical image is measured. You may make it perform exposure control using the measurement result by the photometry control part 46, and the calculation result which computed the image from the A / D converter 16 with the image processing circuit 20. FIG.

  The flash 48 has an AF auxiliary light projecting function and a flash light control function. The photometry control unit 46 also has an EF (flash dimming) processing function in cooperation with the flash 48.

  The camera body 100 includes an interface 120 and a connector 122. The interface 120 is an interface for connecting the camera body 100 to the lens unit 300 in the lens mount 106. The connector 122 is a connector that electrically connects the camera body 100 to the lens unit 300. Further, whether or not the lens unit 300 is attached to the lens mount 106 and the connector 122 is detected by a lens attachment / detachment detection unit (not shown). The connector 122 transmits a control signal, a status signal, a data signal, and the like between the camera body 100 and the lens unit 300, and also has a function of supplying currents of various voltages. Further, the connector 122 may be configured to perform communication by optical communication and voice communication as well as electrical communication.

  The camera body 100 further includes a power switch 72, a power control unit 80, connectors 82 and 84, and a power source 86.

  The power switch 72 can switch and set the power on and power off modes of the camera body 100. Also, the power on and power off settings of various accessory devices such as the lens unit 300, the external flash 112, the recording medium 200, and the PC 210 connected to the camera body 100 can be switched.

  The power supply control unit 80 is configured by a battery detection circuit, a DC-DC converter, a switch circuit that switches blocks to be energized, and the like. The power supply control unit 80 detects the presence / absence of a battery, the type of battery, and the remaining battery level, controls the DC-DC converter based on the detection result and the instruction of the system control circuit 50, and requires the necessary voltage. It is supplied to each part including the recording medium for a period. The power source 86 is a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery, Li-ion battery, or Li polymer battery, or an AC adapter. The connectors 82 and 84 connect the power control unit 80 and the power source.

  The camera body 100 further includes a system control circuit 50, a memory 52, a notification unit 54, and a nonvolatile memory 56. The system control circuit 50 controls the entire camera body 100 and incorporates a known CPU or the like. The memory 52 is a memory that records constants, variables, programs, and the like for operation of the system control circuit 50.

  The notification unit 54 notifies the outside of an operation state, a message, and the like using characters, images, sounds, and the like in accordance with the execution of the program in the system control circuit 50. As the notification unit 54, for example, a display unit that performs visual display using an LCD, an LED, or the like, a sound generation element that performs notification by voice, and the like are used. The notification unit 54 is configured by a combination of one or more of these. . In particular, in the case of a display unit, it is installed at a single or a plurality of locations in the vicinity of an operation unit 70 (to be described later) of the camera body 100 that is easy to see. In addition, the notification unit 54 has a part of its functions installed in the optical viewfinder 104.

  Among the display contents of the notification unit 54, the contents displayed on the image display unit 28 such as an LCD include the following. First, there are contents related to the imaging mode such as single-shot / continuous-shot imaging display and self-timer display. Further, there are contents relating to recording such as compression rate display, recording pixel number display, recording number display, and remaining image pickup possible number display. In addition, there are contents relating to imaging conditions such as shutter speed display, aperture value display, exposure correction display, dimming correction display, external flash emission amount display, and red-eye reduction display. In addition, there are a macro imaging display, a buzzer setting display, a battery remaining amount display, an error display, an information display with a multi-digit number, and a recording medium 200 and PC 210 attachment / detachment state display. In addition, there are display of the attachment / detachment state of the lens unit 300, communication I / F operation display, date / time display, contents indicating the connection state with an external computer, and the like.

  Among the display contents of the notification unit 54, the contents displayed in the optical finder 104 include, for example, the following. In-focus display, imaging preparation completion display, camera shake warning display, flash charge display, flash charge completion display, shutter speed display, aperture value display, exposure correction display, recording medium writing operation display, and the like.

  The nonvolatile memory 56 is an electrically erasable / recordable memory in which a program and the like to be described later are recorded. For example, an EEPROM or the like is used.

  The camera body 100 also includes a mode dial 60, shutter switches 62 and 64, a playback switch 66, a single / continuous shooting switch 68, and an operation unit 70.

  These are operation means for inputting various operation instructions of the system control circuit 50, and are configured by a single or a combination of a switch, a dial, a touch panel, pointing by eye-gaze detection, a voice recognition device, or the like. Here, a specific description of these operating means will be given.

  The mode dial 60 can switch and set various function imaging modes such as an automatic imaging mode, a program imaging mode, a shutter speed priority imaging mode, an aperture priority imaging mode, a manual imaging mode, and a depth of focus priority (depth) imaging mode. In addition, the mode dial 60 can switch and set each function imaging mode such as a portrait imaging mode, a landscape imaging mode, a close-up imaging mode, a sports imaging mode, a night scene imaging mode, a panoramic imaging mode, and the like.

  The shutter switch 62 corresponds to SW1 and is turned on while a shutter button (not shown) is operated (for example, half-pressed), and instructs to start operations such as AF processing, AE processing, AWB processing, and EF processing.

  The shutter switch 64 corresponds to SW2 and is turned on when the operation of a shutter button (not shown) is completed (for example, fully pressed), and instructs the start of a series of processing operations including exposure processing, development processing, and recording processing. . First, in the exposure process, a signal read from the image sensor 14 is written to the memory 30 via the A / D converter 16 and the memory control circuit 22, and further, an operation in the image processing circuit 20 and the memory control circuit 22 is used. Development processing is performed. Further, in the recording process, an image is read from the memory 30, compressed by the compression / decompression circuit 32, and written to or transmitted to the recording medium 200 or the PC 210.

  The playback switch 66 instructs to start a playback operation for reading an image captured in the imaging mode state from the memory 30 or the recording medium 200 and the PC 210 and displaying the image on the image display unit 28. In addition, the playback switch 66 can set various function modes such as a playback mode, a multi-screen playback / erase mode, and a PC connection mode.

  The single-shot / continuous-shot switch 68, when the shutter switch 64 is pressed, takes a single frame and stands by, and continuous shooting continues while the shutter switch 64 is pressed. The copy mode can be set.

  The operation unit 70 includes various buttons and a touch panel. Examples of the various buttons include a normal imaging mode / multiple exposure imaging mode switching button, a multiple exposure imaging number setting button, a live view start / stop button, a menu button, a set button, a multi-screen playback page break button, a flash setting button, a single button. Including a copy / continuous shooting / self-timer switching button, menu shift + (plus) button, menu shift− (minus) button. Further, as examples of various buttons, a playback image shift + (plus) button, a playback image shift-(minus) button, an imaging quality selection button, an exposure correction button, a dimming correction button, an external flash emission amount setting button, a date / time Includes setting buttons. Each function of the plus button and the minus button is provided with a rotary dial switch, so that the user can select a numerical value and a function more easily.

  Further, there are an image display ON / OFF switch for setting ON / OFF of the image display unit 28 and a quick review ON / OFF switch for setting a quick review function for automatically reproducing an image captured immediately after imaging. Further, there is a compression mode switch that is a switch for selecting a compression rate for JPEG compression or for selecting a RAW mode in which a signal from an image sensor is directly digitized and recorded on a recording medium. In addition, there is an AF mode setting switch capable of setting a one-shot AF mode and a servo AF mode. In the one-shot AF mode, an autofocus operation is started when the shutter switch 62 is pressed, and once focused, the focused state is maintained. In the servo AF mode, the autofocus operation is continued while the shutter switch 62 is being pressed.

  The camera body 100 includes an accessory shoe 110 and an external flash 112.

  The accessory shoe 110 is an interface between the system control circuit 50 and the external flash 112.

  The camera body 100 further includes a recording medium attachment / detachment detection circuit 98, interfaces 90 and 94, and connectors 92 and 96. The interfaces 90 and 94 are interfaces with recording media such as memory cards and hard disks, and PCs. The connectors 92 and 96 are connectors for connecting to a recording medium such as a memory card or a hard disk or a PC. The recording medium attachment / detachment detection circuit 98 detects whether the recording medium 200 or the PC 210 is attached to the connectors 92 and / or 96.

  Although the present embodiment has been described as having two systems of interfaces and connectors for attaching the recording medium, the interface and connectors for attaching the recording medium may have a single or a plurality of systems. . Moreover, it is good also as a structure provided with combining the interface and connector of a different standard.

  As the interface and the connector, it is possible to configure using interfaces that comply with various recording medium standards. For example, a PCMCIA (Personal Computer Memory Card International Association) card, a CF (Compact Flash (registered trademark)) card, an SD card, or the like. When the interfaces 90 and 94 and the connectors 92 and 96 are configured using a standard conforming to a PCMCIA card or a CF card, various communication cards can be connected.

  Communication cards include LAN cards, modem cards, USB (Universal Serial Bus) cards, and IEEE (Institute of Electrical and Electronic Engineers) 1394 cards. In addition, there are P1284 card, SCSI (Small Computer System Interface) card, PHS, and the like. By connecting these various communication cards, it is possible to transfer images and management information attached to the images to and from other computers and peripheral devices such as printers.

  The recording medium 200 is a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 202 composed of a semiconductor memory, a magnetic disk, and the like, an interface 204 with the camera body 100, and a connector 206 that connects to the camera body 100.

  As the recording medium 200, a memory card such as a PCMCIA card or compact flash (registered trademark), a hard disk, or the like can be used. Of course, it may be composed of a micro DAT, a magneto-optical disk, an optical disk such as a CD-R or CD-RW, a phase change optical disk such as a DVD.

  The PC 210 includes a recording unit 212 composed of a magnetic disk (HD) or the like, an interface 214 with the camera body 100, and a connector 216 for connecting with the camera body 100. Examples of the interface 94 include USB and IEEE1394, but are not particularly limited.

  2 is a diagram showing a multiple exposure imaging mode setting screen displayed on the image display unit 28 in FIG. 1, (A) shows an imaging mode switching screen, (B) shows a multiple exposure imaging number screen, (C) shows a multiple mode selection screen in which the imaging combination mode is selected, and (D) shows a multiple mode selection screen in which the selection combination mode is selected.

  2A, the imaging mode switching screen is displayed by operating a normal imaging mode / multiple exposure imaging mode switching button included in the operation unit 70, and the user can select the normal imaging mode and the multiple exposure imaging mode. Can be switched.

  In FIG. 2B, the multiple exposure imaging number screen is displayed by operating the multiple exposure imaging number setting button included in the operation unit 70, and the user captures the number of images when the multiple exposure imaging is performed (see FIG. 2B). Then you can set 3).

  Further, by operating an imaging synthesis mode / selective synthesis mode switching button included in the operation unit 70, a switching menu is displayed on the image display unit 28, and the imaging synthesis mode and the selection synthesis mode can be switched.

  The “imaging composition mode” in FIG. 2C is a mode in which captured images are sequentially synthesized. Further, the “selective combination mode” in FIG. 2D is a mode in which an image recorded in advance on the recording medium 200 is selected, and images captured with respect to this image are sequentially combined.

  FIG. 3 is a flowchart showing a procedure of image processing executed by the system control circuit 50 in FIG. 1, and shows a procedure of image processing in the imaging composition mode.

  In FIG. 3, the “imaging synthesis mode” is selected by the user by operating the multiple exposure imaging number setting button and the imaging synthesis mode / selection synthesis mode switching button included in the operation unit 70 (step S <b> 200).

  When the shutter switch 62 is turned on, the photometry control unit 46 performs photometry calculation, and the focus adjustment unit 42 performs focus adjustment on the subject. When the shutter switch 64 is turned on, the image sensor 14 is driven to capture an image (step S201). At this time, the image is subjected to predetermined pixel interpolation processing and color conversion processing in the image processing circuit 20 and temporarily stored in the image buffer in the memory 30.

  Next, it is determined whether or not the image capturing in step S201 is the first image capturing in multiple exposure (step S202). If the image capturing is the first image capturing (YES in step S202), there is no image to be combined, and the process returns to step S201. Then, it waits for the user to turn on the shutter switch 64 again. If it is not the first imaging (NO in step S202), a composition process is performed (step S203). In step S203, the image composition circuit 31 performs composition processing on the two images recorded in the image buffer of the memory 30.

  Next, it is determined whether or not the set number of multiple exposure images has been reached (step S204). If it has been reached (YES in step S204), the process proceeds to step S205, and if it has not been reached (NO in step S204). Then, the processing after step S201 is executed.

  For example, as shown in FIG. 2B, when three images are set as the number of multiple exposure images, if the third image has been captured, the process proceeds to step S205. If the third image has not been captured, the process returns to step S201.

  In step S205, since the multiple exposure imaging and the composition process have been completed, the composition-processed image recorded in the memory 30 is read out, and the compression / decompression circuit 32 performs a compression process such as JPEG compression to obtain a predetermined file. It is formed into a format and written to the recording medium 200 via the interface 90.

  Further, the combined image recorded in the memory 30 is read again, converted into an image for display by the image processing circuit, and recorded in the image display memory 24. The display image written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26.

  FIG. 4 is a flowchart showing a procedure of image processing executed by the system control circuit 50 in FIG. 1, and shows a procedure of image processing in the selective synthesis mode.

  In FIG. 4, the “selective combination mode” is selected by the user by operating the multiple exposure imaging number setting button and the imaging combination mode / selection combination mode switching button included in the operation unit 70 (step S <b> 600).

  Next, a captured image to be combined with the captured image is selected by the user (step S601). In this case, it is assumed that an image captured in advance is recorded on the recording medium 200. When the user operates the playback switch 66, the image recorded on the recording medium 200 is read out via the interface 90 and recorded in the memory 30.

  The image recorded in the memory 30 is read again, converted into a display image by the image processing circuit, and recorded in the image display memory 24. The display image written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26. Thereafter, when the user operates an image moving button included in the operation unit 70, it is possible to select a captured image to be combined with the captured image.

  When the shutter switch 62 is turned on, the photometry control unit 46 performs photometry calculation, and the focus adjustment unit 42 performs focus adjustment on the subject. When the shutter switch 64 is turned on, the image sensor 14 is driven to capture an image (step S602). The image is subjected to predetermined pixel interpolation processing and color conversion processing in the image processing circuit 20 and temporarily stored in an image buffer in the memory 30.

  The two images recorded in the image buffer of the memory 30 are combined by the image combining circuit 31 (step S603). In the operation after the first imaging, the selected image and the captured image are combined. In the operation after the second and subsequent images are picked up, the image that has undergone the first combining process and the image picked up by the second sheet are combined. The same operation is sequentially repeated for the third and subsequent sheets.

  Next, it is determined whether or not the set number of multiple exposure images has been reached (step S604). If it has been reached (YES in step S604), the process proceeds to step S605, and if it has not been reached (NO in step S604). Then, the processing after step S602 is executed. For example, as shown in FIG. 2B, when three images are set as the number of multiple exposure images, if the third image has been captured, the process proceeds to step S605. If the third image has not been captured, the process returns to step S602.

  In step S605, since the multiple exposure imaging and the composition process have been completed, the composition-processed image recorded in the memory 30 is read out, and the compression / decompression circuit 32 performs a compression process such as JPEG compression to obtain a predetermined file. It is formed into a format and written to the recording medium 200 via the interface 90.

  Further, the combined image recorded in the memory 30 is read again, converted into an image for display by the image processing circuit, and recorded in the image display memory 24. The display image written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26.

  As shown in the processing of FIGS. 3 and 4 described above, the difference between the imaging synthesis mode and the selection synthesis mode is whether or not to combine with previously captured data.

  Next, digital watermark processing used in the present embodiment will be described. First, an image is divided into a plurality of blocks each having n pixels × m pixels (division processing). Next, the divided blocks are subjected to orthogonal transformation such as discrete cosine transformation (DCT transformation) to obtain an n × m frequency component matrix (orthogonal transformation processing).

  Prior to the embedding process of embedding information, an embedding position that indicates where to embed embedding information is determined by a random number in the frequency component matrix obtained by the orthogonal transformation process, and how much the value of the frequency component is changed is determined. A change amount to be shown is determined, and the embedding position and the change amount are acquired and stored as key information.

  When embedding the embedding information, it is not necessary to embed all in the frequency component matrix for one block, and it may be embedded across the frequency component matrices of a plurality of blocks. In that case, a block group having an appropriate contrast in the screen is selected.

  By selecting, for example, a low frequency part of a frequency component matrix as an embedding position, it is possible to embed so that it cannot be perceived by humans. Further, since the difference from the original value of the frequency component matrix can be changed by changing the change amount, it is possible to control the deterioration of the image quality.

  Embedding information is embedded (embedding processing) by changing the value of the frequency component matrix of each block described above based on the embedding position and amount of key information. Further, the frequency component matrix of each block in which the embedding information is embedded is subjected to inverse orthogonal transformation to obtain an image of a plurality of blocks of n pixels × m pixels (inverse orthogonal transformation processing). Finally, a plurality of blocks of images obtained by the inverse orthogonal transform process are connected to obtain a watermark image in which embedded information is embedded (reconstruction process).

  Next, the flow of processing when extracting embedded information from a watermark image will be described. First, a watermark image is divided into a plurality of blocks each having n pixels × m pixels (division process). Next, orthogonal division such as discrete cosine transformation (DCT transformation) is performed on each divided block to obtain an n × m frequency component matrix (orthogonal transformation processing). Further, the embedding position and the change amount are obtained from the key information used when embedding, and the embedding information is extracted from the frequency component matrix of each block (extraction processing).

  FIG. 5 is a flowchart showing a procedure of image processing executed by the system control circuit 50 in FIG. 1, and shows a procedure of image processing when a digital watermark is embedded in the selective synthesis mode.

  In FIG. 5, the image in which the digital watermark is embedded is the selected image (first image). Further, it is assumed that “multiple exposure mode” is set as the imaging mode, “3” is set as the number of multiple exposure imaging, and “selective combination mode” is set as the multiple mode. Therefore, a plurality of captured images (images different from the first image) obtained by multiple exposure imaging are combined with the selected image. Therefore, it is combined with at least one image different from the captured image.

  The captured image and the captured image to be combined are selected by the user (step S100). A captured image obtained by imaging in advance is recorded on the recording medium 200. An image selected by operating the playback switch 66 and recorded on the recording medium 200 is read out via the interface 90 and recorded in the memory 30.

  The image recorded in the memory 30 is read again, converted into a display image by the image processing circuit, and recorded in the image display memory 24. The display image written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26. Thereafter, when the user operates an image moving button included in the operation unit 70, it is possible to select a captured image to be combined with the captured image.

  Next, digital watermark extraction processing is performed on the image selected in step S100 (step S101) (extraction means). The image recorded in the memory 30 is read again, and the digital watermark circuit 33 performs digital watermark extraction processing. At this time, if a digital watermark is embedded, the embedded digital watermark is extracted. The extracted digital watermark is held in the memory 30 (step S102) (holding means). FIG. 6 shows an example of the extracted embedded information. When the embedded information is extracted, it may be displayed on the image display unit 28 that the image is an image with an embedded digital watermark.

  Next, when the shutter switch 62 is turned on, the photometry control unit 46 performs photometry calculation, and the focus adjustment unit 42 performs focus adjustment on the subject. When the shutter switch 64 is turned on, the image sensor 14 is driven to capture an image (step S103). The image is subjected to predetermined pixel interpolation processing and color conversion processing in the image processing circuit 20 and temporarily stored in an image buffer in the memory 30.

  Next, the two images recorded in the image buffer of the memory 30 are combined by the image combining circuit 31 to generate a combined image (Step S104) (generating unit). In the operation after the first imaging, the selected image and the captured image are combined. In the operation after the second and subsequent images are picked up, the image that has undergone the first combining process and the image picked up by the second sheet are combined. The same operation is sequentially repeated for the third and subsequent sheets. That is, every time a subject is imaged, an image obtained by imaging is sequentially combined with the selected image.

  Next, a digital watermark extraction process is performed on the composite image to determine whether or not a digital watermark has been extracted (step S105) (determination step). The image recorded in the memory 30 is read again, and the digital watermark circuit 33 performs digital watermark extraction processing. At this time, if embedded information is extracted, it is compared with the embedded information extracted in step S101. As a result of comparison, if they match, it is determined that a digital watermark has been extracted, and if they do not match, it is determined that a digital watermark has not been extracted.

  If it is determined in step S105 that a digital watermark has been extracted (YES in step S105), the process proceeds to step S106. When the digital watermark is not extracted (NO in step S105), a warning is generated by notifying the user that the embedded information has been lost using the image display unit 28 (step S110) (warning generating means), Proceed to step S106.

  Next, it is determined whether or not multiple exposure has been completed when the set number of multiple exposure images has been reached (step S106). When multiple exposure has been completed (YES in step S106), the process proceeds to step S107, where multiple exposure is performed. If the exposure has not ended (NO in step S106), the processes in and after step S103 are executed. For example, as shown in FIG. 2B, when three images are set as the number of multiple exposure images, if the third image has been captured, the process proceeds to step S107. If the third image has not been captured, the process returns to step S103.

  Next, it is determined whether or not a digital watermark is embedded in the composite image (step S107). In step S107, if the process proceeds to step S107 via step S110, since the originally embedded information is lost, it is determined to be embedded. On the other hand, if the process has not passed through step S110, since the information originally embedded in the image is retained, it is determined that the information is not embedded.

  If the result of determination in step S107 is that no digital watermark is to be embedded (NO in step S107), processing proceeds to step S109. As a result of the determination in step S107, when it is determined that the digital watermark is to be embedded (YES in step S107), the digital watermark is embedded in the image based on the embedded information held in step S102 (step S108) (embedding means). In step S <b> 108, the composite image recorded in the memory 30 is read out, and a digital watermark is embedded by the digital watermark circuit 33. The image embedded with the digital watermark is recorded in the memory 30 again.

  Next, the image recorded in the memory 30 is read out, compression processing such as JPEG compression is performed by the compression / decompression circuit 32, the image is formed into a predetermined file format, and the image is transferred to the recording medium 200 via the interface 90. Recording is performed (step S109) (recording means).

  Further, the combined image recorded in the memory 30 is read again, converted into an image for display by the image processing circuit, and recorded in the image display memory 24. The display image recorded in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26.

  According to the process of FIG. 5, when the generated composite image is an image from which a digital watermark cannot be extracted (NO in step S105), the digital watermark of the selected image held in the composite image is embedded (step S108). ). As a result, it is possible to synthesize an image that has been subjected to digital watermark processing in advance and another image without impairing the effectiveness of the digital watermark. In addition, since the digital watermark is embedded only once at the end, it is possible to prevent image deterioration due to unnecessary embedding processing. Further, according to the processing of FIG. 5, it is possible to enhance the evidence ability of the image to be recorded or protect the copyright.

  Further, according to the process of FIG. 5, when the generated composite image becomes an image from which a digital watermark cannot be extracted (NO in step S105), a warning is generated (step S110). It is possible to notify the disappearance. In step S109, the user can select whether to execute the digital watermark embedding process again before image recording, thereby preventing image deterioration due to the unintended embedding process. Is possible.

  In the processing of FIG. 5, the digital watermark is a non-visualized watermark. Next, an embodiment in the case where the digital watermark is a visible watermark will be described.

  The visible watermark based on the visualization information has a role of explicitly showing the copyright of the image creator to the user who uses the image. Therefore, it is not the intention of the image producer that the visible watermark becomes difficult to see in multiple exposure imaging.

  7A and 7B are diagrams illustrating an example of a visible watermark. FIG. 7A illustrates an image including a visible watermark, and FIG. 7B illustrates embedded information of an invisible watermark included in the image.

  In FIG. 7A, the visible watermark 901 is indicated by a rectangle having a height V and a width H, and the coordinates of one vertex of the rectangle are (X, Y). In addition, in FIG. 7B, as the invisible embedding information, the above-described embedding position (X, Y) of the visible watermark and the height V and the width H representing the size are embedded in addition to the photographer, the imaging date and time, and the camera ID. It is.

  FIG. 8 is a flowchart showing the procedure of image processing executed by the system control circuit in FIG. 1, and shows the procedure of image processing when a visible watermark is embedded in the selective synthesis mode.

  In FIG. 8, the image in which the visible watermark is embedded is the selected image. Further, it is assumed that “multiple exposure mode” is set as the imaging mode, “3” is set as the number of multiple exposure imaging, and “selective combination mode” is set as the multiple mode.

  Note that the basic procedure shown in the flowchart shown in FIG. 8 is the same as the procedure in the process shown in FIG.

  In FIG. 8, the user selects a captured image to be combined with the captured image (step S100). An image captured in advance is recorded on the recording medium 200. By operating the playback switch 66, the image recorded on the recording medium 200 is read out via the interface 90 and recorded in the memory 30.

  The image recorded in the memory 30 is read again, converted into a display image by the image processing circuit, and recorded in the image display memory 24. The display image written in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26. Thereafter, when the user operates an image moving button included in the operation unit 70, it is possible to select a captured image to be combined with the captured image.

  Next, digital watermark extraction processing is performed on the image selected in step S100 (step S101). The image recorded in the memory 30 is read again, and the digital watermark circuit 33 performs digital watermark extraction processing. At this time, if the digital watermark is embedded, the embedded information is extracted. The extracted embedded information is held in the memory 30.

  Next, it is determined whether or not a visible watermark has been extracted (step S1000). If not extracted (NO in step S1001), the process proceeds to step S102, and if extracted (YES in step S1001), the visible watermark is retained. (Step S1001), the process proceeds to Step S102. Preserving the visible watermark specifically means copying the visible watermark area.

  Next, when the shutter switch 62 is turned on, the photometry control unit 46 performs photometry calculation, and the focus adjustment unit 42 performs focus adjustment on the subject. When the shutter switch 64 is turned on, the image sensor 14 is driven to capture an image (step S102). The image is subjected to predetermined pixel interpolation processing and color conversion processing in the image processing circuit 20 and temporarily stored in an image buffer in the memory 30.

  Next, the two images recorded in the image buffer of the memory 30 are combined by the image combining circuit 31 (step S103). In the operation after the first imaging, the selected image and the captured image are combined. In the operation after the second and subsequent images are picked up, the image that has undergone the first combining process and the image picked up by the second sheet are combined. The same operation is sequentially repeated for the third and subsequent sheets.

  Next, a digital watermark extraction process is performed on the combined image to determine whether or not a digital watermark has been extracted (step S104). The image recorded in the memory 30 is read again, and the digital watermark circuit 33 performs digital watermark extraction processing. At this time, if embedded information is extracted, it is compared with the embedded information extracted in step S101. As a result of comparison, if they match, it is determined that a digital watermark has been extracted, and if they do not match, it is determined that a digital watermark has not been extracted.

  If it is extracted as a result of the determination in step S104 (YES in step S104), the process proceeds to step S106. If not extracted (NO in step S104), the image display unit 28 is used to notify the user that the embedded information has disappeared (step S110), and the process proceeds to step S106.

  Next, it is determined whether or not multiple exposure has been completed when the set number of multiple exposure images has been reached (step S106). When multiple exposure has been completed (YES in step S106), the process proceeds to step S107, where multiple exposure is performed. If the exposure has not ended (NO in step S106), the processes in and after step S102 are executed. For example, as shown in FIG. 2B, when three images are set as the number of multiple exposure images, if the third image has been captured, the process proceeds to step S107. If the third image has not been captured, the process returns to step S102.

  Next, it is determined whether or not the digital watermark is re-embedded in the composite image (step S107). In step S107, if the process proceeds to step S107 via step S110, since the originally embedded information is lost, it is determined to be embedded. On the other hand, if the process has not passed through step S110, since the information originally embedded in the image is retained, it is determined that the information is not embedded. Further, even when it is determined not to be embedded, the process branches depending on the presence or absence of a visible watermark.

  If the result of determination in step S107 is that re-embedding is determined (YES in step S107), a digital watermark is embedded in the image based on the embedded information held in step S102 (step S108), and the process proceeds to step S1002. In step S <b> 108, the composite image recorded in the memory 30 is read out, and a digital watermark is embedded by the digital watermark circuit 33.

  If it is determined in step S107 that re-embedding is not to be performed (NO in step S107), it is determined whether or not there is a visible watermark. If there is no visible watermark (NO in step S111), the process proceeds to step S109, If there is (YES in step S111), a visible watermark is embedded (step S1002), and the process proceeds to step S109. The image embedded with the digital watermark is recorded in the memory 30 again.

  Next, the image recorded in the memory 30 is read out, compression processing such as JPEG compression is performed by the compression / decompression circuit 32, the image is formed into a predetermined file format, and the image is transferred to the recording medium 200 via the interface 90. Recording is performed (step S109), and this process is terminated.

  Further, the combined image recorded in the memory 30 is read again, converted into an image for display by the image processing circuit, and recorded in the image display memory 24. The display image recorded in the image display memory 24 is displayed by the image display unit 28 via the D / A converter 26.

  According to the processing of FIG. 8, when the generated composite image is an image from which a digital watermark cannot be extracted (NO in step S104), the digital watermark held in the composite image is embedded (in the case of an invisible watermark). In step S108, in the case of a visible watermark, step S1002), it is possible to synthesize an image on which digital watermark processing has been performed in advance and another image without impairing the effectiveness of the digital watermark. In addition, since the digital watermark is embedded only once at the end, it is possible to prevent image deterioration due to unnecessary embedding processing.

  In the process of FIG. 8, control is performed so that the detected visible watermark is re-embedded last, but it is also possible to configure so that the visible watermark area is excluded from the synthesis target in the synthesis process in step S103. It is. In this case as well, it becomes possible not to impair the usefulness of the visible watermark.

  In the multiple exposure processing described with reference to FIG. 8, in step S109, the image after all the synthesis processing is performed is recorded, but there are cases where it is desired to separately record all the captured images.

  FIG. 9 is a flowchart showing the procedure of a captured image recording process for recording all captured images executed by the system control circuit 50 in FIG.

  The captured image recording process in FIG. 9 is a process inserted between step S102 in FIG. 8 and step S103 in FIG.

  Accordingly, when the image is captured in step S102, it is determined whether or not to embed a digital watermark in the captured image (step S1100). When not performed (NO in step S1100), the process proceeds to step S1102, and when it is performed. (YES in step S1100), the captured image recorded in the memory 30 is read, and the digital watermark circuit 33 embeds a digital watermark in the captured image (step S1101). The image embedded with the digital watermark is recorded in the memory 30 again.

  Next, the captured image recorded in the memory 30 is read out, and compression processing such as JPEG compression is performed by the compression / decompression circuit 32, and the image formed into a predetermined file format is sent to the recording medium 200 via the interface 90. (Step S1102). Then, it returns to step S102 in preparation for the next imaging.

  According to the process of FIG. 9, since the composite image in which the digital watermark is embedded by the digital watermark circuit 33 is further recorded (step S1102), the digital watermark is also embedded in the individually recorded image. It is possible to prevent image deterioration due to the image quality, enhance the evidence ability of the image recorded individually, or protect the copyright.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the recording medium on which the program is recorded constitute the present invention.

30, 52 Memory 31 Image composition circuit 33 Digital watermark circuit 50 System control circuit 100 Camera body 200 Recording medium 300 Lens unit 1000 Imaging device

Claims (7)

Extraction means for extracting the digital watermark from the image in which the digital watermark is embedded;
Holding means for holding the digital watermark extracted by the extraction means from the first image in which the digital watermark is embedded;
Generating means for combining the first image and at least one image different from the first image to generate a combined image;
Embedding means for embedding the digital watermark held by the holding means in the synthesized image when the synthesized image generated by the generating means becomes an image from which the digital watermark cannot be extracted by the extracting means. An image processing apparatus characterized by that. The extraction means can extract a visible digital watermark and an invisible digital watermark,
The image processing apparatus according to claim 1, wherein the embedding unit can embed a visible digital watermark and an invisible digital watermark.   3. The warning generation unit according to claim 1, further comprising a warning generation unit that generates a warning when the composite image generated by the generation unit becomes an image in which the digital watermark cannot be extracted by the extraction unit. Image processing device.   4. The image processing apparatus according to claim 1, further comprising a recording unit configured to record a composite image generated by the generation unit and embedded with the digital watermark. 5. Having imaging means for imaging a subject;
Each time the generation unit captures an image of the subject, the generation unit sequentially synthesizes the image obtained by the imaging with the first image,
The image processing apparatus according to claim 4, wherein the recording unit records a combined image that is sequentially combined by the generating unit and in which the digital watermark is embedded. An extraction step of extracting the digital watermark from the first image in which the digital watermark is embedded;
A holding step of holding the digital watermark extracted by the extraction step in a holding unit;
Generating the composite image by combining the first image and at least one image different from the first image;
A determination step of determining whether or not the digital watermark can be extracted from the composite image generated in the generation step;
An image processing method comprising: an embedding step of embedding the digital watermark held in the holding step in the synthesized image when the digital watermark cannot be extracted in the determination step. A program for causing a computer to execute an image processing method of an image processing apparatus,
The image processing method includes:
An extraction step of extracting the digital watermark from the first image in which the digital watermark is embedded;
A holding step of holding the digital watermark extracted by the extraction step in a holding unit;
Generating the composite image by combining the first image and at least one image different from the first image;
A determination step of determining whether or not the digital watermark can be extracted from the composite image generated in the generation step;
A program comprising: an embedding step of embedding the digital watermark held in the holding step in the synthesized image when the digital watermark cannot be extracted in the determination step.

Images