JP2011130208A - Imaging apparatus, apparatus and method for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a fuss-free image composition system without using a special apparatus. <P>SOLUTION: In an image composition system, image composition is carried out by automatically adding a composition image depending on the motion of an object immediately before photographing, without a special operation for image composition. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing system, and more particularly to an image taken by an imaging unit.

  2. Description of the Related Art Conventionally, when a photographed image taken with a digital camera or the like is printed with a printer, there is provided means for printing with a special composition effect added to the image according to the user's preference. For example, "Sharpness" that makes the outline of an image stand out, "Softness" that blurs the outline, "Cross filter" that gives the effect of applying a cross filter to the light source in the image, star marks and speech bubbles, etc. There is a “stamp” that pastes an image for composition into the image. It is also possible to prepare a stamp image for synthesis in advance and synthesize it with the photographed image.

  When adding a composite effect to a photographic image to be printed, the user uses an operation button, a touch pen, or the like to select a composite effect mode and, if necessary, select the degree, size, position, etc. of the composite effect.

  Japanese Patent Application Laid-Open No. 2004-151561 discloses means for automatically combining an image for synthesis with a light source position area by detecting the light source position as a specific area at the time of shooting.

JP 2004-297754 A

  In order to add a composition effect to an image to be printed, the user has to perform various settings by operating the printer, which is troublesome.

  According to Patent Document 1, since the image for synthesis is automatically synthesized at the light source position, it is possible to reduce the user's operation. However, it is necessary to prepare a light source in advance when photographing. Further, since the position of the light source is fixed, it is not possible to synthesize the synthesis data at the position desired by the user.

  An object of the present invention is to provide an imaging apparatus and an image processing apparatus that can add a synthesis effect to a position desired by a user without using a special device such as a light source or the like and without taking time and effort.

  In order to solve the above-described problem, an imaging apparatus according to the present invention includes an imaging unit that captures a captured image including a subject, a detection unit that detects a movement of the subject for a predetermined period, and a subject. And image processing means for performing image processing on the photographed image in accordance with the movement trajectory.

  The image processing apparatus of the present invention performs image processing on a captured image based on the acquired information, an acquisition unit that acquires motion information regarding the motion of a subject included in the captured image for a predetermined period, and the motion information. And an image processing unit, wherein the image processing unit determines a trajectory of the subject based on the motion information, and performs image processing on the captured image according to the trajectory of the subject.

  According to the present invention, it is possible to automatically add a synthesis effect according to the movement of the subject during a predetermined period at the time of shooting. Therefore, it is possible to provide an imaging apparatus and an image processing apparatus that can add a synthesis effect to a position desired by a user without performing a special operation for synthesis.

1 is a schematic diagram of an imaging apparatus and a printing apparatus according to the present invention. It is a block diagram of the configuration of the imaging apparatus. 1 is a block diagram illustrating a configuration of a printing apparatus. 3 shows a flowchart of an imaging process in the imaging apparatus according to the first embodiment. 2 shows a flowchart of a printing process in a printing apparatus. It is a schematic diagram which shows the example which synthesize | combines the image for a synthesis | combination with a picked-up image. It is a schematic diagram which shows the example which synthesize | combines the image for a synthesis | combination with a picked-up image. 9 shows a flowchart of an imaging process in an imaging apparatus according to a second embodiment. It is a schematic diagram which shows the example which synthesize | combines the image for a synthesis | combination with a picked-up image.

  Embodiments of the present invention will be described below with reference to the drawings.

<First Embodiment>
<Configuration of image processing system>
FIG. 1 is a schematic diagram of an imaging apparatus and a printing apparatus according to the group invention. In this embodiment, a printing apparatus will be described as an example of an image processing apparatus, but an image display apparatus, an image recording apparatus, or the like may be used.

Reference numeral 100 denotes an image pickup apparatus such as a digital camera that takes an image of a subject. The imaging apparatus 100 can shoot a subject and record subject movement information.
Captured image data and motion information data recorded by the imaging apparatus 100 are transferred to the printing apparatus 300 via the recording medium 200 such as a memory card or the data communication medium 400. The data communication medium 400 may perform wired communication as a physical cable, and the data communication medium 400 may perform wireless communication as a radio wave.

  The printing apparatus 300 that has acquired the data acquires the motion information recorded by the imaging apparatus 100, and prints the captured image data with a synthesis effect according to the motion information.

  A composite effect may be added to the captured image data according to the motion information on the imaging device side, and the image may be printed by the printing device.

<Configuration of imaging device>
FIG. 2 is a diagram illustrating a configuration of the imaging apparatus 100.

  Reference numeral 100 denotes an imaging device. Reference numeral 101 denotes a photographic lens, 102 a shutter having a diaphragm function, 103 an image sensor that converts an optical image into an electrical signal, and 104 an A / D converter that converts an analog signal output from the image sensor 103 into a digital signal. A timing generation circuit 105 supplies a clock signal and a control signal to the image sensor 103, the A / D converter 104, and the D / A converter 106, and is controlled by the memory control circuit 107 and the system control circuit 108. An image processing circuit 109 performs predetermined pixel interpolation processing and color conversion processing on the data from the A / D converter 104 or the data from the memory control circuit 107. The image processing circuit 109 performs predetermined calculation processing using the captured image data, and the system control circuit 108 controls the exposure control unit 110 and the distance measurement control unit 111 based on the obtained calculation result. I do. In the system control circuit 108, exposure control means 110, and distance measurement control means 111, TTL (through-the-lens) type autofocus processing (AF processing), automatic exposure processing (AE processing), flash pre-flash processing (EF) It is carried out. Further, the image processing circuit 109 performs predetermined calculation processing using the captured image data, and also performs TTL auto white balance processing (AWB processing) based on the obtained calculation result.

A memory control circuit 107 controls the A / D converter 104, the timing generation circuit 105, the image processing circuit 109, the image display memory 112, the D / A converter 106, the memory 113, and the compression / decompression circuit 114. The data of the A / D converter 104 is transferred to the image display memory 112 or the memory 113 via the image processing circuit 109 and the memory control circuit 107, or the data of the A / D converter 104 is directly transferred to the memory control circuit 107. Written.
Reference numeral 115 denotes an image display unit composed of a TFT LCD or the like. Display image data written in the image display memory 112 is displayed by the image display unit 115 via the D / A converter 106. If image data captured using the image display unit 115 is sequentially displayed, an electronic viewfinder function can be realized. The image display unit 115 can arbitrarily turn on / off the display according to an instruction from the system control circuit 108. When the display is turned off, the power consumption of the imaging apparatus 100 can be significantly reduced. it can. Further, the image display unit 115 is coupled to the main body of the image pickup apparatus 100 by a rotatable hinge unit, and can use an electronic finder function, a reproduction display function, and various display functions by setting a free orientation and angle. It is. Further, it is possible to store the display portion of the image display unit 115 toward the imaging apparatus 100, and in this case, the storage state can be detected by the image display unit open / close detection unit 116. Here, if the storage state is detected, the display operation of the image display unit 115 can be stopped and unnecessary power consumption can be prohibited.

  117 is a connector for outputting the output of the D / A converter 106 to an external monitor. When the connector is inserted into the external output connector 117, the system control circuit 108 can know the external output state by the external output connection detecting means 118. Here, if it is detected that the connection state is established, an external monitor can be used as a display device instead of the image display unit 115.

  Reference numeral 113 denotes a memory for storing captured still images and moving images, and has a sufficient storage capacity to store a predetermined number of still images and a predetermined time of moving images. This makes it possible to write a large amount of images to the memory 113 at high speed even in continuous shooting or panoramic shooting in which a plurality of still images are continuously shot. The memory 113 can also be used as a work area for the system control circuit 108. The memory 113 is also used as a write buffer for the recording medium 200.

  Reference numeral 114 denotes a compression / decompression circuit that compresses / decompresses image data by adaptive discrete cosine transform (ADCT) or the like, reads an image stored in the memory 113 and performs compression processing or decompression processing, and stores the processed data in the memory 113. Write to.

  An exposure control unit 110 controls the shutter 102 having an aperture function, and has a flash light control function in cooperation with the flash 119.

  Reference numeral 111 denotes a distance measuring control unit that controls focusing of the photographing lens 101, 120 denotes a zoom control unit that controls zooming of the photographing lens 101, and 121 denotes a barrier control unit that controls the operation of the protection unit 122 serving as a barrier.

  Reference numeral 119 denotes a flash which has an AF auxiliary light projecting function and a flash light control function.

  Reference numeral 108 denotes a system control circuit that controls the entire imaging apparatus 100, and reference numeral 123 denotes a memory that stores constants, variables, programs, and the like for operation of the system control circuit 108. The system control circuit reads a program from the memory 123, controls each unit based on the read program, and performs processing and operations described later.

  A liquid crystal display device 124 displays an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program by the system control circuit 108. In the liquid crystal display device 124, one or a plurality of locations are provided near the operation unit of the imaging device 100 so as to be easily visible, and are configured by a combination of, for example, an LCD, an LED, and a sounding element.

  Reference numeral 125 denotes an optical viewfinder, which can take an image using only the optical viewfinder without using the electronic viewfinder function of the image display unit 115.

  The display unit 124 is partially installed in the optical viewfinder 125. Among the display contents of the display unit 124, what is displayed on the LCD or the like includes single shot / continuous shooting display, self-timer display, compression rate display, number of recorded pixels, number of recorded pixels, number of remaining images that can be captured, shutter There are speed display, aperture value display, exposure compensation display and so on. In addition, flash display, red-eye reduction display, macro shooting display, buzzer setting display, clock battery level display, battery level display, error display, information display with multiple digits, display / removal status display of recording medium 200, communication I / F operation display, date / time display, etc. In addition, among the display contents of the display unit 124, what is displayed in the optical viewfinder 125 includes in-focus display, camera shake warning display, flash charge display, shutter speed display, aperture value display, exposure correction display, and the like.

  Reference numeral 126 denotes an electrically erasable / recordable nonvolatile memory such as a flash ROM.

  Reference numerals 127, 128, 129, and 130 are operation means for inputting various operation instructions of the system control circuit 108, and may be a single or a combination of a switch, a dial, a touch panel, pointing by gaze detection, a voice recognition device, or the like. Composed.

  Here, a specific description of these operating means will be given.

  Reference numeral 127 denotes a mode dial switch, which is used to switch and set each function mode such as power off, automatic shooting mode, shooting mode (including panorama shooting mode and moving image shooting mode), playback mode, multi-screen playback / erase mode, and PC connection mode. Can do.

  Reference numeral 128 denotes a shutter switch SW1 which is turned on when a shutter button (not shown) is being operated (half-pressed), and AF (auto focus) processing, AE (auto exposure) processing, AWB (auto white balance) processing, EF (flash pre-flash). ) Instruct the start of operations such as processing.

  Reference numeral 129 denotes a shutter switch SW2, which is turned on when the operation of a shutter button (not shown) is completed (fully pressed). When SW2 is turned on, an exposure process is performed in which a signal read from the image sensor 103 is written into the memory 113 via the A / D converter 104 and the memory control circuit 107. Further, image data is read from the memory 113 by the memory control circuit 107, the read image data is subjected to development processing by the image processing circuit 109, and then compressed by the compression / decompression circuit 114, and the image data is written to the recording medium 200. Perform the recording process. This series of photographing processing is executed by inputting ON of SW2.

  Reference numeral 130 denotes an operation unit including various buttons, a touch panel, etc., a menu button, a set button, a macro button, a multi-screen playback page break button, a flash setting button, a single shooting / continuous shooting / self-timer switching button, a menu movement + (plus) There are buttons. Further, there are a menu movement- (minus) button, a reproduction image movement + (plus) button, a reproduction image- (minus) button, a shooting image quality selection button, an exposure correction button, a date / time setting button, and the like.

  Reference numeral 131 denotes a power supply control means, which includes a battery detection circuit, a DC-DC converter, a switch circuit for switching a block to be energized, and the like. Here, the presence / absence of a battery, the type of battery, the remaining battery level are detected, the DC-DC converter is controlled based on the detection result and the instruction of the system control circuit 108, and the necessary voltage is recorded for a necessary period. Supply to each part including the medium.

  Reference numeral 132 denotes a connector, 133 denotes a connector, and 134 denotes a power supply means including a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery, or Li battery, or an AC adapter.

  135 is an interface with a recording medium such as a memory card or hard disk, 136 is a connector for connecting to a recording medium such as a memory card or hard disk, and 137 is a recording medium for detecting whether or not the recording medium 200 is attached to the connector 136. It is attachment / detachment detection means. In the present embodiment, it is assumed that the system has one interface and connector for attaching a recording medium. Of course, the interface and the connector for attaching the recording medium may have a single or a plurality of systems and any number of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. The interface and connector may be configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like that conforms to a standard. Further, when the interface 135 and the connector 136 are configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like, a communication card such as a LAN card or a modem card can be mounted. By connecting such a communication card, image data and management information attached to the image data can be transferred to and from other computers and peripheral devices such as a printer.

  Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 201 composed of a semiconductor memory, a magnetic disk, and the like, an interface 202 with the imaging device 100, and a connector 203 that connects to the imaging device 100. When the recording medium 200 is a PCMCIA standard PC-Card, CF card, or the like, an information storage circuit in which performance is recorded may be incorporated.

  Reference numeral 138 denotes acceleration detection means that can detect the moving direction and speed of the imaging apparatus 100. The acceleration detection unit 138 may be a unit that detects the posture of the imaging apparatus 100.

  A communication unit 139 has various communication functions such as RS232C, USB, IEEE1394, P1284, SCSI, modem, LAN, and wireless communication. Reference numeral 140 denotes a connector for connecting the imaging apparatus 100 to another device by the communication means 139 or an antenna in the case of wireless communication.

Reference numeral 141 denotes a microphone, which is an audio data acquisition unit. Reference numeral 142 denotes an A / D converter that performs A / D conversion so that the system control circuit 108 acquires audio data obtained by the microphone 141.
Reference numeral 143 denotes a speaker, which is an audio data reproducing means. Reference numeral 144 denotes a D / A converter that D / A converts digital audio data output from the system control circuit 108 so as to be reproduced by the speaker 143.

<Configuration of printing device>
FIG. 3 is a diagram illustrating a configuration of the printing apparatus 300.

  Reference numeral 300 denotes a printing apparatus. A timing generation circuit 305 supplies a clock signal and a control signal to the D / A converter 306, and is controlled by the memory control circuit 307 and the system control circuit 308. An image processing circuit 309 performs predetermined pixel interpolation processing and color conversion processing on data from the memory control circuit 307. A memory control circuit 307 controls the timing generation circuit 305, the image processing circuit 309, the image display memory 312, the D / A converter 306, the memory 313, and the compression / decompression circuit 314.

  Reference numeral 315 denotes an image display unit composed of a TFT LCD or the like, and display image data written in the image display memory 312 is displayed by the image display unit 315 via the D / A converter 306. The memory 313 can be used as a work area for the system control circuit 308. The memory 313 is also used as a write buffer for the recording medium 200.

  A compression / decompression circuit 314 compresses / decompresses image data by adaptive discrete cosine transform (ADCT) or the like, reads an image stored in the memory 313, performs compression processing or decompression processing, and stores the processed data in the memory 313. Write to.

  A system control circuit 308 controls the entire printing apparatus 300, and a memory 323 stores constants, variables, programs, and the like for operation of the system control circuit 308. The system control circuit 308 reads a program from the memory 323, controls each unit based on the read program, and performs processing and operations described later.

  Reference numeral 324 denotes a liquid crystal display device that displays an operation state, a message, and the like using characters, images, sounds, and the like in accordance with execution of a program in the system control circuit 308. The liquid crystal display device 324 is installed in a single or a plurality of locations near the operation unit of the printing apparatus 300 so that the liquid crystal display device 324 is easily visible, and is configured by a combination of, for example, an LCD, an LED, and a sounding element. Among the display contents of the display unit 324, what is displayed on the LCD or the like is the number display, the remaining battery level display, the error display, the recording medium attachment / detachment state display, the communication I / F operation display, the date / time display, and the printing. There is information.

  Reference numeral 326 denotes an electrically erasable / recordable nonvolatile memory such as a flash ROM. Here, a plurality of composite images (stamps, frames, speech balloons, etc.) to be combined with the photographed image are recorded. Select and synthesize. The selection of the composition image may be performed by the user or automatically by the system control circuit. An operation unit 330 includes various buttons and a touch panel. As the operation section, menu button, set button, multi-screen playback page break button, number of copies + (plus) button, number of copies-(minus) button, zoom + (plus) button, zoom-(minus) button, print button, etc. There is.

  Reference numeral 331 denotes power control means, which includes a battery detection circuit, a DC-DC converter, a switch circuit that switches a block to be energized, and the like. Here, the presence / absence of a battery, the type of battery, and the remaining battery level are detected, the DC-DC converter is controlled based on the detection result and an instruction from the system control circuit 308, and the necessary voltage is recorded for a necessary period. Supply to each part including the medium. 332 is a connector, 333 is a connector, 334 is a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, NiMH battery or Li battery, an AC adapter, or the like.

  335 is an interface with a recording medium such as a memory card or a hard disk, 336 is a connector for connecting to a recording medium such as a memory card or a hard disk, 337 is a recording medium for detecting whether or not the recording medium 200 is attached to the connector 336 It is attachment / detachment detection means. In the present embodiment, it is assumed that the system has one interface and connector for attaching a recording medium. Of course, the interface and the connector for attaching the recording medium may have a single or a plurality of systems and any number of systems. Moreover, it is good also as a structure provided with combining the interface and connector of a different standard. The interface and connector may be configured using a PCMCIA card, a CF (Compact Flash (registered trademark)) card, or the like that conforms to a standard. Further, when the interface 335 and the connector 336 are configured using a standard conforming to a PCMCIA card or a CF (Compact Flash (registered trademark)) card, a communication card such as a LAN card or a modem card can be connected. By connecting such a communication card, image data and management information attached to the image data can be transferred to and from other computers and peripheral devices such as a digital camera.

  Reference numeral 200 denotes a recording medium such as a memory card or a hard disk. The recording medium 200 includes a recording unit 201 composed of a semiconductor memory, a magnetic disk, and the like, an interface 202 with the printing apparatus 300, and a connector 203 that connects to the printing apparatus 300. When the recording medium 200 is a PCMCIA standard PC-Card, a CF card, or the like, an information storage circuit describing performance may be incorporated.

  A communication unit 339 has various communication functions such as RS232C, USB, IEEE 1394, P1284, SCSI, modem, LAN, and wireless communication. Reference numeral 340 denotes a connector for connecting the printing apparatus 300 to other devices by the communication means 339 or an antenna in the case of wireless communication. A recording paper conveyance control unit 350 conveys the recording paper 360 based on an instruction from the system control circuit 308. A print control unit 351 controls the ink cassette 361 based on an instruction from the system control circuit 308, and prints on the recording paper 360 conveyed by the recording paper conveyance unit 350.

<Detailed operation of image processing system>
Hereinafter, operations of the imaging apparatus and the printing apparatus according to the present invention will be described with reference to FIGS.

<Motion detection in imaging device>
FIG. 4 shows a flowchart of an imaging process in the imaging apparatus of this embodiment.

  When the imaging apparatus 100 starts the shooting mode, it is determined in S101 whether the shutter switch SW1 is ON (the shutter button is half-pressed). If the shutter switch SW1 is not ON, the process returns to S101. If the shutter switch SW1 is ON, the shooting preparation processing such as the AF, AE, AWB, and EF processing described above is performed in S102.

  When the shooting preparation process is completed, the position of the subject is periodically detected in S103, and the detected position information is recorded and stored in the memory 113. Here, accumulation of position information will be described. In the present embodiment, in order to automatically add various synthesis effects according to the movement of the subject immediately before photographing, it is necessary to record the movement information of the entire subject or a specific part for a certain period of time. The movement information mentioned here is information such as a movement trajectory (position change) and movement speed. In this embodiment, in order to calculate the movement locus and the movement speed, the position of the specific part of the subject is recorded in time series as coordinates. This is the accumulation of position information. For example, the position of the face of the subject may be recorded as coordinates by the image processing circuit 109, or the movement of the arm of the subject may be used. Further, the position of a part having a large position change may be recorded as coordinates. When detecting a change in position, the difference between the captured images is calculated, a part having a change in position is specified using the difference information, and the position is recorded. Further, position information of a plurality of parts may be recorded. Further, the movement of the subject may be detected by moving the focus position. In this case, information on the position of the focus area is recorded as position information. In this way, the position of the specific area of the subject may be accumulated, or the position of the entire subject may be accumulated. Any information that can analyze the movement of the subject later may be used.

  In S104, it is determined whether or not the shutter switch SW2 is ON (fully pressed). If the shutter switch SW2 is not ON, the process returns to S103 and the subject position information is accumulated again. The amount of information to be stored is set in advance, and when the amount exceeds the set amount, the oldest information is overwritten and stored in order. In addition, the time interval (cycle) for storing information is set in advance so as to have a sufficient amount of information when subject movement analysis described later is performed.

  When the shutter switch SW2 is ON (the shutter button is fully pressed) in S104, the above-described shooting operation is performed, and the shot image obtained by the shooting operation is recorded on the recording medium 200 (S105).

  When the photographing process is finished, the photographed image and the accumulated position information of the subject are recorded on the recording medium 200 (S106). The recording of the position information of the subject may be included in management information such as Exif information of the captured image, or may be saved as a separate file. Here, in the present embodiment, the accumulated position information of the subject is recorded on the recording medium 200 as motion information. However, the locus of movement of the subject and the moving speed may be calculated and recorded from the position information. In addition, when the position information is accumulated for a part with a position change, only the movement information of the part with the largest position change may be recorded, or the movement information of the part with a large position change near the face may be recorded. It may be recorded.

  When the recording of the captured image and the position information is finished, it is determined in S107 whether or not to end the shooting mode. If the shooting mode is not ended, the process returns to S101.

  FIG. 5 shows a flowchart of the printing process in the printing apparatus of this embodiment.

  When the printing apparatus 300 starts the printing mode, a captured image and position information are acquired in S201. Here, the data may be acquired by reading data recorded by the imaging device from the recording medium 200, or may be acquired from the imaging device 100 by wired or wireless communication using the communication units 139 and 339.

  When the data is acquired, the position information is analyzed in S202. Here, based on the accumulated position information, an analysis is made of how the subject has behaved during photographing. For example, if the accumulated position information is recorded with the movement of the arm of the subject as coordinates, the movement path and movement speed of the arm can be calculated. Similarly, when only the movement trajectory and moving speed of the subject are recorded instead of the accumulated position information, the data is acquired in S201, and the acquired data is analyzed in S202 to analyze the movement trajectory of the subject. Determine the moving speed.

  After analyzing the movement of the subject, in S203, a composite image is combined with the captured image and displayed on the image display unit 315. The type and position of the image to be combined are determined based on the subject motion information analyzed in S202. For example, if there is a movement of waving an arm on the subject, a plurality of star mark stamps may be pasted on the locus of movement of the arm, or a movement of drawing a circle around the face position. If there is, the balloon image may be pasted in the range where the arm or hand has moved. When combining speech bubbles, it is preferable to detect the position of the face and direct the beak (triangular cutout) to the face side. Alternatively, the balloons may be combined after rotating and reversing so that the beak comes to the movement start position and movement end position of the arm and hand. In this way, a composition image to be synthesized is selected from a plurality of composition images recorded in the non-volatile memory 326 of the printing apparatus in accordance with the movement trajectory pattern (circle, top / bottom, left / right pattern, etc.). It is good to choose. Furthermore, when a stamp image is synthesized as a synthesis image, the size or number of stamp images may be changed according to the moving speed. In this synthesis, the image processing circuit 309 may actually synthesize the synthesized image with the captured image data, or the synthesized image is superimposed on the image display unit 315 so that the user appears to be synthesized. You can just display it.

  After combining and displaying the combining image, a user operation by the operation unit 330 is accepted in S204. Here, the user can change the type and position of the image to be synthesized. For example, it is possible to add or delete a composition image, change the type, or adjust the position. When the change of the composition image is received, the process proceeds to S207, and the image is displayed on the image display unit 315 by reflecting the change.

  If it is determined in S204 that the composition of the image has not been changed but the start of the printing process has been instructed, the printing process is executed in S205. Note that in S203, when the composition image is not actually combined with the photographed image data, printing is performed after performing the process of adding the composition image within the print process.

  After the end of printing, it is determined whether or not the printing apparatus 300 ends the print mode in S206. If the print mode is not ended, the process returns to S201.

  FIG. 6 is a schematic diagram showing an example of adding a synthesis effect to the captured image of the present embodiment.

  As shown in FIG. 6, when the subject shows a movement that swings his arm before shooting, the shot image is only a stationary subject. However, the printing apparatus analyzes the movement information and automatically moves to the position where the movement has occurred. You can paste a star stamp and print it.

  FIG. 7 is a schematic diagram showing another example of adding a synthesis effect to the photographed image of this embodiment.

  As shown in FIG. 7, when the subject moves his / her arm so as to draw a circle at the time of photographing, the photographed image is only a stationary subject, but the printing apparatus analyzes the movement information and automatically moves to the position where the movement has occurred. A balloon image can be pasted and printed.

  In FIGS. 6 and 7, an image to be synthesized is selected in accordance with the pattern of hand movement of the subject. If the position information of the subject's hand is recorded in S103 and the position information is analyzed in S202, and it is determined that the hand has moved up and down as shown in FIG. 6, an image of the star is displayed on the locus of the hand movement. Synthesizing. Also, as shown in FIG. 7, when the hand moves in a circle, the blowout image is synthesized at the position where the circle is drawn in accordance with the size of the circle. In this way, since the image for synthesis is synthesized according to the movement of the hand immediately before shooting, the user can move the hand at the time of shooting without performing a troublesome operation for image synthesis at the time of printing. A desired image for synthesis can be automatically synthesized and printed.

  In FIGS. 6 and 7, the image to be synthesized is determined by hand movement. However, as described above, it may be determined that the portion with the largest position change is the hand position, or the position change is large and close to the face. Object movement information may be determined as hand movement information. Further, instead of hand movement information, a face movement or a focus position movement may be recorded, and a synthesis image may be synthesized based on these movements.

  In addition, the movement object (hand, face, focus position) and the accumulation of position information are associated and recorded as movement information, and a star is synthesized when the hand moves up and down, and a flower is synthesized when the face moves up and down. As described above, the image to be synthesized may be changed according to the moving object. Further, not only a composite image such as a stamp or a balloon, but also a frame image serving as a frame of the image may be combined around the entire image depending on a motion trajectory pattern.

  FIG. 9 is a diagram illustrating a synthesis result when the face position is accumulated as position information and recorded together with an image as motion information.

  Reference numeral 901 denotes an image displayed on the display unit 124 when the shutter switch SW1 is pressed. Here, in this embodiment, it is assumed that the position information holds information regarding the position of the face. Here, a square face frame 901-1 is displayed to indicate that the face of the subject on the left side of the image is recognized. Thereafter, the face position information is accumulated as described above until the shutter switch SW2 is pressed. Reference numeral 902 shows an image taken at S105. It can be seen that the face of the subject detected in 901 has moved to the center of the image. Reference numeral 901-2 represents a locus of a position where the face of the subject being detected has moved from the input of 901 SW1 to the input of SW2 in 902. In this way, the position information acquired from SW1 to SW2 is recorded as coordinate data together with the captured image as indicated by 903. In 903, information is divided by tags surrounded by <>. For example, <Product> describes the product name of the imaging apparatus 100, and <Image> describes the name of the image file that was captured at the same time. <PixelXDimension> is the maximum value of the x-axis coordinate of the image, and <PixelYDimension> is the maximum value of the y-axis coordinate of the image. Information on individual positions is described in <MoveData>, and time, x-axis coordinates, and y-axis coordinates are described by a space. The time is held every 100 milliseconds in milliseconds. Here, it can be seen that information on seven positions is held.

  From the motion information composed of the accumulated position information thus recorded, the printer analyzes the motion trajectory of the face area of the subject. Then, as in 904, a composition image is synthesized on the locus. Here, the print image is not added to the information at the last position so that the image for synthesis is not added to the face of the print image.

  Reference numeral 905 denotes an example in which a star is added on the moved locus. Reference numeral 906 denotes an example in which not only the star added in 905 but also the angle moved from the previous position is calculated, and the flow of motion is added and synthesized. Reference numeral 904 denotes an example in which a bee is added on the locus. Here, the direction and direction of the bees are determined by determining which direction the information is directed from the information before the position to be combined. That is, the image having the orientation is synthesized by rotating the composition image in accordance with the moving direction. When moving in the reverse direction, the composition image may be reversed and combined. Also, by setting the alpha value higher for older positions and increasing the transparency to make it transparent, it is possible to express how far before the time of shooting. In this embodiment, the compositing image is synthesized on the trajectory. However, in the case of a face, not only the face but also the body should move, so the compositing image may be synthesized slightly below the trajectory. .

<Second Embodiment>
In the above, the imaging process and the accumulation of position information are performed in the image capturing apparatus 100, and the process of analyzing the position information and adding the synthesis effect in the printing apparatus 300 has been described. However, these processes are performed only by the image capturing apparatus 100. You can also.

  Hereinafter, with reference to FIG. 8, an operation of the embodiment in which the processing up to adding a recording effect to a captured image and recording it is performed only by the imaging apparatus 100 will be described. Since the basic processing is the same as in the first embodiment, description thereof is omitted.

  FIG. 8 shows a flowchart of an imaging process in the imaging apparatus of this embodiment.

  When the imaging apparatus 100 starts the shooting mode, it is determined in S301 whether or not the shutter switch SW1 is ON. If the shutter switch SW1 is not ON, the process returns to S301.

  If the shutter switch SW1 is ON, the shooting preparation process described above is performed in S302.

  When the shooting preparation process is completed, the position information of the subject is accumulated in S303.

  Thereafter, in S304, it is determined whether or not the shutter switch SW2 is ON. If the shutter switch SW2 is not ON, the process returns to S303, and the subject position information is accumulated again.

  If the shutter switch SW2 is ON in S304, the photographing process described above is performed in S305.

  When the photographing process is finished, the position information accumulated so far is analyzed in S306, and the trajectory and moving speed of the subject are determined.

  After analyzing the motion of the subject, the composition image to be synthesized and the position to be synthesized are determined according to the analysis result in S306, and the composition image is displayed on the image display unit 115 for the photographed image in S307. To do. While being displayed, the synthesized image may be recorded on the recording medium 200.

  Thereafter, in S308, it is determined whether or not to end the shooting mode. If the shooting mode is not ended, the process returns to S301.

  The method for accumulating the position information of the subject, analyzing the position information, and synthesizing the image for synthesis in this embodiment are performed in the same manner as described above.

  According to the present embodiment, immediately after shooting, a combined effect is added to a captured image for display or recording, and processing up to combining can be performed only by the imaging apparatus 100.

<Other embodiments>
The embodiment of the present invention has been described above with reference to FIGS.

  As described above, according to the present embodiment, the composition effect can be automatically added according to the movement of the subject for a predetermined period at the time of shooting, so that the user's operation can be performed without performing a special operation for composition. An imaging apparatus and an image processing apparatus that can add a synthesis effect to a desired position can be provided.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included.

  Also, when a software program that realizes the functions of the above-described embodiments is supplied from a recording medium directly to a system or apparatus having a computer that can execute the program using wired / wireless communication, and the program is executed Are also included in the present invention. Accordingly, the program code itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention. In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

Claims (24)

Photographing means for photographing a photographed image including a subject;
Detecting means for detecting a movement of the subject for a predetermined period with respect to the subject to be photographed by the photographing means;
An image processing apparatus comprising: an image processing unit that performs image processing on the captured image in accordance with a movement trajectory of the subject.   The imaging apparatus according to claim 1, wherein the image processing unit synthesizes a synthesis image with the captured image in accordance with a trajectory of movement of the subject. Selecting means for selecting a composite image to be combined with the captured image from a plurality of composite images;
The imaging apparatus according to claim 2, wherein the image processing unit synthesizes the synthesis image selected by the selection unit with the captured image. Having storage means for storing a plurality of compositing images;
The image processing means selects a composite image to be combined with the photographed image from the plurality of composite images according to the movement trajectory of the subject, and combines the selected image with the photographed image. The imaging device according to claim 2.   The imaging device according to claim 1, wherein the detection unit detects a movement of a specific area of the subject.   The imaging apparatus according to claim 5, wherein the specific area is a face area of a subject.   The imaging apparatus according to claim 5, wherein the specific area is a focus area.   The imaging device according to claim 5, wherein the specific area is an area close to the face of the subject and having a large positional change. A shutter button for instructing photographing of a photographed image by the photographing means;
The imaging apparatus according to claim 1, wherein the predetermined period is a period from when the shutter button is half-pressed to when the shutter button is fully pressed. An operation unit for instructing photographing by the photographing unit;
9. The predetermined period is a period from when the preparation for photographing is instructed by the operation unit to when the photographing is instructed by the operation unit. Imaging device.   The imaging apparatus according to claim 10, wherein the photographing preparation includes at least one of an autofocus process, an automatic exposure process, an auto white balance process, and a flash pre-flash process.   12. The display device according to claim 1, further comprising a display unit configured to display an image subjected to image processing in accordance with a trajectory of movement of the subject after the image is captured by the image capturing unit. Imaging device. An acquisition means for acquiring a captured image and movement information relating to a movement of a subject included in the captured image for a predetermined period;
Image processing means for performing image processing on the captured image based on the motion information;
The image processing apparatus, wherein the image processing unit determines a trajectory of a subject based on the motion information, and performs image processing on the captured image according to the trajectory of the subject.   The image processing apparatus according to claim 13, wherein the image processing unit synthesizes a synthesis image with the captured image in accordance with a locus of movement of the subject. Selecting means for selecting a composite image to be combined with the captured image from a plurality of composite images;
The image processing apparatus according to claim 14, wherein the image processing unit combines the combination image selected by the selection unit with the captured image. Having storage means for storing a plurality of compositing images;
The image processing means selects a composite image to be combined with the photographed image from the plurality of composite images according to the movement trajectory of the subject, and combines the selected image with the photographed image. The image processing apparatus according to claim 14.   The image processing means determines a movement locus of a specific area of the subject based on the movement information, and performs image processing on the captured image according to the movement locus of the specific area of the subject. The image processing apparatus according to claim 13.   The image processing apparatus according to claim 17, wherein the specific area is a face area of a subject.   The image processing apparatus according to claim 17, wherein the specific area is a focus area.   The image processing apparatus according to claim 17, wherein the specific area is an area close to the face of the subject and having a large positional change.   21. The image processing apparatus according to claim 13, further comprising a printing unit that prints an image subjected to image processing by the image processing unit. An acquisition step of acquiring a captured image and movement information relating to movement of a subject included in the captured image for a predetermined period;
An image processing step of performing image processing on the captured image based on the motion information,
In the image processing step, a trajectory of a subject movement is determined based on the motion information, and the captured image is subjected to image processing according to the trajectory of the subject motion.   A program for executing the image processing method according to claim 22 on a computer.   A computer-readable recording medium on which the program according to claim 23 is recorded.

Images