JP6318547B2 - Image processing apparatus, imaging apparatus, and image processing program - Google Patents

Description

  The present invention relates to an image processing device, an imaging device, and an image processing program.

  2. Description of the Related Art Conventionally, a digital camera that reproduces an image by detecting a behavior of a digital camera in a three-dimensional space and applying a more dynamic effect according to the detected behavior information is known (for example, Patent Documents). 1).

JP 2008-193333 A

  However, there is a problem in that an effect in accordance with the actual movement state of the subject cannot be expressed because an effect is applied based on the behavior information of the digital camera at the time of shooting.

The image processing apparatus according to claim 1, for determining whether there is a convergence point that converges to one point while maintaining the movement direction of the moving subject substantially the same direction based on the plurality of image data. And a plurality of straight lines extending radially behind the moving subject are generated as movement effect data, and the movement effect data is combined with the target image data when the determination unit determines that the convergence point exists. When image data with a movement effect is generated and the convergence point does not exist, a plurality of straight lines extending in the same direction along the movement direction behind the moving subject are generated as the movement effect data, and the movement effect data is And image processing means for generating the image data with movement effect by combining with the target image data.
An imaging apparatus according to a sixth aspect is the image processing apparatus according to any one of the first to fifth aspects, wherein a release button operated by a user and imaging for imaging a subject to generate image data. And the imaging means sequentially generates the image data while the release button is half-pressed by the user, and the determination means includes a plurality of the images generated by the imaging means. It is determined whether or not the convergence point exists based on the data, and the imaging unit generates the image data as the target image data in response to the release button being fully pressed by the user. An imaging apparatus characterized by that.
The image processing program according to claim 7 determines whether or not there is a convergence point that converges to one point while maintaining the movement direction of the moving subject substantially the same direction based on a plurality of image data. When it is determined that the convergence point exists by the processing and the determination processing, a plurality of straight lines extending radially behind the moving subject are generated as movement effect data, and the movement effect data is combined with the target image data. When image data with a movement effect is generated and the convergence point does not exist, a plurality of straight lines extending in the same direction along the movement direction behind the moving subject are generated as the movement effect data, and the movement effect data is An image processing program causing a computer to execute image processing for generating the image data with movement effect by combining with the target image data.

The block diagram explaining the principal part structure of the digital camera by embodiment of this invention Block diagram for explaining the functions of the image processing unit according to the embodiment Conceptual diagram explaining generation processing of image data with movement effect The figure which shows an example of image data with a movement effect Conceptual diagram explaining generation processing of image data with movement effect Conceptual diagram explaining generation processing of image data with movement effect Flowchart for explaining generation process of image data with movement effect of digital camera according to embodiment Flowchart for explaining generation process of image data with movement effect of digital camera according to embodiment Flowchart for explaining generation process of image data with movement effect of digital camera according to embodiment The figure which shows an example of the image data with a movement effect in a modification The figure which shows an example of the image data with a movement effect in a modification A block diagram for explaining a main configuration of an image processing apparatus according to a modification.

With reference to the drawings, an interchangeable lens digital camera will be described as an example of an imaging apparatus according to an embodiment of the present invention. Note that a lens-fixed digital camera is also included in one embodiment of the present invention.
FIG. 1 is a block diagram showing a main configuration of the digital camera 1. The digital camera 1 includes an interchangeable lens 200 and a camera body 100, and the interchangeable lens 200 is attached to the camera body 100 via a mount portion (not shown).

  The interchangeable lens 200 includes a photographing lens 201 and a lens control unit 202. The taking lens 201 includes various optical lens groups such as an imaging lens and a focus adjustment lens. In FIG. 1, the photographing lens 201 is represented by a single lens. The lens control unit 202 includes a CPU, a ROM, a RAM, and the like (not shown), and is an arithmetic circuit that executes various types of data processing based on a control program. The lens control unit 202 controls driving of the photographing lens 201 and the diaphragm (not shown) based on the result of the data processing, and is connected to the camera body 100 via the electrical contact 211 provided on the mount unit (not shown). In this state, state information and lens information are transmitted and camera information is received.

  The camera body 100 includes an image sensor 101, a control circuit 102, a buffer memory 103, an LCD drive circuit 104, a liquid crystal display 105, an operation unit 106, a memory card interface 107, and a microphone 108. The imaging element 101 is provided with imaging pixels such as a CCD and a CMOS in a two-dimensional manner (rows and columns), and is driven in accordance with the control of the control circuit 102 described later to capture a subject image input through the photographing lens 201. Imaging is performed, and an imaging signal obtained by imaging is output. On the imaging surface of the imaging device 101, color filters of R (red), G (green), and B (blue) are provided so as to correspond to the pixel positions, respectively. Since the image sensor 101 captures a subject image through the color filter, the image signal output from the image sensor 101 has color information of the RGB color system.

  The control circuit 102 includes a CPU, a ROM, a RAM, and the like (not shown), and is an arithmetic circuit that controls each component of the digital camera 1 and executes various data processes based on a control program. The control program is stored in a nonvolatile memory (not shown) in the control circuit 102. The control circuit 102 includes an image processing unit 112 as a function, and performs various image processing such as white balance adjustment and gamma correction on the imaging signal output from the imaging element 101 to generate image data. Predetermined compression processing is performed and recorded in a memory card 109 described later. The image processing unit 112 also displays display image data to be displayed on the liquid crystal display 105 described later based on the imaging signal output from the image sensor 101 and image data recorded on the memory card 109 described later. Generate. The details of the function of the image processing unit 112 will be described later.

  The buffer memory 103 temporarily stores data before image processing, after image processing, and during image processing, and also stores an image file before recording to the memory card 109, and stores an image file read from the memory card 109. Used to remember. The microphone 108 inputs sound during moving image shooting in the moving image mode, performs processing such as amplification, and outputs it as sound data to the control circuit 102.

  The LCD drive circuit 104 is a circuit that drives the liquid crystal display 105 based on a command from the control circuit 102. The liquid crystal display 105 displays an image corresponding to the display image data created by the image processing unit 112. The liquid crystal display 105 displays a menu screen for setting various operations of the digital camera 1.

  The memory card interface 107 is an interface to which the memory card 109 can be attached and detached. The memory card interface 107 is an interface circuit that writes an image file to the memory card 109 or reads an image file recorded on the memory card 109 based on the control of the control circuit 102. The memory card 109 is a semiconductor memory card such as a compact flash (registered trademark) or an SD card.

  The operation unit 106 includes various switches provided corresponding to various operation members operated by the user, and outputs an operation signal corresponding to the operation of the operation member to the control circuit 102. For example, the operation member determines a release button, a menu button for displaying a menu screen on the liquid crystal display 105, a cross key operated when selecting various settings, and a setting selected by the cross key. And a mode switching button for switching the operation of the digital camera 1 between a shooting mode including a moving image shooting mode and a still image shooting mode and a playback mode. Furthermore, a movement line indicating the movement state of the main subject on the image is drawn by using the operation unit 106 using a plurality of temporally continuous image data acquired before the still image data is generated. An image generation mode with a movement effect for generating image data with a movement effect is selectable. The operation of the digital camera 1 in the movement effect-added image generation mode will be described in detail below.

-Image generation mode with movement effect-
A case where the movement effect-added image generation mode is set by the operation of the operation unit 106 by the user will be described. In the image generation mode with a movement effect, the image processing unit 112 uses a predetermined number of time-sequential image data acquired from when the release button is pressed halfway down by the user until it is fully pressed, that is, by pre-capture. Using the acquired image data, movement information such as a movement amount and a movement direction of the moving main subject (moving subject) is detected. Then, the image processing unit 112 uses the movement information regarding the detected moving subject to move the drawing of a moving line indicating the moving state of the moving subject in the still image data generated in response to the user fully pressing the release button. The effect data is synthesized to generate image data with a movement effect.

  First, the image data acquisition operation in the movement effect-added image generation mode will be described. When an operation signal is output from the operation unit 106 in response to a half-press operation of the release button by the user, the control circuit 102 causes the image sensor 101 to output an image signal at a predetermined cycle (for example, 60 fps). The image processing unit 112 generates image data of one frame using an imaging signal output every predetermined cycle, and sequentially stores the image data in the buffer memory 103. In the present embodiment, the image processing unit 112 generates one frame of moving image data with a resolution of 1920 × 1080 pixels, for example.

  When the above processing is repeated and an operation signal is output from the operation unit 106 in response to the user fully pressing the release button, the image processing unit 112 outputs the imaging signal output at this timing. To generate still image data. In the present embodiment, the image processing unit 112 generates still image data with a resolution of, for example, 3840 × 2160 pixels. Note that the values of the resolution, the number of frames, and the frame rate described above are examples, and are not limited to these values.

  FIG. 2 shows a function of the image processing unit 112 for generating image data with a movement effect using a plurality of image data (hereinafter referred to as frame image data) generated by the pre-capture and still image data. It is a block diagram for demonstrating. As shown in FIG. 2, the image processing unit 112 includes an image selection unit 120, a moving subject detection unit 121, a movement information detection unit 122, a drawing effect selection unit 124, and a processing unit 125 as functions. . The image selection unit 120 selects this still image data as target image data. The moving subject detection unit 121 detects a moving subject using a known technique for each of the frame image data.

The movement information detection unit 122 detects the moving direction of the moving subject by calculating the movement vector by comparing the positions on the image of the moving subject detected from the frame image data. In addition, the moving direction detection unit 122 detects a convergence point of the moving direction of the moving subject based on the moving direction of the moving subject and a comparison result obtained by comparing the sizes of the detected moving subjects on the image. . The moving direction detection unit 122 determines that a convergence point exists when the moving subject satisfies the following conditions (1) and (2), and specifies the position of the convergence point on the still image data.
(1) When the moving direction of the moving subject has a direction greater than a predetermined angle with respect to the horizontal direction of the image, and the moving direction is substantially constant between the frame image data.
(2) The size of the moving subject becomes smaller (the moving subject moves away from the digital camera 1) or becomes larger (the moving subject moves closer to the digital camera 1). If.

  The drawing effect selection unit 124 determines the shape of the movement line, the number of movement lines, the movement line corresponding to the movement effect data to be generated based on the presence / absence of the convergence point and movement information such as the movement direction and movement amount regarding the moving subject. Select and determine the movement effect such as the length. The processing unit 125 generates trajectory data according to the movement effect selected by the drawing effect selection unit 124, and generates image data with movement effect by synthesizing with the target image data.

The generation of movement effect-added image data by the digital camera 1 will be described separately for a case where a convergence point exists and a case where a convergence point does not exist.
-When there is a convergence point-
FIG. 3A shows frame image data F1 to Fn (generally referred to as frame image data F) whose generation is started in response to a half-press operation of the release button by the user at time t1, and time t2. 4 schematically shows still image data S generated in response to the user's full depression operation of the release button. The frame image data F and the still image data S shown in FIG. 3A are images of a vehicle moving away from the right side of the paper. This vehicle is detected as the moving subject MS by the moving subject detection unit 121. Still image data S is the target image data selected by the image selection unit 120 as described above.

  The movement information detection unit 122 detects a moving subject detected for each of two frame image data F (m−1) and Fm (2 <m <n) that are temporally adjacent to each other among the frame image data F1 to Fn. In the MS, the movement vector is calculated by calculating the difference between the coordinates on the image of the corresponding position. Furthermore, the movement information detection unit 122 compares the size of the moving subject MS detected in each of the two frame image data F (m−1) and Fm. That is, the movement information detection unit 122 calculates the ratio of the size of the moving subject MS detected in the frame image data Fm to the size of the moving subject MS detected in the frame image data F (m−1). . When the calculated ratio is larger than the first threshold, the movement information detection unit 122 determines that the moving subject MS is approaching, and the calculated ratio is smaller than the second threshold (<first threshold). Is determined that the moving subject MS is moving away. It is assumed that suitable values are set for the first threshold value and the second threshold value based on experiments or the like and stored in a predetermined storage area.

  In the example shown in FIG. 3A, as described above, the moving object MS is a vehicle that moves away from the right hand in the drawing. Therefore, the direction of the motion vector calculated by the movement information detection unit 122, that is, the movement direction of the moving subject MS has a predetermined angle with respect to the horizontal plane of the screen. Furthermore, the movement information detection unit 122 determines that the rate of change in the size of the moving subject MS is smaller than the second threshold value. Therefore, the movement information detection unit 122 determines that a convergence point exists in the movement direction of the moving subject MS detected from the frame image data F shown in FIG.

  When it is determined that a convergence point exists, the movement information detection unit 122 identifies the position of the convergence point on the image. In this case, the movement information detection unit 122 specifies the position of the calculated motion vector, that is, the position where the moving direction of the moving subject MS is extended as the position of the convergence point. In the example shown in FIG. 3A, since the direction of the motion vector is in the upper right part of the figure, the movement information detecting unit 122 is near the upper right end of the image data, as shown in FIG. Is determined as the position of the convergence point P. In FIG. 3B, the moving subject MS and the background in the still image data are indicated by broken lines for convenience. When the position of the convergence point P is specified, the drawing effect selection unit 124 determines an area (drawing area) for drawing the movement effect. FIG. 3C shows the drawing area R determined by the drawing effect selection unit 124. In FIG. 3C, dots are given inside the drawing region R for convenience.

  As shown in FIG. 3C, the drawing area R is surrounded by two straight lines L1 and L2 passing through the outer edge of the moving subject MS starting from the convergence point P, and opposite to the moving direction of the moving subject MS. This is an area outside the boundary of the moving subject MS in the direction. Note that the straight lines L1 and L2 have the maximum angle formed by the straight lines L1 and L2 among the straight lines connecting the edge of the moving subject MS and the convergence point P detected by a known edge detection process.

  When the drawing region R is determined, the drawing effect selection unit 124 selects and determines the movement effect used for the movement effect data to be generated. When the convergence point P exists, the drawing effect selection unit 124 selects a plurality of straight lines extending radially behind the moving subject MS as the expression of the movement effect, that is, the movement line. The drawing effect selection unit 124 determines the movement line mode, that is, the number of straight lines to be drawn, the length of each straight line, the distance between adjacent straight lines, the magnitude of the detected motion vector, that is, the moving speed of the moving subject MS. Decide accordingly. In this case, it is preferable that the drawing effect selection unit 124 determines to increase the number of straight lines, increase the length of the straight lines, and close the intervals between the straight lines as the moving speed of the moving subject MS increases. .

  The processing unit 125 generates movement effect data in which the movement effect determined by the drawing effect selection unit 124 is expressed in the determined drawing area R. FIG. 3D shows an example of the movement effect data. As shown in FIG. 3D, each of the six linear movement lines LE extends radially behind the moving subject MS. The processing unit 125 synthesizes the generated movement effect data with the still image data S that is the target image data, thereby generating the image data SE with movement effect shown in FIG.

  When the moving subject MS is approaching along the optical axis direction of the digital camera 1, the drawing effect selection unit 124 can select the following movement line LE as the expression of the movement effect. FIG. 4A shows an example of the moving effect-added image data SE generated in this case. In the example shown in FIG. 4A, it is assumed that a person approaching here is detected as the moving subject MS. In this case, the drawing effect selection unit 124 sets the drawing region R above the moving subject MS and outside the boundary of the moving subject MS. Then, the drawing effect selection unit 124 detects the size of the motion line LE (number of straight lines, length, interval between adjacent straight lines) as described above, that is, the moving speed of the moving subject MS. And the thickness of the movement line LE is determined so as to gradually change. For example, as shown in FIG. 4A, the effect that the moving subject MS is approaching here is drawn by rendering the moving line LE to be thicker as the distance from the moving subject MS increases. it can.

  When the moving subject MS moves away along the optical axis direction of the digital camera 1, the drawing effect selection unit 124 can select the following movement line LE as the expression of the movement effect. FIG. 4B shows an example of the image data SE with movement effect generated in this case. In the example shown in FIG. 4B, it is assumed that a person who moves away is detected as the moving subject MS. In this case, the drawing effect selection unit 124 sets the drawing region R below the moving subject MS and outside the boundary of the moving subject MS. Then, the drawing effect selection unit 124 detects the size of the motion line LE (number of straight lines, length, interval between adjacent straight lines) as described above, that is, the moving speed of the moving subject MS. To be decided. As a result, the effect that the moving subject MS is moving away from here can be expressed.

  Note that the processing unit 125 may combine the movement effect data after performing the following processing on the still image data S when generating the movement effect-added image data SE. That is, the processing unit 125 performs a monochrome process on the still image data S and then performs a process of increasing the contrast. As a result, the contour of the subject in the still image data S is emphasized, so that the image corresponding to the generated moving effect-added image data SE can be reproduced as a cartoon-like still image. The processing unit 125 generates the moving effect-added image data SE by drawing the moving line LE on the still image data S instead of generating the moving effect data and combining it with the still image data S. Also good.

-When there is no convergence point-
If the moving subject does not satisfy the condition (1) or (2) described above, the moving direction detection unit 122 determines that there is no convergence point. As an example of the case where the convergence point does not exist, there are a case where the moving subject MS moves while keeping the relative distance to the digital camera 1 substantially constant, and a case where the moving direction of the moving subject MS is not substantially constant. An example will be described.

--When the moving subject moves with the relative distance substantially constant--
FIG. 5A shows the frame image data F1 to Fn that have been generated in response to the user's full-press operation of the release button at time t1, and the user's full-press operation of the release button at time t2. The generated still image data S is schematically shown. The frame image data F and the still image data S shown in FIG. 5A are images of a person who moves from the right side to the left side while keeping the relative distance from the digital camera 1 substantially constant. This person is detected as the moving subject MS by the moving object detection unit 121. Still image data S is the target image data selected by the image selection unit 120 as described above.

  The movement information detection unit 122 calculates a motion vector of the moving subject MS detected for each of the frame image data F1 to Fn, and determines an area where the magnitude of the motion vector exceeds a predetermined value as the drawing area R. When the drawing region R is determined, the drawing effect selection unit 124 selects and determines the movement effect used for the movement effect data to be generated. In this case, the drawing effect selection unit 124 selects a plurality of straight lines extending in the same direction along the moving direction behind the moving subject MS as a moving effect expression, that is, the moving line LE. As described above, the drawing effect selection unit 124 sets the number of straight lines to be drawn, the length of each straight line, the distance between adjacent straight lines, that is, the size of the motion vector detected as the movement line LE. This is determined according to the moving speed of the subject MS. Also in this case, the drawing effect selection unit 124 determines to increase the number of straight lines, increase the length of the straight lines, and close the intervals between the straight lines as the moving speed of the moving subject MS increases. Is preferred.

  The processing unit 125 generates movement effect data in which the movement effect determined by the drawing effect selection unit 124 is expressed in the determined drawing area R. FIG. 5B shows an example of the movement effect data. As shown in FIG. 5B, each of the three linear movement lines LE extends behind the moving subject MS in the same direction, that is, behind the moving subject MS along the moving direction. The processing unit 125 synthesizes the generated movement effect data with still image data S that is target image data, thereby generating image data SE with movement effect shown in FIG. 5C.

  Note that the processing unit 125 may combine the movement effect data after performing the following processing on the still image data S when generating the movement effect-added image data SE. That is, the processing unit 125 performs a monochrome process on the still image data S and then performs a process of increasing the contrast. As a result, the contour of the subject in the still image data S is emphasized, so that the image corresponding to the generated moving effect-added image data SE can be reproduced as a cartoon-like still image. The processing unit 125 generates the moving effect-added image data SE by drawing the moving line LE on the still image data S instead of generating the moving effect data and combining it with the still image data S. Also good.

--When the moving direction of the moving subject MS is not substantially constant--
FIG. 6A shows frame image data F1 to Fn that have been generated in response to the user's half-press operation of the release button at time t1, and the user's full-press operation of the release button at time t2. The generated still image data S is schematically shown. The frame image data F and the still image data S shown in FIG. 6A are images of a state where the roller coaster traveling from the right side flips over. This roller coaster is detected as the moving subject MS by the moving subject detection unit 121. Still image data S is the target image data selected by the image selection unit 120 as described above.

  The movement information detection unit 122 calculates a motion vector of the moving subject MS detected for each of the frame image data F1 to Fn, and determines an area where the magnitude of the motion vector exceeds a predetermined value as the drawing area R. When the drawing region R is determined, the drawing effect selection unit 124 selects and determines the movement effect used for the movement effect data to be generated. In this case, the drawing effect selection unit 124 draws the movement line LE along the shape of the drawing region R. The movement line LE may use an approximate curve, or may be formed by connecting straight lines having a length corresponding to the magnitude of the motion vector. As a result, a plurality of lines extending in the same direction along the moving direction behind the moving subject MS are selected as expressions for the moving effect. Note that, as described above, the drawing effect selection unit 124 detects the number of the movement lines LE, the length of each movement line LE, the size of the motion vector detected from the distance between adjacent movement lines LE, that is, the moving subject. It is determined according to the moving speed of MS. Also in this case, the drawing effect selection unit 124 increases the number of the movement lines LE, increases the length of the movement lines LE, and increases the distance between the movement lines LE as the moving speed of the moving subject MS increases. It is preferable to decide to do so.

  The processing unit 125 generates movement effect data in which the movement effect determined by the drawing effect selection unit 124 is expressed in the determined drawing area R. FIG. 6B shows an example of the movement effect data. As shown in FIG. 6B, behind the moving subject MS, each of the movement lines LE that draws a curve along the two moving directions extends behind the moving subject MS. The processing unit 125 synthesizes the generated movement effect data with the still image data S that is target image data, thereby generating image data SE with movement effect shown in FIG.

  Note that the processing unit 125 may combine the movement effect data after performing the following processing on the still image data S when generating the movement effect-added image data SE. That is, the processing unit 125 performs a monochrome process on the still image data S and then performs a process of increasing the contrast. As a result, the contour of the subject in the still image data S is emphasized, so that the image corresponding to the generated moving effect-added image data SE can be reproduced as a cartoon-like still image. The processing unit 125 generates the moving effect-added image data SE by drawing the moving line LE on the still image data S instead of generating the moving effect data and combining it with the still image data S. Also good.

  The image processing unit 112 records the movement effect-added image data SE generated by the processing unit 125 as described above on the memory card 109. Note that the image processing unit 112 may record the still image data S and the movement effect data generated by the processing unit 125 in the memory card 109 in association with each other. In this case, every time display is performed, the processing unit 125 may synthesize the movement effect data and the still image data S to generate the movement effect-added image data SE.

  The operation of the digital camera 1 according to the embodiment of the present invention will be described with reference to FIGS. The processing in FIG. 7 is performed by executing a program in the control circuit 102. This program is stored in a memory (not shown), and is activated and executed by the control circuit 102 when the movement effect load image mode is selected by the user.

  In step S1, the movement information detection unit 122 acquires movement information such as a movement direction and a movement amount related to the moving subject MS by detecting the moving subject MS on the frame image data and calculating a motion vector. Proceed to S2. In step S2, the moving direction detector 122 determines whether or not the convergence point P exists in the moving direction of the moving subject MS. If a convergence point exists, step S2 is affirmed and the process proceeds to step S3. If no convergence point exists, step S2 is determined to be negative and the process proceeds to step S4.

  In step S3, a moving effect-added image data generation process is performed when there is a convergence point whose details will be described later, and the process ends. In step S4, a moving effect-added image data generation process is performed when there is no convergence point whose details will be described later, and the process ends.

With reference to the flowchart shown in FIG. 8, the moving effect-added image data generation process when there is a convergence point in step S3 will be described.
In step S101, the movement information detection unit 122 identifies the position of the convergence point P on the image data, and proceeds to step S102. In step S102, the drawing effect selection unit 124 determines the drawing region R based on the position of the specified convergence point P and the shape of the moving subject MS, and proceeds to step S103.

  In step S103, the drawing effect selection unit 124 detects the movement vector LE that is drawn in the drawing area R, that is, the number of straight lines to be drawn, the length of each straight line, and the distance between adjacent straight lines. The determination is made according to the size, that is, the moving speed of the moving subject MS, and the process proceeds to step S104. In step S104, the processing unit 125 generates movement effect data expressing the movement effect determined by the drawing effect selection unit 124 in the determined drawing area R, and the process proceeds to step S105. In step S105, the processing unit 125 synthesizes the movement effect data and the still image data S to generate the movement effect-added image data SE and ends the movement effect-added image data generation process.

With reference to the flowchart shown in FIG. 9, the moving effect-added image data generation processing when there is no convergence point in step S4 will be described.
In step S201, the movement information detection unit 122 determines an area where the magnitude of the motion vector of the moving subject MS exceeds a predetermined value as the drawing area R, and proceeds to step S202. In step S202, the drawing effect selection unit 124 sets the mode of the movement line LE to be drawn in the drawing area R, that is, the number of movement lines LE to be drawn, the length of each movement line LE, and the interval between adjacent movement lines LE. The determination is made according to the detected magnitude of the motion vector, that is, the moving speed of the moving subject MS, and the process proceeds to step S203. In step S203, the processing unit 125 generates movement effect data in which the movement effect determined by the drawing effect selection unit 124 is expressed in the determined drawing area R, and the process proceeds to step S204. In step S204, the processing unit 125 synthesizes the movement effect data and the still image data S to generate the movement effect-added image data SE and ends the movement effect-added image data generation process.

According to the digital camera according to the above-described embodiment, the following operational effects can be obtained.
(1) The digital camera 1 stops the movement effect based on the movement information detection unit 122 that detects movement information including the movement direction of the moving subject MS detected by the moving subject detection unit 121 and the detected movement information. And a processing unit 125 for drawing the image data S. Therefore, it is possible to generate a still image in which the moving effect according to the moving direction of the moving subject MS can be visually observed.

(2) The drawing effect selection unit 124 applies the still image data S as the target image data according to whether or not the moving direction of the moving subject MS converges to one point on the image while maintaining the substantially same direction. The movement effect to draw was selected. Specifically, the drawing effect selection unit 124, if the moving direction of the moving subject MS converges to a convergence point P that is one point on the image while maintaining the substantially same direction, the outer edge portion of the moving subject MS A movement line LE extending radially backward is selected as the movement effect. In the case where the moving direction of the moving subject MS does not converge to one point on the image while maintaining the substantially same direction, a trajectory extending behind the outer edge of the moving subject MS is selected as the moving line LE. Accordingly, since the manner of drawing the movement line LE can be varied according to the moving direction of the moving subject MS, the effect according to the moving state of the moving subject MS can be visually observed.

(3) The drawing effect selection unit 124 increases or decreases the number and length of the moving lines LE and the interval between the moving lines LE according to the moving speed of the moving subject MS. As a result, it is possible to visually grasp the moving speed of the moving subject MS.

The following modifications are also within the scope of the present invention, and one or a plurality of modifications can be combined with the above-described embodiment.
(1) One aspect of the present invention includes generating moving effect-added image data using a plurality of still image data acquired in the continuous shooting mode. In this case, the image selection unit 120 may select target image data from a plurality of still image data, for example, according to a user's selection operation.

(2) The movement effect-added image data may be generated using a plurality of frame image data F acquired in the moving image mode. In this case, the image selection unit 120 sequentially selects each of the plurality of frame image data F as target image data. The moving subject detection unit 121 detects the moving subject MS using a predetermined number (for example, six frames) of frame image data generated immediately before the target image data is acquired. The movement information detection unit 122, the drawing effect selection unit 124, and the processing unit 125 perform the same processing as in the embodiment to generate the movement effect-added image data SE. Since the target image data is sequentially selected for each of the frame image data F, the processing unit 125 generates a plurality of pieces of image data SE with movement effect.

  Each of the plurality of image data SE with movement effect reflects the movement information of the moving subject MS in the frame image data F for 6 frames immediately before the corresponding target image data is acquired. As a result, when a plurality of image data SE with movement effect is reproduced as a moving image, the movement line LE based on the movement information for six frames is drawn and reproduced together with the moving subject MS. In this case, when the moving subject MS is moving at the same moving speed, the moving line LE having the same length is located behind the moving subject MS along with the reproduction of the image data SE with the movement effect of a plurality of frames. It is drawn and played. As the moving speed of the moving subject MS increases / decreases, the length of the moving line LE is gradually increased / decreased and is drawn and reproduced behind the moving subject MS. When the moving subject MS stops moving, the moving line LE is erased. . When the stopped moving subject MS starts moving again, the moving line LE is again drawn and reproduced.

(3) When the magnitude of the motion vector exceeds a predetermined value, that is, when the moving speed of the moving subject MS exceeds a predetermined speed, image processing for deforming the shape of the moving subject MS may be performed. An example is shown in FIG. FIG. 10 shows a case where the moving object MS is a vehicle moving from left to right. As shown in FIG. 10A, when the moving speed of the moving subject MS is equal to or lower than a predetermined speed, image data with a movement effect in which a movement line LE is drawn is generated in the same manner as in the embodiment. . On the other hand, when the moving speed of the moving subject MS exceeds the predetermined speed, the processing unit 125 performs image processing on the moving subject MS on the target image data, for example, FIG. To the shape shown in Then, the processing unit 125 may synthesize the movement effect data and the target image data after the image processing illustrated in FIG. 10B to generate the image data with the movement effect illustrated in FIG.

(4) When the magnitude of the motion vector is less than the predetermined value, that is, when the moving speed of the moving subject MS is very slow, the moving line LE is not extended in a direction along the direction of the motion vector. Can be. An example is shown in FIG. FIG. 11 also shows a case where the moving subject MS is a vehicle that moves from left to right as in FIG. FIG. 11A is the same as FIG. On the other hand, when the moving speed of the moving subject MS is very slow, the drawing effect selecting unit 124 selects a moving line LE as shown in FIG. 11B as the moving effect. As a result, the moving effect-added image data SE shown in FIG.

(5) The movement effect is not limited to drawing the movement line LE. For example, dust or the like may be drawn behind the moving subject MS. In this case, the drawing effect selection unit 124 may select a moving effect that increases the moving speed of the moving subject MS so that the dust range is wider and the amount of dust increases.

(6) In the embodiments, the digital camera 1 has been described as an example, but the present invention can also be applied to a portable information display terminal such as a tablet including a camera and a display.
(7) Instead of performing the generation process of the image data with movement effect by the digital camera 1, the generation process of the image data with the movement effect may be performed by an image processing apparatus (for example, a personal computer).

  FIG. 12 is a block diagram illustrating a personal computer as an example of the image processing apparatus. The image processing apparatus 300 displays and edits image data of a subject image captured by the digital camera 1 and stores the image data. The image processing apparatus 300 includes a control circuit 301, an HDD 302, a monitor control circuit 303, a monitor 304, a buffer memory 305, an input device 306, a memory card interface 307, and an external interface 308. The external interface 308 performs data communication with an external device such as the digital camera 1 via a predetermined cable or wireless transmission path.

  The input device 306 is operated by a user, and is composed of, for example, a keyboard and a mouse. In the HDD 302, for example, an image file corresponding to a moving image or a still image taken by the digital camera 1 is recorded. The external interface 308 is a USB interface for connecting an external device such as the digital camera 1 to the image processing apparatus 300, for example. The image processing apparatus 300 inputs an image file or the like from the memory card 109 or an external device via the memory card interface 307 or the external interface 308. The input image file is recorded on the HDD 302 by the control circuit 301. The image file generated by the digital camera 1 described above is also recorded in the HDD 302 in the same manner.

  The control circuit 301 is a microcomputer that controls the image processing apparatus 300 and includes a CPU, a ROM, and other peripheral circuits. The control circuit 301 includes the image processing unit 320 as a function. The image processing unit 320 generates display image data to be displayed on the monitor 304 based on the input image file. The image processing unit 320 includes an image selection unit 321, a moving subject detection unit 322, a movement information detection unit 323, a drawing effect selection unit 324, and a processing unit 325 as functions. Unit 120, moving subject detection unit 121, movement information detection unit 122, drawing effect selection unit 124, and processing unit 125.

  That is, when frame image data F and still image data S are input via the memory card 109, the image selection unit 321 selects the still image data S as target image data, and the moving subject detection unit 322 The moving subject MS is detected using the data F. The movement information detection unit 323 calculates a motion vector, determines the presence / absence of the convergence point P, and determines the drawing region R. The drawing effect selection unit 324 selects a movement effect mode based on the calculated motion vector, and the processing unit 325 generates movement effect data in which the selected movement effect is drawn, and is a still image that is target image data. The image data SE with the movement effect is generated by combining with the image data S.

  As long as the characteristics of the present invention are not impaired, the present invention is not limited to the above-described embodiments, and other forms conceivable within the scope of the technical idea of the present invention are also included in the scope of the present invention. .

DESCRIPTION OF SYMBOLS 1 ... Digital camera, 300 ... Image processing apparatus,
102, 301 ... control circuit, 103, 305 ... buffer memory,
106: operation unit, 112, 320 ... image processing unit,
120, 321 ... image selection unit, 121, 322 ... moving subject detection unit,
122, 323 ... movement information detection unit, 124, 324 ... drawing effect selection unit,
125, 325 ... processing section

Claims (7)

Determining means for determining whether there is a convergence point that converges to one point while maintaining the moving direction of the moving subject substantially the same direction based on a plurality of image data;
When the determination means determines that the convergence point exists, a plurality of straight lines extending radially behind the moving subject are generated as movement effect data, and the movement effect data is combined with the target image data to add a movement effect. When image data is generated and the convergence point does not exist, a plurality of straight lines extending in the same direction along the moving direction are generated behind the moving subject as the moving effect data, and the moving effect data is the target image. An image processing apparatus comprising: image processing means for generating image data with a movement effect by combining with data . The image processing apparatus according to claim 1.
The determination means has a direction in which the moving direction of the moving subject is a predetermined angle or more with respect to a horizontal direction of an image, and the moving direction is substantially constant between each of the plurality of image data, When there is a change in the size of the moving subject between each of the plurality of image data, it is determined that the convergence point exists.
An image processing apparatus. The image processing apparatus according to claim 1 or 2,
When the determination unit determines that the convergence point exists, the image processing unit increases the number of the plurality of straight lines radially extending behind the moving subject as the moving speed of the moving subject increases. When the effect data is generated and the movement effect data is combined with the target image data to generate image data with a movement effect, and the convergence point does not exist, the higher the moving speed of the moving subject, The movement effect data is generated by increasing the number of a plurality of straight lines extending backward in the same direction along the movement direction, and the movement effect data is combined with the target image data to generate the image data with movement effect. about
An image processing apparatus. The image processing apparatus according to any one of claims 1 to 3,
In the case where the determination unit determines that the convergence point exists, the image processing unit increases the length of each of the plurality of straight lines extending radially behind the moving subject as the moving speed of the moving subject increases. The movement effect data is generated, and the movement effect data is combined with the target image data to generate image data with movement effect. When the convergence point does not exist, the movement speed of the moving subject increases as the movement speed increases. The movement effect data is generated by increasing the length of each of a plurality of straight lines extending in the same direction along the movement direction behind the subject, and the movement effect data is combined with the target image data to generate the image with the movement effect. Generating data
An image processing apparatus. In the image processing device according to any one of claims 1 to 4,
In the case where the determination unit determines that the convergence point exists, the image processing unit increases the distance between a plurality of straight lines extending radially behind the moving subject as the moving speed of the moving subject increases. When the moving effect data is generated, the moving effect data is combined with the target image data to generate image data with moving effect, and the convergence point does not exist, the moving subject increases as the moving speed of the moving subject increases. The movement effect data in which the intervals of a plurality of straight lines extending in the same direction along the movement direction are densely rearward are generated, and the movement effect data is combined with the target image data to obtain the image data with movement effect. To generate
An image processing apparatus. The image processing apparatus according to any one of claims 1 to 5,
A release button operated by the user;
Imaging means for imaging a subject and generating image data;
While the release button is half-pressed by the user, the imaging means sequentially generates the image data, and the determination means is based on the plurality of image data generated by the imaging means. Determine if a point exists,
In response to the release button being fully pressed by the user, the imaging unit generates the image data as the target image data.
An imaging apparatus characterized by the above. A determination process for determining whether there is a convergence point that converges to one point while maintaining the movement direction of the moving subject substantially the same direction based on a plurality of image data;
If it is determined by the determination process that the convergence point exists, a plurality of straight lines extending radially behind the moving subject are generated as movement effect data, and the movement effect data is combined with the target image data to add a movement effect. When image data is generated and the convergence point does not exist, a plurality of straight lines extending in the same direction along the moving direction are generated behind the moving subject as the moving effect data, and the moving effect data is the target image. Causing a computer to execute image processing for generating image data with movement effect by combining with data
An image processing program characterized by the above.

Images