JP5597069B2 - Image editing apparatus, control method thereof, and program - Google Patents

Description

  The present invention relates to an image editing technique.

  It is difficult for the video to be a video having a composition or effect desired by the user by one shooting. Therefore, there is a need to change the composition and effect after shooting. Specifically, it is desired to change the composition by changing the position and size of the subject, or to add a zoom blur to the background area only to give a feeling of dynamism.

In order to actually perform these, for example, in PHOTOSHOP of ADOBE, an image is divided into layers for each region. For each layer, the image processing to be applied is selected, the parameters of each image processing are set one by one, and the layers are overlapped for synthesis. For example, in order to change the position and size of the subject, it is necessary to specify the magnification of the size of the x and y coordinates of the position of the subject area. In the case of zoom blur, it is necessary to specify the direction and intensity of the zoom blur and the x and y coordinates of the center position. (Non-Patent Document 1)
In addition, processing such as zooming, panning, and rotation is performed in accordance with operations on the touch panel. (Patent Document 1)

Special table 2008-508601 gazette

United States Adobe Systems "Adobe Photoshop CS4 User Guide" <http://help.adobe.com/en_US/Photoshop/11.0/photoshop_cs4_help.pdf>

  When changing the composition of a video or applying an effective effect, it is necessary to perform image processing with accurate parameters specified. However, in the above prior art, it is necessary to set each image processing parameter accurately one by one, which is troublesome.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique capable of intuitively and effectively performing image editing on an intended object in an image including a plurality of objects.

In order to solve this problem, for example, an image editing apparatus of the present invention has the following configuration. That is,
An image editing apparatus for editing an image in which a plurality of objects are mixed,
Display means for displaying an image;
Presence / absence of an instruction by an operator on the display screen of the display means, and detection means for detecting an indicated position;
In a state where an image having a plurality of objects is displayed on the display screen of the display unit, when the detection unit detects an instruction to the display screen by the operator, the plurality of objects are determined from the coordinate position of the instruction. Determining means for determining which of the processing targets,
An instruction position referred to when determining the object by the determination means, in accordance with the variation of the gesture and the indicated position represented by the indicated position detected by subsequently continuously according to the indicated position, the edit type and edit Editing means for determining parameters for editing, and editing the determined object according to the determined editing type and parameters ,
The editing means determines that the type of editing is motion blur when the gesture to be detected is an operation for performing parallel movement of two designated positions, and performs parallel movement when the gesture for the initial stage of the gesture is performed. The quantity is used as a motion blur parameter .

  According to the present invention, intuitive and effective image editing can be performed on an intended object in an image including a plurality of objects.

1 is a block configuration diagram of an image editing apparatus according to an embodiment. The figure which shows the example of operation in each mode in transition of the browsing mode of 1st Embodiment, and edit mode. The flowchart which shows the process at the time of the browsing mode in 1st Embodiment. The figure which shows the example of an edit in 1st Embodiment. 6 is a flowchart illustrating processing in an edit mode according to the first embodiment. The figure which shows the example of an edit in 2nd Embodiment. 10 is a flowchart illustrating processing in an edit mode according to the second embodiment. The figure which shows the example of an edit in 3rd Embodiment. 10 is a flowchart illustrating processing in an edit mode according to the third embodiment. The figure which shows the example of an edit in 4th Embodiment. 10 is a flowchart illustrating processing in an edit mode according to the fourth embodiment. The figure which shows the example of an edit in 5th Embodiment. 10 is a flowchart illustrating processing in an edit mode according to the fifth embodiment. The flowchart of the parameter determination in 5th Embodiment. The figure which shows the example of an edit in 6th Embodiment. 10 is a flowchart illustrating processing in an edit mode according to the sixth embodiment. The figure which shows the example of an edit in 8th Embodiment. The flowchart which shows the process of the edit mode in 8th Embodiment. 10 is a flowchart illustrating processing in an edit mode according to the seventh embodiment.

  Embodiments according to the present invention will be described below in detail with reference to the accompanying drawings.

[First Embodiment]
FIG. 1 is a block diagram of an image editing apparatus that implements the present embodiment. As shown in the figure, this apparatus includes a CPU 101 that controls the entire apparatus, a ROM 102 that stores programs executed by the CPU 101 and data to be used in advance, and a RAM 103 that is used as a work area for the CPU 101. In addition, the apparatus includes a storage device 104 that stores and holds image files, and a display control unit 106 that expands an image to be displayed on a display 105 such as a liquid crystal or performs a drawing process under the control of the CPU 101. Furthermore, the present apparatus includes a position detection unit 108 that detects whether or not the operator touches the transparent touch panel 107 provided on the front surface of the display screen of the display 105, a touch position, and an interface for communicating with an external device. 109. Note that the coordinate input device represented by the touch panel 107 and the position detection unit 108 according to the embodiment can detect at least a plurality of touch positions, regardless of the type.

  In the present embodiment, a photo frame will be described as an example of the image editing apparatus. In the following description, it is assumed that a plurality of image files are already stored in the storage device 104. Furthermore, it is assumed that the image file stored in the storage device 104 is a file having a format in which a plurality of objects such as a foreground object and a background object are mixed, and the depth (layer) and position can be independently defined for each object. A representative example of such a file format is a PSD file (file with extension “psd”) created by PHOTOSHOP of ADOBE, but the file type is not limited to this.

  When a switch (not shown) is operated to turn on the power of the apparatus according to the present embodiment, the CPU 100 executes the program stored in the ROM 102 so that the apparatus functions as the photo frame described above. become.

  A series of UI flows performed in this embodiment will be described using screen shots of the display 105. Although the operator does not touch the display but strictly touches the touch panel 107 on the front surface, it is hereinafter referred to as touching the display 105 for convenience.

  FIG. 2A shows a screen shot in the browsing mode. The browsing mode is a mode in which an image stored in the storage device 104 in the photo frame is browsed. 301 is a screen shot in the browsing mode. One image stored in the storage device 104 is displayed. When the flick operation 302 is performed on the display 105, the next image stored in the storage device 104 is displayed. Also, as shown in FIG. 2B, when a double tap operation 303 is performed on the display 105, the screen transitions from the browsing mode to the editing mode screen 304. A black frame on the screen indicates that you are currently in edit mode. In other words, when the black frame is not displayed, it means the browsing mode. Note that the flick operation refers to an operation in which a finger is touched on the display 105 and moved while the touch state is maintained. In addition, the double tap operation refers to an operation of touching the same position twice without any short period.

  FIG. 2C shows a screen shot when image processing for subject movement in the edit mode is performed. In the edit mode, each object of the displayed image is managed by the layer. Accordingly, in the case of an image composed of the foreground and background layers, an image having a foreground area in which only the foreground is cut out and holding the alpha matte and an image different from the image in which the foreground is cut out are superimposed and displayed as a background area. Image processing is performed on each displayed image in accordance with a user operation.

  In this embodiment, the foreground / background region is separated by taking a difference from an image of only a background region that does not show a foreground region prepared in advance. However, this is an example, and the present invention is not limited to this. For example, as in Japanese Patent Application Laid-Open No. 2009-059060, the moving object may be separated as the foreground, or the user may specify the foreground and background areas. In the present embodiment, the editing target has two areas, the foreground and the background, but this is an example and the present invention is not limited to this. For example, it may have a plurality of regions or only one region.

  In this embodiment, the foreground area and the background area are generated from different images. However, this is an example and the present invention is not limited to this. For example, from the same image, the foreground may be cut out and used as a foreground area, and an area other than the forward tilt area may be generated as a background area.

  The display image shown in FIG. 2C has a foreground area 401 and a background area 402. When the operator touches the foreground area 401 on the display 201 and performs a drag operation 403, the display position of the foreground area 401 changes according to the drag operation 403 to become the foreground area 404. Also, as shown in FIG. 2D, when a double tap operation is performed on the display 105, the view mode screen 301 is displayed, and a composite image in which the foreground area 404 and the background area 402 subjected to the moving process are combined is displayed. To do. At this time, the CPU 101 updates the original image file to reflect the edited content. Specifically, the position information of the foreground object in the original image file is updated. Here, the foreground area 404 is the movement target, but the background area 402 may be moved.

  Processing executed by the CPU 101 will be described with reference to the flowchart of FIG. When the power is turned on and activated as a photo frame, a browse mode screen 301 is displayed (S501). Next, it is determined whether or not the power is turned off (S502). If it is determined that the power is not turned off, the touch on the display 105 (the presence or absence of touch and the coordinate position when touched) is detected (S503). It is determined whether or not the input gesture is a double tap from the change of the input coordinates (S504). If it is determined to be a double tap (S504), the foreground and background of the image being displayed are stored as separate layers in the RAM 103, and the process proceeds to edit mode processing (S505). After the edit mode processing is completed, the process proceeds to power-off determination (S502). Details of the edit mode process (S505) will be described later.

  If it is not determined that the operation is a double tap operation (S504), it is determined whether or not the input gesture is a flick operation from a change in input coordinates (S506). If it is determined that a flick has occurred (S506), the next image after the currently displayed image is read from the storage medium 104, displayed on the display 201 (S507), and the process proceeds to power-off determination (S502). When it is determined that the operation is not a flick operation (S506), the process proceeds to power-off determination (S502). If it is determined that the power has been turned off (S503), the process ends.

  With reference to FIG. 4, the operation of the object moving process in the edit mode in the present embodiment will be described. The object moving process is performed by dragging the object area. Reference numeral 601 denotes input coordinates at the start of dragging. In the present embodiment, the touched is one place, the touched input coordinate 601 is included, and the area 602 assigned to the layer located at the foremost is a movement target. Reference numeral 603 denotes an input detection position during the drag operation. Reference numeral 604 denotes a drag movement amount. The movement amount 604 is set as the movement amount of the target movement process, and processing for moving the movement target area 602 to the position 605 is performed. The drag operation is an operation for changing the touch position while maintaining the touch state, and is synonymous with the flick operation described above, and there is no difference when viewed from the CPU 101. However, in general, the drag operation has a meaning in the position in the non-touch state, and there is no limitation on the time taken from the touch to the non-touch state. On the other hand, in the flick operation, it is an operation to pay a finger, and it is important in which direction the touch position has changed, and compared with the drag operation, from the touch state to the non-touch state Says a relatively short period. Also in this embodiment, the flick operation and the drag operation will be described in the above meaning, in order to facilitate understanding of the technical contents.

  FIG. 5A shows a flowchart of processing in the editing mode in the present embodiment. When the edit mode is started, input coordinates to the display 105 are detected (S701). It is determined from the change of the input coordinates whether or not the input gesture is a double tap (S702). If it is determined that it is not a double tap (S702), it is determined from the change of the input coordinates whether or not the input gesture is a drag (S703). When it is determined that the operation is a drag operation (S703), a target movement process (S704) is performed, and the process proceeds to input position detection (S701). Details of the target movement process will be described later. When it is determined that the operation is not a drag operation (S703), the process proceeds to input position detection (S701). If it is determined to be a double tap (S702), composition is performed based on the performed image processing, the synthesized image is stored in the storage medium 104, and the editing mode ends.

  FIG. 5B shows a flowchart of object movement processing in the present embodiment.

  When the object movement process is started, the object in the region including the input coordinates at the start of the drag operation is set as a target image to be subjected to image processing (S801). Input coordinates to the display 105 are detected (S802). It is determined whether the input gesture is a drag operation (S803). If it is determined that the operation is a drag operation (S803), a drag movement amount is detected (S804). The detected movement amount is determined as the movement amount of the target movement process (S805). A movement process is performed on the determined object image based on the determined movement amount (distance between the start position and end position of the drag operation) (S806). Then, the process returns to the input coordinate detection process (S802) again. At this stage, the coordinate position information of the object to be moved of the original image file is also updated. And then. If it is not determined to be a drag (S803), the target movement process is terminated.

  In this embodiment, since the foreground area and the background area are separate images, no blank space is generated even if each area is moved. However, if the foreground area and the background area are generated from the same image, the object has moved. This creates a blank space. The generated blank portion is complemented by using neighboring image data as disclosed in Japanese Patent Application Laid-Open No. 2004-104232. Alternatively, as in Japanese Patent Application Laid-Open No. 2010-20581, a part where a blank part appears in another image may be detected, and the complementary process may be performed using that part. By configuring and processing in this way, the target can be moved intuitively, and the user can generate a desired image.

[Second Embodiment]
A second embodiment will be described. The apparatus configuration in the second embodiment is assumed to be the same as that in FIG. Note that switching between the viewing mode and the editing mode is the same as in the first embodiment, and description of the viewing mode is omitted.

  The operation of the enlargement / reduction process in the editing mode in the second embodiment will be described with reference to FIGS. 6 (a) and 6 (b). The enlargement / reduction operation has two touch positions, and a pinch-in / pinch-out operation for expanding or reducing the input interval between the two locations (two points) is performed on the object to enlarge or reduce. In FIG. 6A, reference numeral 901 denotes an input position at the start of pinch-in / out. Reference numeral 902 denotes an intermediate point of the input position 901. In the present embodiment, an object 903 that is included in the midpoint 902 and managed in the foremost layer is set as a target area to which image processing is applied. In FIG. 6B, reference numeral 904 denotes an input position during pinch-in / out. The enlargement / reduction ratio (magnification) is determined from the amount of change between the distance 905 between the input positions at the start and the distance 906 between the input positions being operated. Enlargement / reduction processing is performed with the intermediate point 902 as the center of enlargement / reduction processing. In the present embodiment, the enlargement / reduction processing is performed with the intermediate point 902 as the center, but this is an example and the present invention is not limited to this. For example, the enlargement / reduction process may be performed around a predetermined point for each target area, or the enlargement / reduction process may be performed around the center of gravity of the target area.

  FIG. 7A shows a flowchart of processing in the editing mode in the present embodiment. The description of the same parts as in FIG. It is determined whether or not the input gesture is pinch-in / out (S1001). When it is determined that the pinch-in / out is performed (S1001), enlargement / reduction processing is performed (S1002). At this time, the original image file is also updated according to the enlarged object. Thereafter, the process proceeds to input position detection (S701). Details of the enlargement / reduction processing will be described later. When it is determined that it is not pinch-in / out (S1001), the process proceeds to input position detection (S701).

  FIG. 7B shows a flowchart of the enlargement / reduction processing in this embodiment. The description of the same parts as in FIG. 5B is omitted.

  An image in an area including the intermediate point 902 of the pinch-in / out input position is set as a target image to be subjected to image processing (S1101). It is determined whether or not the input gesture is pinch-in / out (S1102). When it is determined that pinch-in / out is performed (S1102), the ratio of the distances 905 and 906 between the two input positions at the start and during operation is calculated (S1103). The calculated ratio is set as the enlargement ratio, and the intermediate point 902 is set as the center point of the enlargement / reduction process (S1104). Based on the determined parameters, enlargement processing is performed using affine transformation (S1105). If it is determined that it is not pinch-in / out (S1102), the enlargement / reduction process is terminated.

  In the present embodiment, the center point of the enlargement / reduction processing is set to the intermediate point 902. However, this is an example and the present invention is not limited to this. For example, one of the two input positions at the start may be used as the center point of the enlargement / reduction process, or the center point of the enlargement / reduction process may be determined for each region in advance. In this embodiment, the enlargement / reduction processing is performed using affine transformation, but this is only an example and the present invention is not limited to this. For example, complement processing may be performed by a bicubic method or the like after affine transformation processing. By configuring and processing in this way, the target can be intuitively enlarged / reduced, and the user can generate a desired image.

[Third Embodiment]
In the third embodiment, an image (object) rotation process in the edit mode will be described. The apparatus configuration is the same as in FIG. The switching between the browsing mode and the editing mode is the same as that in the first embodiment, and the description of the browsing mode is omitted.

  With reference to FIGS. 8A and 8B, the operation of the target rotation process in the edit mode in the third embodiment will be described. The rotation processing operation is performed by a rotation operation that rotates and moves two input positions on the rotation target. Reference numeral 1201 in FIG. 8A denotes an input position at the start of the rotation operation. Reference numeral 1202 denotes an intermediate point of the input position 1201. In the present embodiment, an object area 1203 that includes the midpoint 1202 and is managed by the layer located at the forefront is set as an area to be subjected to image processing. Reference numeral 1204 denotes an input position during the rotation operation. The rotation angle is determined from the amount of change in the inclination of the straight line 1205 connecting the input positions in FIG. 8A at the start and the inclination of the straight line 1206 connecting the input positions in FIG. The object determined as a target is rotated around 1202.

  FIG. 9A shows a flowchart of processing in the edit mode in the third embodiment. The description of the same parts as in FIG.

  The CPU 101 determines whether or not the input gesture is a rotation operation (S1301). When it is determined that the operation is a rotation operation (S1301), a rotation process is performed (S1302). At this time, the original image file is also updated. Thereafter, the process returns to the input position detection process (S701). Details of the rotation processing will be described later. If it is not determined to be a rotation operation (S1301), the process proceeds to input position detection (S701).

  FIG. 9B shows a flowchart of the rotation process in the third embodiment. The description of the same parts as in FIG. 5B and FIG. 7B is omitted.

  The CPU 101 determines whether or not the input gesture is a rotation operation (S1401). If it is determined that the operation is a rotation operation (S1401), the amount of change in the inclination of the straight lines 1205 and 1206 connecting the two fingers is calculated (S1402). The calculated change amount is set as the rotation amount, and the intermediate point 1202 is set as the rotation center (S1403). Based on the determined parameters, rotation processing is performed using affine transformation (S1404). If the rotation operation is not determined (S1401), the rotation process ends.

In the present embodiment, the intermediate point 1202 is the center of rotation, but this is only an example and is not limited to this. For example, a predetermined point for each region may be used as the center of rotation, and one of the input detection positions may be used as the center of rotation.
In the present embodiment, the rotation processing is performed using affine transformation, but this is an example and the present invention is not limited to this. For example, complement processing may be performed by a bicubic method or the like after affine transformation processing. By configuring and processing in this way, the target can be intuitively rotated, and the user can generate a desired image.

[Fourth Embodiment]
In the fourth embodiment, an example in which motion blur is given to a selected object will be described. The apparatus configuration is the same as in FIG. The switching between the browsing mode and the editing mode is the same as that in the first embodiment, and the description of the browsing mode is omitted.

  The operation of the motion blur processing in the edit mode in the fourth embodiment will be described with reference to FIG. The motion blur processing is performed by so-called double drag, in which two points on the motion blur object are touched and the two touched points are moved in parallel. Reference numeral 1501 denotes an input position at the start of double dragging. Reference numeral 1502 denotes an intermediate point of the input position 1501. An object that includes this intermediate position and is managed in a layer in the nearer side is determined as a processing target. In the illustrated case, since the intermediate point 1501 is a background image, the background area is an area to which image processing is applied. Reference numeral 1503 denotes an input position during double dragging. Reference numeral 1504 denotes an intermediate point of the input position 1503. Reference numeral 1505 denotes a parallel movement amount between the intermediate points 1502 and 1504 of the operation position at the start and during the operation. The direction of the motion blur is determined from the direction of the parallel movement amount 1505, the strength of the motion blur is determined from the length of the movement amount 1505, and the motion blur is given to the background area.

  FIG. 11A shows a flowchart of processing in the edit mode in the fourth embodiment. The description of the same parts as in FIG.

  The CPU 101 determines whether or not the input gesture is a double drag (S1601). If it is determined to be double drag (S1601), motion blur processing is performed (S1602). Details of the motion blur processing will be described later. If it is not determined to be a double drag (S1601), the process proceeds to input position detection (S701).

  FIG. 11B shows a flowchart of motion blur processing in the fourth embodiment. The description of the same parts as those in FIGS. 5B and 7B is omitted.

  The CPU 101 determines whether or not it is a double drag (S1701). When it is determined that the drag is a double drag (S1701), the movement amount 1505 of the intermediate points 1502 and 1505 is calculated (S1702). The direction of the calculated movement amount 1505 is set as the direction of the motion blur, and the length is set as the strength of the motion blur (S1703). The same image as the target region is given transparency, and motion blur is given by synthesizing the target region while shifting the image in the specified direction at a predetermined interval up to a distance corresponding to the specified intensity (S1704). At this time, the original image file is updated. If it is not determined that the double drag operation is continued (S1701), the motion blur process is terminated.

  In this embodiment, the motion blur is performed by synthesizing the original image while maintaining the transparency, but this is an example and the present invention is not limited to this. For example, it may be performed by giving a Gaussian blur in a range corresponding to the designated direction and the designated intensity. By configuring and processing in this way, an effective motion blur can be intuitively provided, and a user can generate a desired image.

[Fifth Embodiment]
In the fifth embodiment, an example in which zoom blur is applied to a selected object will be described. The apparatus configuration is the same as in FIG. The switching between the browsing mode and the editing mode is the same as that in the first embodiment, and the description of the browsing mode is omitted.

  An operation of zoom blur processing in the edit mode in the fifth embodiment will be described with reference to FIG. The zoom blur process is performed by two sets of double drags when a pair of touches with two fingers on the zoom blur application target area is taken as one set. 1801 and 1802 shown in the figure are input positions of each group of double drags at the start of double dragging. Reference numerals 1803 and 1804 denote intermediate points between the input positions 1801 and 1802. In the present embodiment, the background area that includes the midpoints 1803 and 1804 and is the innermost layer is an area to which image processing is applied. However, if the midpoints 1803 and 1804 are included in different areas, both areas are areas to which image processing is applied.

  In the present embodiment, when the midpoints 1803 and 1804 are included in different areas, both areas are targets for image processing. However, this is only an example, and the present invention is not limited to this. For example, only a region including the previously detected intermediate point may be set as an area to which image processing is applied.

  Reference numerals 1805 and 1806 denote input positions of each group of double drags during double drag. Reference numerals 1807 and 1808 are intermediate points between the input positions 1805 and 1806. A vector 1809 connecting 1803 and 1807 and a vector 1810 connecting 1804 and 1808 are calculated. An intersection point 1811 between a straight line connecting 1803 and 1807 and a straight line connecting 1804 and 1808 is calculated. The average length of the calculated vectors 1809 and 1810 is set as the zoom blur intensity, and the direction of the vectors 1809 and 1810 with respect to the intersection 1811 is set as the zoom blur direction. The calculated intersection 1811 is set as an approximate vanishing point, and the center of the zoom blur is determined. Based on the determined parameter, zoom blur is added to the background area.

  FIG. 13A shows a flowchart of processing in the edit mode in the present embodiment. The description of the same parts as in FIG.

  The CPU 101 determines whether or not the input is two sets of double drag (S1901). If it is determined that there are two sets of double drags (S1901), zoom blur processing is performed (S1902). At this time, the original image file is also updated to reflect the editing result. Details of the zoom blur processing will be described later. If it is not determined that the two sets of double drags (S1901), the process proceeds to input position detection (S701).

  FIG. 13B shows a flowchart of zoom blur processing in the present embodiment. The description of the same parts as in FIG. 5B is omitted.

  The CPU 101 sets an image in an area including the intermediate points 1803 and 1804 as a target image to be subjected to image processing. When each intermediate point is included in a different area, both images are set as target images to be subjected to image processing (S2001). Then, it is determined whether or not there are two sets of double drags (S2002). When it is determined that two sets of double drags have been made (S2002), vectors 1809 and 1810 connecting intermediate points 1803 and 1807 and 1804 and 1808 at the start and during operation are calculated. An intersection 1811 of straight lines connecting the intermediate points 1803 and 1807 and 1804 and 1808 at the start and in operation is calculated (S2003). A parameter is determined based on the calculated operation change amount (S2004). Details of parameter determination will be described later. The enlargement ratio is calculated according to the determined direction and strength. The same image as the target area with transparency is gradually changed to the determined enlargement ratio, and the enlargement / reduction process is performed around the calculated center position to compose the target area ( S2005). In the present embodiment, the base image is given transparency, and the zoom blur is given by enlarging / reducing and synthesizing the base image based on the determined parameters. However, this is only an example, and the present invention is not limited to this. is not.

  FIG. 14A shows a flowchart of parameter determination in the fifth embodiment.

  The CPU 101 determines the zoom blur intensity so as to be proportional to the average value of the lengths of the vectors 1809 and 1810 (S2101). In this embodiment, the average value of the lengths of the vectors 1809 and 1810 is used, but this is an example and the present invention is not limited to this. For example, the longer or shorter value may be used. The direction of zoom blur is calculated based on the direction of the vectors 1809 and 1810 toward the intersection 1811. However, if the vector directions are different, the zoom blur direction is determined by the direction of the longer vector (S2102). A center point is determined from the intersection point 1811 (S2103), and parameter determination is terminated. Details of the center point determination will be described later.

  FIG. 14B shows a flowchart of the center point determination process in the fifth embodiment.

  The CPU 101 detects the edge of the image using the Canny method (S2201). An arbitrary distance from the intersection 1811 is set as a vanishing point candidate range (S2202). Only a straight line passing through the vanishing point candidate range is detected using the Hough transform (S2203). By detecting only the straight line that passes through the vanishing point candidate range, the detection of the straight line that does not converge to the vanishing point can be suppressed, the misrecognition of the vanishing point can be avoided, and unnecessary line detection processing can be omitted, thereby reducing the processing. Is possible. The point at which the detected straight line is most converged is detected and set as the vanishing point, which is the center point of the zoom blur (S2204).

In this embodiment, the edge of the image is detected by using the Canny method, but this is an example and the present invention is not limited to this. For example, the Robert cross method may be used.
In this embodiment, the Hough transform is used for detecting a straight line, but this is an example and the present invention is not limited to this. In the present embodiment, the vanishing point is detected using the information of the intersection 1811 and is used as the center of the zoom blur. However, this is an example and the present invention is not limited to this. For example, the intersection 1811 may be the center of the zoom blur. In the present embodiment, zoom blur processing is performed, but this is an example and the present invention is not limited to this. For example, processing may be performed such that the closer the specified center is, the lower the saturation is. By configuring and processing in this way, zoom blur can be intuitively applied even to an image separated into a plurality of regions, and a user can generate a desired image.

[Sixth Embodiment]
In the sixth embodiment, an example in which focus change processing is performed on a selected object will be described. The apparatus configuration is the same as in FIG. The switching between the browsing mode and the editing mode is the same as that in the first embodiment, and the description of the browsing mode is omitted.

  With reference to FIGS. 15A and 15B, the focus change processing operation in the edit mode according to the sixth embodiment will be described. The focus change process is performed by tapping on the object to be focused. In FIG. 15A, reference numeral 2301 denotes a background area, and 2302 denotes a foreground area. Before the tap, only the foreground area 2302 is blurred and the background area is focused. When the tap operation is performed, only the background area 2301 that does not include the area of the input position 2303 in FIG. 15B is blurred and the foreground area 2302 is in focus. That is, the focus target is alternately switched every time the tap is performed.

  FIG. 16A shows a flowchart of the edit mode in the sixth embodiment. The description of the same parts as in FIG.

  The CPU 101 determines whether or not the input gesture is a tap (S2401). When the tap is determined (S2401), focus change processing is performed (S2402). Details of the focus change process will be described later. When it is not determined as a tap (S2401), the process proceeds to input position detection (S701).

  FIG. 16B shows a flowchart of the focus change process in the sixth embodiment.

  An image in an area not including the tap input position 2303 is set as a target image to be subjected to image processing (S2501). A Gaussian blur having a predetermined intensity is applied to the determined target image (S2502).

  In this embodiment, the blur strength is set to a predetermined value. However, this is an example, and the present invention is not limited to this. For example, the strength of the blur may be determined according to the duration of the tap. In this embodiment, the blur is given by the Gaussian blur, but this is an example and the present invention is not limited to this. By configuring and processing in this way, the focus change process can be performed intuitively, and the user can generate a desired image.

[Seventh Embodiment]
In the first to sixth embodiments, the processing according to the gesture performed by the operator on the display 105 has been described. However, these functions may be installed in one editing apparatus. An example of this case will be described as a seventh embodiment. The apparatus configuration is the same as in FIG. The switching between the browsing mode and the editing mode is the same as that in the first embodiment, and the description of the browsing mode is omitted.

  FIG. 19 is a flowchart of a processing procedure in the edit mode according to the seventh embodiment.

  In step S <b> 3001, the CPU 101 detects that the display 105 has changed from a non-touch state to a touch state. If a touch input is detected, the process proceeds to S3002, and the coordinate position on the screen intended by the operator is calculated according to the number of touches and each touch position. When there is one touch location, the touch position matches the position on the screen intended by the operator. However, for example, when there are two touch locations, as described above, it can be determined that the operation of the operator is an initial stage for performing a pinch-in / out operation. Therefore, in the latter case, the central position is determined as the coordinate position intended by the operator. Then, an object that includes the coordinate position intended by the determined operator and that is managed in the foremost layer is determined as an object to be edited. Next, the type of gesture of the operator is determined from the change in the touch position that is subsequently input. Then, processing according to the determined gesture is performed in S3003 to S3008. Since S3003 to S3008 correspond to the first to sixth embodiments, refer to the above embodiments for the contents thereof.

  Thereafter, the process proceeds to S3009, where it is determined whether or not a double tap has been input. If not, the processes after S3001 are repeated. If a double-tap input has been made, the process advances to step S3009 to synthesize the edited image, end the editing mode, and shift to the viewing mode.

  In the first to sixth embodiments described above, it is described that the original image file is updated each time each editing process is performed. However, when the editing mode is ended, the original image file is updated. You may make it update.

[Eighth Embodiment]
In the above embodiment, the description has been made on the assumption that the image data to be edited is a still image. However, an example of editing a moving image, specifically, an example of performing time shift processing as image processing will be described as an eighth embodiment. To do. The apparatus configuration is the same as in FIG. The switching between the browsing mode and the editing mode is the same as that in the first embodiment, and the description of the browsing mode is omitted.

  Here, the time shift processing represents the trajectory of a moving object by cutting out the same object from successive frames of a moving image and placing the cut out object on an image of one frame.

  With reference to FIGS. 17A and 17B, the time shift operation in the edit mode in the eighth embodiment will be described. The time shift operation is performed by a three-point rotation operation that rotates an input position of three points (three touches). Reference numeral 2601 in FIG. 17A denotes a certain frame image of the moving image. Reference numerals 2602, 2603, and 2604 denote input positions at the start of the three-point rotation operation. Reference numeral 2605 denotes a circle connecting the input positions 2602, 2603, and 2604. 2606 is the center of the circle 2605. Reference numeral 2607 denotes a line segment connecting the input position 2603 and the center 2606. Reference numerals 2608, 2609, and 2610 in FIG. 17B are input detection positions during operation, and 2602, 2603, and 2604 are moved, respectively. Reference numeral 2611 denotes a circle that connects the operation input positions 2608, 2609, and 2610. 2612 is the center of the circle 2611. Reference numeral 2613 denotes a line segment connecting the operation input position 2609 and the center 2612. Reference numeral 2614 denotes an image set by a time shift operation.

  A region that does not include the center point 2606 is set as a cutout target. A rotation angle (angle) formed by the line segment 2607 and the line segment 2613 is calculated. From the calculated rotation angle, the time range of the frame for cutting out the region is determined. The target area is cut out from the frame within the determined time range, and is set at the same position.

  FIG. 18A shows a flowchart of processing in the edit mode in the eighth embodiment. The description of the same parts as in FIG.

  The CPU 101 determines whether or not the input gesture is a three-point rotation operation (S2701). When it is determined that the operation is a three-point rotation operation (S2701), a time shift process is performed (S2702). Details of the time shift processing will be described later. If it is not determined that the operation is a three-point rotation operation (S2701), the process proceeds to input detection (S701).

  FIG. 18B shows a flowchart of the time shift process in the eighth embodiment. The description of the same parts as in FIG. 5B is omitted.

  The CPU 101 calculates the center 2606 of a circle connecting the three entry positions. An image of a region not including the calculated center 2606 is set as a target image to be subjected to image processing (S2801). It is determined whether or not the input gesture is a three-point rotation operation (S2802). If it is determined in S2802 that the operation is a three-point rotation operation, the center 206 of the circle connecting the input positions of the three points being operated is calculated. An angle formed by a line segment 2607 connecting the input position 2603 at the start and the center 2606 of the circle at the start and a line segment 2613 connecting the input position 2609 during the operation and the center 2612 of the operated circle is calculated (S2803). From the calculated angle, the time range of the frame for cutting out the target area is determined (S2804). A target area is cut out from frames at arbitrary intervals within the determined time range, and placed at the cut out position (S2805). If it is determined in S2802 that the operation is not a three-point rotation operation, the time shift process is terminated.

  In the present embodiment, the region that does not include the center of the circle connecting the three input detection positions is the target region. However, this is an example and the present invention is not limited to this. For example, an area including the center position may be set as the target area. In the present embodiment, each of the straight lines connecting the input detection position and the center of the circle is calculated as the operation change amount, but this is an example and the present invention is not limited to this. For example, the amount of change in the slope of the line segment may be calculated. In this embodiment, the time shift process is performed according to the three-point rotation operation, but this is an example and the present invention is not limited to this. For example, the moving image playback speed may be controlled in accordance with a three-point rotation operation. By configuring and processing in this way, it becomes possible to perform time shift processing intuitively, and the user can generate a desired image.

Claims (9)

An image editing apparatus for editing an image in which a plurality of objects are mixed,
Display means for displaying an image;
Presence / absence of an instruction by an operator on the display screen of the display means, and detection means for detecting an indicated position;
In a state where an image having a plurality of objects is displayed on the display screen of the display unit, when the detection unit detects an instruction to the display screen by the operator, the plurality of objects are determined from the coordinate position of the instruction. Determining means for determining which of the processing targets,
The type of editing and the editing position are determined in accordance with the pointing position referred to when the object is determined by the determining means, the gesture represented by the pointing position detected successively and continuously with respect to the pointing position, and the amount of change in the pointing position. Editing means for determining parameters for editing, and editing the determined object according to the determined editing type and parameters,
The editing means determines that the type of editing is motion blur when the gesture to be detected is an operation for performing parallel movement of two designated positions, and performs parallel movement when the gesture for the initial stage of the gesture is performed. amounts, images editing apparatus characterized by utilizing as a parameter for motion blur. An image editing apparatus for editing an image in which a plurality of objects are mixed,
Display means for displaying an image;
Presence / absence of an instruction by an operator on the display screen of the display means, and detection means for detecting an indicated position;
In a state where an image having a plurality of objects is displayed on the display screen of the display unit, when the detection unit detects an instruction to the display screen by the operator, the plurality of objects are determined from the coordinate position of the instruction. Determining means for determining which of the processing targets,
The type of editing and the editing position are determined in accordance with the pointing position referred to when the object is determined by the determining means, the gesture represented by the pointing position detected successively and continuously with respect to the pointing position, and the amount of change in the pointing position. Editing means for determining parameters for editing, and editing the determined object according to the determined editing type and parameters,
The editing means decides that the type of editing is zoom blur when the gestures to be detected are two sets of drag operations when the instructions at two places are set as one set, and the intersection of the moving directions of each set. was the center point of the zoom blur images editing apparatus characterized by using the average of each set of the movement amount as a parameter of the intensity of zoom blur. An image editing apparatus for editing an image in which a plurality of objects are mixed,
Display means for displaying an image;
Presence / absence of an instruction by an operator on the display screen of the display means, and detection means for detecting an indicated position;
In a state where an image having a plurality of objects is displayed on the display screen of the display unit, when the detection unit detects an instruction to the display screen by the operator, the plurality of objects are determined from the coordinate position of the instruction. Determining means for determining which of the processing targets,
The type of editing and the editing position are determined in accordance with the pointing position referred to when the object is determined by the determining means, the gesture represented by the pointing position detected successively and continuously with respect to the pointing position, and the amount of change in the pointing position. Editing means for determining parameters for editing, and editing the determined object according to the determined editing type and parameters,
When the gesture to be detected is a tap operation at one location, the editing means determines that the type of editing is blurring processing, and each time the tap operation is performed, the object located at the farthest position including the tapped position , foremost objects located switches the blur target alternately, images editing apparatus characterized by the use as a parameter of the intensity blur time instructs. An image editing apparatus for editing an image in which a plurality of objects are mixed,
Display means for displaying an image;
Presence / absence of an instruction by an operator on the display screen of the display means, and detection means for detecting an indicated position;
In a state where an image having a plurality of objects is displayed on the display screen of the display unit, when the detection unit detects an instruction to the display screen by the operator, the plurality of objects are determined from the coordinate position of the instruction. Determining means for determining which of the processing targets,
The type of editing and the editing position are determined in accordance with the pointing position referred to when the object is determined by the determining means, the gesture represented by the pointing position detected successively and continuously with respect to the pointing position, and the amount of change in the pointing position. Editing means for determining parameters for editing, and editing the determined object according to the determined editing type and parameters,
The editing means determines that the type of editing is time shift processing when the gesture to be detected is a rotation operation of three designated positions, and rotates when the gesture is performed with respect to the initial stage of the gesture images editing apparatus characterized by determining a corner, as a parameter of the time range to be cut out from the moving image. By executing read into a computer program to function as the means of the image editing apparatus according to any one of claims 1 to 4 to the computer. Display means for displaying an image, on the display screen of said display means includes the presence or absence of an instruction by the operator, as well, and a detecting means for detecting a pointing position, image editing for editing an image in which a plurality of objects are mixed An apparatus control method comprising:
In a state in which an image having a plurality of objects is displayed on the display screen of the display means, when the detection means detects an instruction to the display screen by the operator, the determination means determines from the coordinate position of the instruction, A determination step of determining which of the plurality of objects is to be processed;
The editing means performs the editing according to the pointing position referred to when the object is determined in the determining step, the gesture represented by the pointing position continuously detected after the pointing position and the change amount of the pointing position. An editing step for determining a type and a parameter for editing, and editing the determined object according to the determined editing type and parameter;
In the editing step, when the gesture to be detected is an operation for performing parallel movement of two designated positions, it is determined that the type of editing is motion blur, and parallel movement is performed when the gesture is performed for the initial stage of the gesture. the amount of the control method of the image picture editing apparatus characterized by utilizing as a parameter for motion blur. Image editing for editing an image in which a plurality of objects are mixed, including display means for displaying an image, and detection means for detecting the presence / absence of an instruction and an indication position on the display screen of the display means An apparatus control method comprising:
In a state in which an image having a plurality of objects is displayed on the display screen of the display means, when the detection means detects an instruction to the display screen by the operator, the determination means determines from the coordinate position of the instruction, A determination step of determining which of the plurality of objects is to be processed;
The editing means performs the editing according to the pointing position referred to when the object is determined in the determining step, the gesture represented by the pointing position continuously detected after the pointing position and the change amount of the pointing position. An editing step for determining a type and a parameter for editing, and editing the determined object according to the determined editing type and parameter;
In the editing step, when two directions are set as one set, when the gesture to be detected is two sets of drag operations, it is determined that the type of editing is zoom blur, and the intersection of the moving directions of each set is determined. Is the center point of the zoom blur, and the average value of the moving amounts of each set is used as a zoom blur intensity parameter. Image editing for editing an image in which a plurality of objects are mixed, including display means for displaying an image, and detection means for detecting the presence / absence of an instruction and an indication position on the display screen of the display means An apparatus control method comprising:
In a state in which an image having a plurality of objects is displayed on the display screen of the display means, when the detection means detects an instruction to the display screen by the operator, the determination means determines from the coordinate position of the instruction, A determination step of determining which of the plurality of objects is to be processed;
The editing means performs the editing according to the pointing position referred to when the object is determined in the determining step, the gesture represented by the pointing position continuously detected after the pointing position and the change amount of the pointing position. An editing step for determining a type and a parameter for editing, and editing the determined object according to the determined editing type and parameter;
In the editing step, if the gesture to be detected is a single tap operation, it is determined that the type of editing is blurring processing, and the object that is located at the farthest position including the tapped position is determined each time the tap operation is performed. A method for controlling an image editing apparatus, characterized in that the object to be blurred is alternately switched as a blurring target, and the designated time is used as a blurring intensity parameter. Image editing for editing an image in which a plurality of objects are mixed, including display means for displaying an image, and detection means for detecting the presence / absence of an instruction and an indication position on the display screen of the display means An apparatus control method comprising:
In a state in which an image having a plurality of objects is displayed on the display screen of the display means, when the detection means detects an instruction to the display screen by the operator, the determination means determines from the coordinate position of the instruction, A determination step of determining which of the plurality of objects is to be processed;
The editing means performs the editing according to the pointing position referred to when the object is determined in the determining step, the gesture represented by the pointing position continuously detected after the pointing position and the change amount of the pointing position. An editing step for determining a type and a parameter for editing, and editing the determined object according to the determined editing type and parameter;
In the editing process, when the gesture to be detected is a rotation operation of three designated positions, the editing type is determined to be a time shift process, and the rotation when the gesture is performed with respect to the initial stage of the gesture is performed. A method of controlling an image editing apparatus, wherein a corner is determined as a parameter of a time range cut out from a moving image.

Images