JP4453365B2 - Image layout apparatus, image layout program, and image layout method - Google Patents

Description

  The present invention relates to an image layout device, and more particularly to an image layout device, an image layout program, and an image layout method that can arrange images naturally in consideration of the directionality of the image.

An apparatus has been proposed that acquires a plurality of elements photographed by a camera, video, etc., and creates an electronic album or the like by pasting the acquired elements on a display screen, a mount, etc. (hereinafter referred to as an album mount). ing.
Also, software (Adobe illustrator) that can set a path and arrange characters and objects along the path is also commercially available.

In the invention of Patent Document 1, a device has been proposed in which a reference line is extended from an image and layout can be performed using the reference line. The reference line is obtained by extending the left, right, up and down sides, the ground line of the image, and the like, and can be moved to an arbitrary position in the image or can be changed in thickness.
JP 2003-99794 A

  However, when a user creates a path and arranges an image along the created path as in commercially available software, the user needs to create a path, which is troublesome. On the other hand, when selecting any one of a plurality of paths prepared in advance and placing an image along the selected path, there is no need to create a path, but the route in which the user places the image And it is necessary to manually place each image on the path.

  Therefore, the present invention has been made paying attention to such an unsolved problem of the conventional technology, and realizes a natural arrangement in consideration of the directionality of the image and reduces the labor of the user. An object of the present invention is to provide an image layout apparatus, an image layout program, and an image layout method suitable for the above.

[Invention 1] The image layout apparatus of the invention 1 comprises:
An image layout device that lays out an image along a path,
Path direction determining means for determining a direction of a path that is a route for arranging the image;
Image placement means for placing the image on the path based on the path direction determined by the path direction determination means.

According to this configuration, the path direction determining unit determines the direction of the path that is a route for arranging the image, and the image arranging unit arranges the image based on the path direction determined by the path direction determining unit. An image can be automatically and naturally laid out along a path, and an effect that a troublesome operation by a user is unnecessary can be obtained.
[Invention 2] The image layout apparatus according to Invention 2 is the same as that of Invention 1,
The path direction determining means determines a base point of a path serving as a reference for arranging the image.

According to this configuration, the path direction determination unit can determine the base point of the path serving as a reference for arranging the image. Therefore, after the first image is placed at this base point, another image along the direction of the path is obtained. Can be sequentially arranged, so that an effect that images can be arranged relatively naturally and easily can be obtained.
[Invention 3] The image layout apparatus according to Invention 3 is the same as that of Invention 2,
Image direction acquisition means for acquiring the direction of the image, wherein the path direction determination means determines the direction of the path and the base point based on the direction of the image acquired by the image direction acquisition means. To do.

  According to this configuration, since the image direction can be acquired by the image direction acquisition unit, the path direction determination unit determines the path direction and the base point when the image is arranged using the directionality of the image. Images can be placed naturally along the path direction. In addition, since the direction of the image can be acquired using the image direction acquisition means, the direction of the image can be objectively determined, and even when the image layout apparatus is automated, different interpretations are made regarding the direction of the image. The effect is that the direction of the path can be determined relatively accurately.

[Invention 4] The image layout apparatus of Invention 4 is the invention 3, wherein
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, an angle formed between the tangential direction of the path at the temporary arrangement position of the image and the direction of the image is calculated, and the number of images where the calculated angle is equal to or less than a predetermined angle is calculated. Counting and selecting a predetermined number of images having the predetermined angle or less from the plurality of candidates based on the value counted for each candidate, and setting the direction and base point of the selected candidate path to the path It is characterized by determining as a direction and a base point.

  According to this configuration, the path direction determination unit sets a plurality of path direction and base point combination candidates, calculates an angle formed by the image direction and the tangential direction of the path, and the angle is equal to or less than a predetermined value. The number of images that are equal to or less than a predetermined value can be selected from a plurality of candidates, and images along the direction of the path can be selected by a relatively simple method. The effect that it can arrange | position naturally is acquired.

[Invention 5] The image layout apparatus according to Invention 5 is the invention 3, wherein
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, obtain the inner product of the vector of the tangential direction of the path and the direction vector of the image at the temporary arrangement position of the image, the number of images for which the obtained inner product is negative, or Count the number of images that are positive, the number of images that are 0 or less, or the number of images that are 0 or more, and the number of images counted from among the plurality of candidates based on the value counted for each candidate Is selected, and the direction and base point of the selected candidate path are determined as the direction and base point of the path.

  According to this configuration, the path direction determining unit sets a plurality of path direction and base point combination candidates, calculates an inner product of the image direction vector and the tangential direction vector of the path, and the calculated inner product is negative. The number of negative images can be counted and the one with the predetermined number of negative images can be selected from a plurality of candidates. The effect that it can arrange | position is acquired.

[Invention 6] The image layout apparatus according to Invention 6 is the invention 3, wherein
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, calculate the angle formed by the tangent direction of the path and the direction of the image at the temporary arrangement position of the image, find the sum of the angles calculated for all images, Based on the sum of angles obtained for each candidate, the one having the predetermined smallest sum of the angles is selected from the plurality of candidates, and the direction and base point of the selected candidate path are determined as the direction and base point of the path. It is characterized by doing.

  According to this configuration, the path direction determination unit sets a plurality of path direction and base point combination candidates, calculates the angle formed by the image direction and the tangential direction of the path, and calculates all the images. Since the sum of the angles can be obtained and the one with the smallest sum of the angles can be selected from a plurality of candidates, the image can be naturally arranged along the direction of the path by a relatively simple method. The effect is obtained.

[Invention 7] The image layout device of Invention 7 is the invention 3,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, calculate the angle formed by the tangential direction of the path and the direction of the image at the temporary arrangement position of the image, to obtain the maximum value of the angle calculated for all images, Based on the maximum value of the angle obtained for each candidate, the one with the predetermined maximum value of the angle is selected from the plurality of candidates, and the path direction and base point of the selected candidate are selected as the direction of the path and It is determined as a base point.

  According to this configuration, the path direction determination unit calculates an angle formed between the tangential direction of the path and the direction of the image at the temporary arrangement position of the image, and calculates the maximum value of the calculated angles for all the images, Based on the maximum value of the angle obtained for each candidate, it is possible to select a candidate having the smallest maximum value of the angle from the plurality of candidates. Thus, the effect that the images can be arranged naturally is obtained.

[Invention 8] The image layout apparatus of Invention 8 is the image layout apparatus according to any one of Inventions 2 to 7,
The base point is a start point where the image is first arranged, an end point where the image is arranged last, or an intermediate point between the start point and the end point.
According to this configuration, the path direction determination unit determines the path direction, the start point, and the end point when placing an image using the image directionality, thereby matching the image direction and the path direction. Since the image is arranged on the path based on the path direction, the start point, and the end point obtained by the path direction determination unit, the image can be arranged relatively appropriately with respect to the path direction. Is obtained.

[Invention 9] The image layout apparatus of Invention 9 is the invention 8,
When the path is an annular path, after the image is arranged by the image arrangement unit, a route other than the route connecting the start point to the end point along the determined path direction is deleted.
According to this configuration, since the path between the start point and the end point of the path is deleted, it is possible to easily determine the path direction, the start point, and the end point.

[Invention 10] The image layout device of Invention 10 is the invention according to any one of Inventions 1 to 9,
When it is determined that there is an interval between the images after the images are arranged by the image arranging means, another image or illustration prepared in advance is inserted above or around the path.

According to this configuration, even if there is a gap between images, the direction of the path becomes natural. For example, if an image or an illustration such as an arrow is inserted, the direction of the path can be understood at a glance. Has the advantage.
[Invention 11] The image layout program of Invention 11 is
An image layout program that lays out an image along a path,
Realized as path direction determining means for determining the direction of the path to be the path for arranging the image, and image arranging means for arranging the image on the path based on the direction of the path determined by the path direction determining means. It is a program for making a computer perform the process performed.
According to this configuration, the same effect as in the first aspect can be obtained.

[Invention 12] The image layout program of Invention 12 is the invention 11,
The path direction determining means determines a base point of a path serving as a reference for arranging the image.
According to this configuration, the same effect as in the second aspect can be obtained.

[Invention 13] The image layout program of the invention 13 in the invention 12,
Image direction acquisition means for acquiring the direction of the image, wherein the path direction determination means determines the direction of the path and the base point based on the direction of the image acquired by the image direction acquisition means. To do.
According to this configuration, the same effect as in the third aspect can be obtained.

[Invention 14] The image layout program of the invention 14 in the invention 13,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, an angle formed between the tangential direction of the path at the temporary arrangement position of the image and the direction of the image is calculated, and the number of images where the calculated angle is equal to or less than a predetermined angle is calculated. Counting and selecting a predetermined number of images having the predetermined angle or less from the plurality of candidates based on the value counted for each candidate, and setting the direction and base point of the selected candidate path to the path The direction and the base point are determined.
According to this configuration, the same effect as in the fourth aspect can be obtained.

[Invention 15] The image layout program of the invention 15 in the invention 13,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, obtain the inner product of the vector of the tangential direction of the path and the direction vector of the image at the temporary arrangement position of the image, the number of images for which the obtained inner product is negative, or Count the number of images that are positive, the number of images that are 0 or less, or the number of images that are 0 or more, and the number of images counted from among the plurality of candidates based on the value counted for each candidate Is selected, and the direction and base point of the selected candidate path are determined as the direction and base point of the path.
According to this configuration, the same effect as in the fifth aspect can be obtained.

[Invention 16] The image layout program of the invention 16 in the invention 13,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, calculate the angle formed by the tangent direction of the path and the direction of the image at the temporary arrangement position of the image, find the sum of the angles calculated for all images, Based on the sum of angles obtained for each candidate, the one having the predetermined smallest sum of the angles is selected from the plurality of candidates, and the direction and base point of the selected candidate path are determined as the direction and base point of the path. It is characterized by doing.
According to this configuration, the same effect as in the sixth aspect can be obtained.

[Invention 17] The image layout program of the invention 17 in the invention 13,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, calculate the angle formed by the tangential direction of the path and the direction of the image at the temporary arrangement position of the image, to obtain the maximum value of the angle calculated for all images, Based on the maximum value of the angle obtained for each candidate, the one with the predetermined maximum value of the angle is selected from the plurality of candidates, and the path direction and base point of the selected candidate are selected as the direction of the path and It is determined as a base point.
According to this configuration, the same effect as in the seventh aspect can be obtained.

[Invention 18] The image layout program of the invention 18 according to any one of the inventions 12 to 17,
The base point is a start point where the image is first arranged, an end point where the image is arranged last, or an intermediate point between the start point and the end point.
According to this configuration, the same effect as in the eighth aspect can be obtained.

[Invention 19] An image layout program of an invention 19 according to the invention 18,
When the path is an annular path, after the image is arranged by the image arrangement unit, a portion of the path where the image is not arranged between the end point and the start point of the path is deleted.
According to this configuration, the same effect as in the ninth aspect can be obtained.

[Invention 20] An image layout program according to Invention 20, in any one of Inventions 11 to 19,
When it is determined that there is an interval between the images after the images are arranged by the image arranging means, another image or illustration prepared in advance is inserted above or around the path.
According to this configuration, the same effect as in the tenth aspect can be obtained.

[Invention 21] The image layout method of the invention 21 includes:
An image layout method for laying out an image along a path,
A path direction determining step for determining a direction of a path to be a route for arranging the image;
And an image placement step of placing the image on the path based on the direction of the path determined in the path direction determination step.
According to this configuration, the same effect as in the first aspect can be obtained.

[Invention 22] The image layout method of Invention 22 is the same as that of Invention 21,
In the path direction determining step, a base point of a path serving as a reference for arranging the image is determined.
According to this configuration, the same effect as in the second aspect can be obtained.

[Invention 23] The image layout method of Invention 23 is the same as that of Invention 22,
An image direction acquisition step of acquiring the direction of the image, wherein the path direction determination step determines the direction of the path and the base point based on the direction of the image acquired in the image direction acquisition step. To do.
According to this configuration, the same effect as in the third aspect can be obtained.

[Invention 24] The image layout method of Invention 24 is the invention 23, in which
The path direction determination step sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired in the image direction acquisition step, an angle formed by the tangential direction of the path at the temporary arrangement position of the image and the direction of the image is calculated, and the number of images in which the calculated angle is equal to or less than a predetermined angle is calculated. Counting and selecting a predetermined number of images having the predetermined angle or less from the plurality of candidates based on the value counted for each candidate, and setting the direction and base point of the selected candidate path to the path It is characterized by determining as a direction and a base point.
According to this configuration, the same effect as in the fourth aspect can be obtained.

[Invention 25] The image layout method of Invention 25 is the invention 23, in which
The path direction determination step sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired in the image direction acquisition step, find the inner product of the vector of the tangential direction of the path at the temporary placement position of the image and the direction vector of the image, or the number of images for which the calculated inner product is negative, or Count the number of images that are positive, the number of images that are 0 or less, or the number of images that are 0 or more, and the number of images counted from among the plurality of candidates based on the value counted for each candidate Is selected, and the direction and base point of the selected candidate path are determined as the direction and base point of the path.
According to this configuration, the same effect as in the fifth aspect can be obtained.

[Invention 26] The image layout method according to Invention 26 is the invention 23, in which
The path direction determination step sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired in the image direction acquisition step, calculate an angle formed by the tangential direction of the path at the temporary arrangement position of the image and the direction of the image, obtain a sum of the angles calculated for all images, Based on the sum of angles obtained for each candidate, the one having the predetermined smallest sum of the angles is selected from the plurality of candidates, and the direction and base point of the selected candidate path are determined as the direction and base point of the path. It is characterized by doing.
According to this configuration, the same effect as in the sixth aspect is obtained.

[Invention 27] The image layout method according to Invention 27 is the invention 23, in which
The path direction determination step sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, calculate the angle formed by the tangential direction of the path and the direction of the image at the temporary arrangement position of the image, to obtain the maximum value of the angle calculated for all images, Based on the maximum value of the angle obtained for each candidate, the one with the predetermined maximum value of the angle is selected from the plurality of candidates, and the path direction and base point of the selected candidate are selected as the direction of the path and It is determined as a base point.
According to this configuration, the same effect as in the seventh aspect can be obtained.

[Invention 28] The image layout method according to Invention 28 is the method according to any one of Inventions 22 to 27,
The base point is a start point where the image is first arranged, an end point where the image is arranged last, or an intermediate point between the start point and the end point.
According to this configuration, the same effect as in the eighth aspect can be obtained.

[Invention 29] The image layout method of Invention 29 is the invention 28, in which
When the path is an annular path, after the image is arranged by the image arrangement step, a route other than the route connecting the start point to the end point along the determined path direction is deleted.
According to this configuration, the same effect as in the ninth aspect can be obtained.

[Invention 30] An image layout method according to Invention 30, in any one of Inventions 21 to 29,
When it is determined that there is an interval between the images after the images are arranged in the image arranging step, another image or illustration prepared in advance is inserted above or around the path.
According to this configuration, the same effect as in the tenth aspect can be obtained.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[First Embodiment]
The first embodiment will be described with reference to FIGS.
FIG. 1 is a block diagram showing the configuration of the image layout apparatus of the present invention.
The image layout apparatus 10 of the present invention selects an image / basic path selection unit 1 for selecting an image to be laid out and a basic form of the path, and the image / basic path selection unit 1 when laying out an image along the path. An image direction acquisition unit 2 that acquires the direction of an image, a path direction determination unit 3 that determines a path direction, a start point, and an end point based on the image direction obtained by the image direction acquisition unit 2, and a path direction determination unit And an image arrangement unit 4 that arranges the selected image on the basic shape of the path using the path direction, the start point, and the end point obtained in 3.

The image / basic path selection unit 1 selects an image to be laid out and checks the corresponding image, and selects and checks a desired path from various basic forms of paths.
The image direction acquisition unit 2 detects the direction of the image from a plurality of images having different shooting times. At that time, the direction of the image is obtained by identifying the characteristic object in the peripheral part and obtaining the moving direction, or detecting the movement of the object such as optical flow, or extracting the object in the image. Can do.

The path direction determination unit 3 determines the optimum path direction from the deviation angle between the image direction obtained by the image direction acquisition unit 2 and the path direction as a normal path in which the image is arranged, and a reference for arranging the image The optimum starting point and ending point are determined from the starting point candidate and the ending point candidate as the base point of the path.
The image placement unit 4 knows the shooting time of the image, and determines the image interval using the time information when arranging the image corresponding to the shooting time, or equidistant according to the number of images to be arranged. To place.

FIG. 2 is a block diagram illustrating a hardware configuration of the host terminal.
As shown in FIG. 2, the host terminal 100 includes a CPU 50 that controls operations and the entire system based on a control program, a ROM 52 that stores a control program for the CPU 50 in a predetermined area, and data read from the ROM 52 and the like. And a RAM 54 for storing calculation results required in the calculation process of the CPU 50 and an I / F 58 for mediating input / output of data to / from an external device. These are signals for associating data. The buses 59 are connected to each other so as to be able to exchange data.

The I / F 58 includes, as external devices, an input device 60 such as a keyboard and a mouse that can input data as a human interface, a storage device 62 that stores data, tables, and the like as files, and a screen based on image signals. Is connected to a display device 64.
FIG. 3 is a diagram showing an album mount on which a plurality of photographic images are temporarily arranged along a path.

  As shown in the figure, photographic images 101, 102, 103, 104, and 105 of five car races are arranged along a path 111 imitating a race course. The direction of pass 111 is the direction in which the race car runs on the race course. Further, the photographic images 101, 102, 103, 104, 105 are taken by a plurality of cameras and include those having different shooting directions. In other words, the direction in which the race car is facing differs depending on whether it is a photographic image taken from the outside of the race course or a photographic image taken from the inside of the race course. As described above, when the images having the respective directions are arranged, the path and the directionality of the images do not match each other. That is, if the photographic images 101, 102, 103, 104, and 105 of the car race of a complicated course are simply arranged in the path 111 such as a track, the direction of the photographic images 101, 102, 103, 104, and the path 111 Does not match, so it lacks power.

  Further, in order to match the direction of the photographic images 101, 102, 103, 104, and 105 with the path 111, it is necessary to prepare all the paths of the car race course, which is practically impossible. . Furthermore, there are many fields other than car races, such as field athletic courses and ski cross country courses, so it is virtually impossible to prepare all these passes. is there.

  In the present embodiment, the image direction acquisition unit 2 acquires the direction of each image to be laid out. Next, the path direction determination unit 3 sets a plurality of path direction and base point combination candidates, and temporarily arranges images on the path based on the candidate path direction and base point for each candidate. Based on the direction acquired by the acquisition unit 2, the angle formed between the tangent direction of the path at the temporary arrangement position of the image and the direction of the image is calculated, and the number of images in which the calculated angle is equal to or smaller than a predetermined angle is counted. Based on the value counted for each, a plurality of candidates having a predetermined number of images having a predetermined angle or less are selected, and the direction and base point of the selected candidate path are determined as the path direction and base point. The base point is a reference point for arranging the image, and includes a start point or an end point.

FIG. 4 is a flowchart showing the overall operation of the image layout apparatus of the present invention.
The image layout process is a process realized as the image / basic path selection unit 1, the image direction acquisition unit 2, the path direction determination unit 3, and the image arrangement unit 4, and when executed in the CPU 50, it is shown in FIG. As shown, the following steps are taken. First, the image / basic path selection unit 1 selects an image to be laid out and a basic form of a path (step S301). Subsequently, the image direction acquisition unit 2 acquires the direction of each selected image (step S302), and the path direction determination unit 3 determines the optimal path direction, start point, and end point (step 303). The placement unit 4 places each image on the basic shape of the path based on the optimum path direction, start point, and end point (step S304).

Hereinafter, the contents of each step will be described in detail.
First, a method (step S301) for selecting an image and a basic form of a path by the image / basic path selection unit 1 will be described with reference to FIGS.
FIG. 5 is a diagram illustrating an example of an image selection screen.
In this example, a total of six images are arranged. That is, three images 101, 102, and 103 are sequentially arranged from the upper left to the lower left of the drawing, and then three images 106, 104, and 105 are sequentially arranged from the upper right to the lower right of the drawing, and photographs taken in time series are arranged. Check boxes 401 to 406 are provided at the upper left of each image. The user selects an image to be laid out, and the check boxes 401 to 406 are marked with an X. Here, since the check boxes 401, 402, 403, 405, and 406 are marked with ×, the upper left image 101, the left middle image 102, the lower left image 103, the right middle image 104, and the lower right image A total of 105 images are selected.

FIG. 6 is a diagram illustrating an example of a screen for selecting a basic form of a path.
In this example, a total of 6 paths are arranged as a basic form of the path. That is, three paths 511, 512, and 513 are arranged in order from the upper left to the lower left of the figure, and then three paths 514, 515, and 516 are arranged in order from the upper right to the lower right of the figure, and check boxes 501- 506 is provided, the user selects an image to be laid out, and the check boxes 501 to 506 are marked with a cross. Here, since the check box 501 is marked with an X, the ellipse that is the upper left path 511 is selected.

Next, a method (step S302) of acquiring the image direction by the image direction acquisition unit 2 will be described with reference to FIGS.
FIG. 7 is a diagram illustrating a method for acquiring the direction of an image (a method designated by the user). In this example, the direction of the image is designated by the user clicking the mouse. (A) is an example of designating in the horizontal direction, and (b) is an example of designating in an oblique direction.

  As shown in the figure, when the user designates the directions of the photographic images 611 and 612 using a mouse or the like, block arrows 602 and 604 are displayed. Here, the position designated by the user is the center position of the block arrows 602 and 604. The direction from the center position of the photographic images 611 and 612 toward the center position of the block arrows 602 and 604 is set as the photographic image directions 601 and 603. For example, when the user designates as shown in FIG. 7A, the direction of the photographic image is the horizontal left direction as shown by an arrow 601, and when the user designates as shown in FIG. The direction of the image is the upper left direction as indicated by an arrow 602.

  FIG. 8 is a diagram illustrating a method for obtaining the direction of an image (a method based on the moving direction of a characteristic object). This example is the use of a characteristic object representing the motion of an image, and the direction of the image is detected by identifying the motion direction of the image. The movement of the image is identified from a plurality of images having different shooting times. The image may be taken with a camera, or may be an image cut out from a video at predetermined time intervals. The direction of the image is obtained by extracting a background object, for example, a characteristic object in the peripheral portion and obtaining the moving direction thereof.

  FIG. 8A and FIG. 8B are images taken at predetermined time intervals in the car race. The photographic image shown in FIG. 8B is a photographic image taken at a time when a predetermined time has elapsed from the time when the photographic image shown in FIG. In the photographic images of FIGS. 8A and 8B, there are areas 701 and 705 (hereinafter referred to as “photographing object”) indicating a race car that is a photographing target at the center. Here, in the photographic image, an area other than the area of the shooting target object is referred to as a background area. The background region of the photographic image in FIG. 8A is divided into small regions by color or the like, and a small region 703 having distinct edges from the surroundings among the plurality of divided small regions is distinct. Detect as an object.

  FIG. 8C and FIG. 8D are diagrams showing the results of extracting characteristic objects from the background (peripheral part). In FIG. 8C, the background region of the photographic image in FIG. 8A is divided into small regions by color or the like, and a small region 703 with distinct edges and distinct features is extracted and characteristic. Shown as an object. For FIG. 8B, as in the case of FIG. 8A, the object is extracted by dividing the area by color or the like, and the characteristic object and color extracted in FIG. 8C are extracted. Then, an object having a close feature amount such as a shape and close in space is extracted as a feature object in FIG. FIG. 8D shows a characteristic object detected from the photographic image in FIG. FIG. 8D shows an area 707 as a characteristic object.

  FIG. 8E is a diagram comparing the spatial position of the characteristic object in FIG. 8C with the spatial position of the characteristic object in FIG. As shown in FIG. 8E, the characteristic object moves from the region 703 to the region 707 as time passes. That is, as indicated by the block arrow 709, it can be seen that the characteristic object movement is in the right direction and the background is moving in the right direction. Therefore, it can be seen that the moving direction of the object to be imaged or the moving direction of the photographer is the left direction opposite to the background. Thereby, it is determined that the direction of the image is the left direction, and the direction of the photographic image is determined. As described above, when images having adjacent shooting times are used, it is easy to associate objects, and it is relatively easy to detect motion. Therefore, it is particularly effective to cut out images from the video at appropriate time intervals.

Next, a method for acquiring the direction of the image using the movement of the image (using the optical flow) will be described.
It is also possible to acquire an image using a method that generally detects the movement of an object such as an optical flow. That is, when the movement direction of the object in the center is known by the optical flow, the movement direction is the direction of the image, and when the movement direction of the peripheral part is known without knowing the movement direction of the central object, the peripheral part The direction opposite to the moving direction is the image direction.

  Next, a method for obtaining the direction of the image by obtaining the directionality from the image will be described. It is also possible to obtain the direction by extracting an object from a predetermined image. For example, by extracting a face object and making the direction the face is facing to be the direction that the image is holding, use an object that has a direction, or extract an object with an acute shape and change the pointed direction Or the direction of the image.

Next, the path direction determination method (step 303) according to the first embodiment of the present invention will be described with reference to FIGS.
FIG. 9 is a diagram illustrating a method for determining the path direction, the start point, and the end point by the path direction determination unit according to the first embodiment of the present invention.
9 (a) to (c) on the left side of the same figure are counterclockwise, and FIGS. 9 (d) to (f) on the right side are clockwise, and FIGS. 9 (a) and 9 (d) have a starting point at the top. 9 (b) and 9 (e) show the case where the starting point is on the left, and FIGS. 9 (c) and 9 (f) show the case where the starting point is on the right. Accordingly, FIG. 9A shows a case where the start point is on the left, FIG. 9B shows a case where the start point is on the left, and FIG. 9C shows a case where the start point is on the left. FIG. 9D shows the case where the start point is on the right, FIG. 9E shows the case where the start point is on the left, and FIG. 9F shows the case where the start point is on the right. The image starts from the image 101, and a total of five images 102, 103, 104, and 105 are arranged.

First, the direction of the path is determined. The basic shape of the path has a starting point candidate and a direction candidate. Temporary placement is performed at intervals determined above by combining these candidates, and the direction, starting point, and direction of the path are selected so that the image direction is naturally placed along the direction of the path.
Here, in order to determine whether the direction of the image and the direction of the path are the same, the inner product of the direction of the image and the direction of the path at the image arrangement position is taken, and whether it is 0 or more, that is, ± 90. When counting whether or not the difference is less than or equal to degrees, if the start point is on the left in FIG. 9A, the images 101, 104, and 105 are three, so that 3/5, as shown in FIG. 9B. If the start point is left and the start point is left, there are two images 102 and 105, so 2/5, and if the start point is right and the start point is right, there are two images 103 and 104. 2/5, when the starting point is on the right in FIG. 9 (d), the images 102 and 103 are two, so 2/5, when the starting point is on the left in FIG. 9 (e), Since the two images 102 and 105 are 2/5, when the start point is right in FIG. 9F, the images 103 and 104 Since One in which a 2/5, the results show that most often the starting point is above about the left in FIG. 9 (a). Therefore, here, the case where the start point is on the left in FIG. 9A is selected.

After determining the path direction, start point, and end point as described above, an image is arranged based on the path direction, start point, and end point (step 304).
This makes it possible to match the directionality of the image with the direction of the path by a simple method. In addition, it is possible to easily make paths corresponding to various courses without bothering the user.
FIG. 10 is a diagram illustrating a state in which the path between the end point and the start point is deleted after the path direction, the start point, and the end point are determined in FIG. By deleting the path between the end point and the start point, it is not necessary to find out which is the start point image or which is the end point image, and it is easy to determine.
[Second Embodiment]
Next, with reference to FIG. 11 to FIG. 14, a path direction determination method according to the second embodiment of the present invention will be described.

Compared with the first embodiment, the present embodiment is different from the first embodiment only in the path direction, the start point, and the end point of step S303. Since step S302 for acquiring the direction is the same as that in the first embodiment described above, description thereof is omitted.
FIG. 11 is a diagram illustrating a method for determining the path direction by the path direction determination unit according to the second embodiment of the present invention.

  This example is a diagram illustrating a method of selecting a direction, a start point, and an end point based on a sum of shift angles between an image direction and a path direction. As shown in the figure, when calculating the sum of the deviation angle between the image direction and the path direction, and selecting the direction with the small sum, the start point, and the end point, the deviation angle between the path of each image and the image direction is Find the smaller angle around or around the left and find the sum. In this example, if the direction of the image 102 is 1001 and the direction of the path 1004 is 1002, the smaller angle in the clockwise or counterclockwise direction is the angle indicated by 1003 and is 120 degrees in this example.

  When the sum of the misalignment angles in each diagram of FIG. 9 is obtained by the method of determining the direction, start point, and end point of the path shown in FIG. 11, if the start point is on the left in FIG. Is 0 degrees, image 102 is 120 degrees, image 103 is 180 degrees, image 104 is 0 degrees, image 105 is 60 degrees, and the sum is 0 + 120 + 180 + 0 + 60 = 360 degrees. In this case, the image 101 is 90 degrees, the image 102 is 0 degrees, the image 103 is 150 degrees, the image 104 is 150 degrees, the image 105 is 0 degrees, and the total is 90 + 0 + 150 + 150 + 0 = 390 degrees, which is the left side of FIG. When the start point is right, the image 101 is 90 degrees, the image 102 is 180 degrees, the image 103 is 30 degrees, the image 104 is 30 degrees, the image 105 is 180 degrees, and the total is 90 + 180 + 30 + 30 + 18 = 510 degrees, and when the start point is on the right in FIG. 9D, the image 101 is 180 degrees, the image 102 is 60 degrees, the image 103 is 0 degrees, the image 104 is 180 degrees, and the image 105 is 120 degrees. , The sum is 180 + 60 + 0 + 180 + 120 = 540 degrees. When the start point is around the right in FIG. 9E, the image 101 is 90 degrees, the image 102 is 0 degrees, the image 103 is 150 degrees, the image 104 is 150 degrees, 105 is 0 degree, and the total is 90 + 0 + 150 + 150 + 0 = 390 degrees. When the start point is right in FIG. 9F, the image 101 is 90 degrees, the image 102 is 180 degrees, the image 103 is 30 degrees, and the image 104 Is 30 degrees, the image 105 is 180 degrees, and the sum is 90 + 180 + 30 + 30 + 180 = 510 degrees. Accordingly, the case where the start point is on the left in FIG. 9A where the sum is the smallest and the most consistent is selected is selected. This selection has the same result as the above-described method of taking the inner product of the image direction and the path direction at the image arrangement position.

  According to this configuration, each direction, start point, and end point can be positioned, which is effective when selecting in combination with other factors. For example, in FIG. 9, if the start point is on the left in FIG. 9A, the illustration for guiding the start point is on the left, and if the user does not like or cannot select FIG. FIG. 9B to FIG. 9E can be selected.

  Further, when the maximum value of the deviation angle in each figure of FIG. 9 is obtained by the method of determining the direction, start point, and end point of the path shown in FIG. 11, the start point is on the left in FIG. 9A. When the start point is 180 degrees in the image 103, the left rotation in FIG. 9B, and the start point is 150 degrees in the images 103 and 104, and in the left rotation in FIG. 180 degrees, when the start point is on the right in FIG. 9D, the images 101 and 104 are 180 degrees, and when the start point is on the right, the image 103 and 104 are 150 degrees, FIG. When the starting point is right around 9 (f), the images 102 and 105 are 180 degrees. Therefore, the case where the start point is the left in the left rotation of FIG. 9B where the maximum value is the smallest and the most consistent is selected, or the case where the start point is the left in the right rotation of FIG. 9E is selected. This selection is different from the above-described method of taking the inner product of the image direction and the path direction at the image placement position or the method using the sum, but the angle between the path direction and the image direction at each placement position. A small deviation is selected. Which of the two candidates is used is determined by rules such as giving priority to the clockwise direction or giving priority to the one that passes the top first.

As described above, in the first embodiment, the direction of the path when the position of the image is arranged based on information such as the shooting time is determined. However, as in the second embodiment, the path is on the path. It is also possible to determine the direction of the path by having points that can be arranged at the same position and arranging them so that the image and the direction of the path match.
Note that the user may specify the start point position and the loop direction using a GUI (graphical user interface).

  In FIG. 12, X marks (1) to (16) on the image path are points at which images can be arranged at predetermined arrangement points. The direction and starting point position are given in the same manner as in FIG. That is, in FIG. 12, the first image 101 is arranged on the start point (1) around the left, and the misalignment angle between the image and the path direction at each arrangement point on the path is obtained for each of the other images, and these are combined. The direction of the path when the deviation angle between the image and the path direction is minimized is determined.

  In this example, the deviation angle between the direction of the image 101 arranged at the arrangement point (1) and the path direction is 0 degree, the deviation angle between the direction of the image 102 arranged at the arrangement point (4) and the path direction is 120 degrees, and the arrangement point. The deviation angle between the direction of the image 103 arranged in (6) and the path direction is 120 degrees, the deviation angle between the direction of the image 104 arranged at the arrangement point (10) and the path direction is 0 degrees, and the arrangement point (16) is arranged. Since the deviation angle between the direction of the image 105 and the path direction is 0 degrees, the total is 0 + 120 + 120 + 0 + 0 = 240 degrees. This calculation is obtained for all directions and starting points, and the direction of the path when it is minimized is determined.

FIG. 13 is a flowchart showing the operation of the second exemplary embodiment of the present invention.
As the image placement points on the path, the placement points (1) to (16) indicated by the crosses in FIG. 12 are set, and the images 101 to 105 are placed at any of these points.
First, for the first image 101, the direction shift at the starting point is obtained in a later step, so that the placement points (1) to (16) are set for the images 102, 103, 104, and 105 other than the first image 101. The deviation angle between the image direction and the pass direction is obtained (step S1301), and it is determined whether or not the deviation angle between the image direction and the pass direction at each arrangement point is obtained for all directions and starting points (step S1302). If there is one for which the deviation angle is not obtained, the process proceeds to step S1303 to step S1305.

  In steps S1303 to S1305, the angle between the direction of the first image 101 and the path direction at the start point is obtained (step S1303), and the images 102 to 105 other than the first image 101 and the arrangement points (1) to (16) are combined. Thus, the sum of the direction deviation angles is obtained (step S1304). For example, in FIG. 12, the arrangement point (1) is the start point, and the angle between the direction of the image 101 and the path direction at the arrangement point (1) is obtained, and the images 102 to 105 other than the first image 101 and the arrangement points (2) to ( The sum of the direction deviation angles is obtained by the combination of 16). Next, an optimum arrangement point of the image is obtained based on a combination with a small sum of direction deviations (step S1305).

When the optimum arrangement point of the image is obtained, the process returns to step S1302, and steps S1303 to S1305 are repeated. When the optimum arrangement point of the image is obtained for all directions and start points, the process proceeds to steps S1306 and S1307.
In step S1306, the optimum direction and starting point are obtained from the optimum arrangement points obtained for all directions and starting points, and in step S1307, an excessively empty part on the path is checked, and an image or a peripheral part is checked. Insert an illustration. This has the advantage that the path is not unnatural, and at the same time the path direction can be seen at a glance.

FIG. 14 is a diagram illustrating an example of a method for obtaining a shift angle at each arrangement point of an image.
First, since the image 101 is the start point, it is placed at the placement point (1) in FIG.
Next, as shown in step S1301 of FIG. 13, for the images 102, 103, 104, and 105 other than the first image 101, the deviation between the image direction and the path direction at each placement point is obtained. In this example, as an example, the shift angle at each arrangement point for the image 102 is obtained.

  First, 180 degrees for arrangement point (2), 150 degrees for arrangement point (3), 120 degrees for arrangement point (4), 90 degrees for arrangement point (5), arrangement point (6 ) Is 60 degrees, arrangement point (7) is 30 degrees, arrangement point (8) is 0 degrees, arrangement point (9) is 0 degrees, and arrangement point (10) is 0 degrees. 30 for arrangement point (11), 60 degrees for arrangement point (12), 90 degrees for arrangement point (13), 120 degrees for arrangement point (14), arrangement point (15) In the case of (1), it is obtained as 150 degrees, and in the case of the arrangement point (16), it is obtained as 180 degrees. In this way, the sum of the shift angles is obtained from the results obtained for the images 103, 104, and 105, and the combination of the placement point and the image.

  For example, when the image 102 is the arrangement point (2), the image 103 is the arrangement point (3), the image 104 is the arrangement point (4), and the image 105 is the arrangement point (5), 180 degrees + 30 degrees + 120 degrees + 90 degrees = 420. Degree. When the image 102 is the arrangement point (4), the image 103 is the arrangement point (6), the image 104 is the arrangement point (10), and the image 105 is the arrangement point (16), 120 degrees + 120 degrees + 0 degrees + 0 degrees = 240. Degree.

In the example of FIG. 12, the total combination of images and arrangement points is a combination in which four arrangement positions are extracted from 15 arrangement points without worrying about the order, so that 15 · 14 · 13 · 12/4 · 3 2 · 1 = 1365 ways × the number of starting points, and the respective deviation angles are calculated.
As a method of determining the arrangement position, a method of determining an optimal position in the order of images, a method of determining an optimal position in order from the end of the image, and the like can be considered. For paths with many arrangement positions, the method of deciding in order instead of the total combination is suitable. However, when there are few total combinations, the optimal solution can be obtained by brute force.

As described above, the direction and the arrangement position are obtained. After that, the path between the start point and the end point is deleted. In addition, an image or an illustration prepared in advance is arranged at a place where there is a gap between images as a result of determining the arrangement position.
FIG. 15 is a diagram illustrating a state where the path between the end point and the start point is deleted after the path direction, the start point, and the end point are determined in FIG. By deleting the path between the end point and the start point, it is possible to determine at a glance which image is the start image or which is the end image, and the appearance is neat.

  In addition, as a result of determining the arrangement position, if there is a gap between images, an image or an illustration prepared in advance is arranged on the path or in the periphery of the path. In this example, since there is a gap between the image 103 and the image 104 and between the image 104 and the image 105, illustrations 1501 and 1502 are arranged around the path. Whether or not there is an interval is determined by whether or not there are a predetermined number or more of non-arranged points. For the images or illustrations prepared in advance, those that match the direction of the path, such as illustrations 1051 and 1052, are selected, or the directions are adjusted so that the directions match. An illustration 1501 is arranged at the periphery of the linear path and shows the direction of the path. The illustration 1502 is arranged at the periphery of the curved path, which also shows the direction of the path. Further, the image or illustration is used to replenish a portion having a gap and eliminate a sense of incongruity, and does not need to have any particular directionality.

  In this way, by determining the path direction using the directionality of the image, the directionality of the image and the direction of the path can be matched, and the image can be arranged relatively appropriately with respect to the direction of the path. Can match the directionality of the path and the directionality of the path, and can express a natural flow. In addition, it is possible to easily make paths corresponding to various courses without bothering the user.

  Further, in the above-described embodiment, the case where the control program stored in advance in the ROM 72 is executed when executing the processing shown in the flowchart of FIG. 4 has been described. The program may be read from the storage medium storing the program into the RAM 74 and executed. Alternatively, the program may be acquired from a network.

  Here, the storage medium is a semiconductor storage medium such as RAM or ROM, a magnetic storage type storage medium such as FD or HD, an optical reading type storage medium such as CD, CDV, LD, or DVD, or a magnetic storage type such as MO. / Optical reading type storage media, including any storage media that can be read by a computer regardless of electronic, magnetic, optical, or other reading methods.

It is a block diagram which shows the structure of the image layout apparatus of the 1st Embodiment of this invention. It is a block diagram which shows the hardware constitutions of a host terminal. It is a figure which shows the album mount by which the some photographic image was temporarily arrange | positioned along the path | pass. 3 is a flowchart illustrating an overall operation of the image layout apparatus according to the first embodiment. It is a figure which shows an example of the screen of an image selection. It is a figure which shows an example of the screen of selection of the basic form of a path. It is a figure which shows the method (method which a user designates) which acquires the direction of an image. (A) is an example of designating in the horizontal direction, and (b) is an example of designating in an oblique direction. It is a figure explaining the direction of the photographic image based on the moving direction of a characteristic object. (A) and (b) are images with different shooting times, (c) and (d) are images obtained by extracting characteristic objects, and (e) is a diagram comparing (c) and (d). It is a figure which shows the path | route direction determination method which concerns on the 1st Embodiment of this invention. (A) is left-handed and the starting point is up, (b) is left-handed and the starting point is left, (c) is left-handed and the starting point is right, and (d) is right-handed and the starting point is above In this case, (e) shows a case where the start point is clockwise and the start point is left, and (f) shows a case where the start point is clockwise and the start point is right. It is the figure which deleted the path | pass from the image of an end point to the image of a start point. It is a figure which shows the principle of the path | route direction determination method which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the path direction determination method. It is a flowchart which shows the path | pass direction determination method. It is a table | surface which shows an example of the method of calculating | requiring the shift | offset | difference angle in each arrangement | positioning point of each image. It is the figure which deleted the path | pass from the image of an end point to the image of a start point.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Image / basic path selection part, 2 ... Image direction acquisition part, 3 ... Path direction determination part, 4 ... Image arrangement | positioning part, 10 ... Image layout apparatus,

Claims (9)

An image layout device that lays out an image along a path,
Path direction determining means for determining a direction of a path that is a route for arranging the image;
Image placement means for placing the image on the path based on the direction of the path determined by the path direction determination means;
Image direction acquisition means for acquiring the direction of the image,
The path direction determining means determines the path direction and a base point of a path serving as a reference for arranging the images based on the direction of the image acquired by the image direction acquiring means. In claim 1,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, an angle formed between the tangential direction of the path at the temporary arrangement position of the image and the direction of the image is calculated, and the number of images where the calculated angle is equal to or less than a predetermined angle is calculated. Counting and selecting a predetermined number of images having the predetermined angle or less from the plurality of candidates based on the value counted for each candidate, and setting the direction and base point of the selected candidate path to the path An image layout apparatus that determines the direction and the base point. In claim 1,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, obtain the inner product of the vector of the tangential direction of the path and the direction vector of the image at the temporary arrangement position of the image, the number of images for which the obtained inner product is negative, or Count the number of images that are positive, the number of images that are 0 or less, or the number of images that are 0 or more, and the number of images counted from among the plurality of candidates based on the value counted for each candidate An image layout apparatus that selects a path having the largest number of paths and determines the direction and base point of the selected candidate path as the direction and base point of the path. In claim 1,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, calculate the angle formed by the tangent direction of the path and the direction of the image at the temporary arrangement position of the image, find the sum of the angles calculated for all images, Based on the sum of angles obtained for each candidate, the one having the predetermined smallest sum of the angles is selected from the plurality of candidates, and the direction and base point of the selected candidate path are determined as the direction and base point of the path. An image layout apparatus. In claim 1,
The path direction determining means sets a plurality of combinations of the path direction and the base point, and temporarily arranges the image on the path based on the path direction and base point of the candidate for each candidate, Based on the direction acquired by the image direction acquisition means, calculate the angle formed by the tangential direction of the path and the direction of the image at the temporary arrangement position of the image, to obtain the maximum value of the angle calculated for all images, Based on the maximum value of the angle obtained for each candidate, the one with the predetermined maximum value of the angle is selected from the plurality of candidates, and the path direction and base point of the selected candidate are selected as the direction of the path and An image layout apparatus characterized by being determined as a base point. In any one of Claims 1-5,
The image layout apparatus according to claim 1, wherein the base point is a start point at which the image is first arranged, an end point at which the image is arranged last, or an intermediate point between the start point and the end point. In claim 6,
When the path is a circular path, an image layout is characterized in that after the image is arranged by the image arrangement means, a route other than the route connecting the start point to the end point along the determined path direction is deleted. apparatus. In any one of Claims 1-7,
After the images are arranged by the image arrangement means, it is determined whether or not there are a predetermined number or more points arranged between the images. An image layout apparatus for inserting another image or illustration. An image layout program that lays out an image along a path,
Path direction determining means for determining the direction of a path to be a path for arranging the image, image arranging means for arranging the image on the path based on the direction of the path determined by the path direction determining means, and The computer executes processing realized as an image direction acquisition unit that acquires the direction of the image , and the path direction determination unit determines the path direction and the image based on the image direction acquired by the image direction acquisition unit. An image layout program for determining a base point of a path serving as a reference for arranging the images.

Images