JP4212430B2 - Multiple image creation apparatus, multiple image creation method, multiple image creation program, and program recording medium - Google Patents

Description

  The present invention relates to a multiple image creation apparatus, a multiple image creation method, a multiple image creation program, and a program recording medium, and more particularly, a plurality of images (continuous shooting) taken by a digital still camera (hereinafter abbreviated as a digital camera) or a mobile phone with a camera. The present invention relates to a multiple image creation apparatus, a multiple image creation method, a multiple image creation program, and a program recording medium that multiplex-synthesizes a plurality of frame images cut out from a camera image and a movie to create one still image.

  Conventionally, when a single still image is created by superimposing a plurality of images, an average value is obtained by averaging pixel values at the same position of each of the plurality of images over a plurality of images, and the average value is subjected to multiple still images. A method of setting the pixel value at the same position in the image is common. In some cases, a technique using image processing such as recognizing a monitoring target in an image and tracking the movement of the monitoring target by object tracking or edge tracking may be applied. However, the latter method using image processing techniques such as object tracking and edge tracking requires high-level image processing, requires a high-speed arithmetic processing unit (CPU) and dedicated image processing hardware. This is difficult to achieve in fields where miniaturization and weight reduction such as mobile phones with cameras are essential.

On the other hand, as an improvement in calculating the former average value, Japanese Unexamined Patent Application Publication No. 2002-298143 “Exercise Motion Analysis Support Device” disclosed in Patent Document 1 and Japanese Patent Application Laid-Open No. 10-290450 disclosed in “Patent Document 2”. The “creation method” has been proposed.
The technique described in Patent Document 1 first generates a reference image as a reference by obtaining an average of a plurality of images, compares the reference image with each processing target image, and the difference in luminance information between the two is determined. This is a technology that superimposes the largest part on the reference image as a part with motion that is greater than or equal to a preset threshold value, and the image part showing the trajectory with motion that rarely occurs is buried. The purpose is to prevent it.

  The technique described in Patent Document 2 calculates an average value and a standard deviation of pixel values at the same pixel position of a plurality of images, and uses a weighting factor that is set in advance according to the number of images to be overlaid. A pixel having a large deviation, that is, a pixel having a large variation between images is given a large weighting factor, a small pixel is given a small weighting factor, and a weighting result obtained by multiplying the given weighting factor and the pixel value is obtained. It is a technique of adding and creating a multiple image, and it is intended to prevent the average value of pixel values in a portion where there is a movement from becoming too small to be seen when superimposed.

JP 2002-298143 A Japanese Patent Laid-Open No. 10-290450

However, when the motion trajectory is superimposed on a single screen and displayed as a multiple still image, all the conventional multiple image creation methods, including the technologies described in the above-mentioned patent documents, are all shown below. For these reasons, it is not possible to obtain multiple still images that are overlaid for general and simple use, and it is essential to reduce the size and weight of digital cameras and camera-equipped mobile phones with low memory capacity and low processing performance. It was impossible to apply to the field. Such circumstances will be described with reference to FIG. FIG. 5 is a schematic diagram for explaining an example of processing for creating one still image by superimposing three images. In FIG. 5, 50, 51, and 52 are the three original images. The portions 54 ₀ , 54 ₁ , and 54 ₂ represented by white backgrounds of the original images 50, 51, and 52, respectively, have no movement (hereinafter referred to as the background 54), and the portions 55 ₀ , 55 ₁ , 55 represented by diagonal lines Reference numeral ₂ denotes a moving part (hereinafter referred to as an object 55).

  Reference numeral 53 denotes a multiple still image created by superimposing three original images 50, 51, and 52. In the multiple still image 53, in general, a portion 56 where the backgrounds 54 of all three original images 50, 51, 52 overlap, and the background of any two original images 50, 51, 52 of the three original images 50, 51, 52. 54 and a portion 57 where the remaining one original image object 55 overlaps, a background 54 of any one of the three original images 50, 51, and 52 and the remaining two original images. A portion 58 where the object 55 overlaps and a portion 59 where all the three original images 50, 51, 52 overlap the object 55 appear.

  Here, when a method for obtaining a simple average value of the three original images 50, 51, 52 is applied, for example, the value of the object 55 is small in the portion 57 where two backgrounds and one object overlap. Become. Thus, there is a problem that the object 55 becomes difficult to see as the number of original images to be combined increases.

  In Japanese Patent Laid-Open No. 2002-298143 disclosed in Patent Document 1, when the number of images is small, there is a possibility that the background 54 opposite to the portion of the moving object 55 is selected. In the example of the schematic diagram shown in FIG. 5, the average value calculated as the reference image is close to the color of the object 55 in the portion 58 where one background and two objects overlap. Therefore, the background 54 is larger than the object 55 in the difference from the average value that is the reference image. Accordingly, in the portion 58, the background 54 is selected instead of the object 55.

  Further, in Japanese Patent Laid-Open No. 10-290450 shown in Patent Document 2, when the number of original images to be combined is k, each of all the number k applied for each pixel position (i, j) is provided. The sum Σw (i, j; k) of the weighting factors w (i, j) must be “1” and it is determined whether or not the variation (standard deviation) is within a certain threshold range. Since the value of the weighting factor w (i, j) to be selected differs depending on the number k, a table in which the weighting factor w (i, j; k) depending on the number k of the original images is set in advance. It is necessary to have. Therefore, it is difficult to apply to a device that does not have a very large memory, such as a digital camera or a mobile phone that requires a reduction in size and weight.

  In view of the circumstances as described above, a single multiple still image is created by superimposing a plurality of images even in a field where a small size and light weight are essential, such as a portable device such as a digital camera or a mobile phone with a camera. In order to make it possible, a versatile and simple multiple image creation device or multiple image creation that can be realized even with a device that cannot be equipped with a large memory and has low arithmetic processing performance (CPU performance) A method is needed.

  The present invention has been made in view of such circumstances, and each pixel information located at the same position in a plurality of original images is extracted and generated from distance information indicating a gap between the pixel information and each pixel information. Based on information such as luminance information and color information, each extracted pixel information is divided into several clusters, and an average value for each element forming each pixel information is calculated as a cluster average value Ai for each cluster. Further, by calculating the average of the calculated cluster average value Ai, or by calculating the total weighted with the calculated cluster average value Ai, the pixel information at the same position of the image to be multiplexed and synthesized is generated. Therefore, even a device that does not have a large memory or a high-performance arithmetic processing unit (CPU) can easily create multiple still images. It is aimed at.

  Furthermore, it is possible to set in advance a weighting coefficient for weighting each of the plurality of cluster average values Ai, which is used when taking the total weighted with the cluster average value Ai, so that the pixel information desired by the user (for example, It is an object to make it possible to create a multiple still image in which pixel information indicating human skin) is further emphasized.

  The first technical means includes storage processing means for storing a plurality of images in a storage section, and pixel information at the same position of each of the images stored in the storage section, and distance information indicating a gap between the pixel information A cluster dividing unit that divides each pixel information into a plurality of clusters based on: a cluster average value obtaining unit that obtains an average value of each element forming each pixel information as a cluster average value for each cluster; and An average image of cluster average values is taken, and a multiple image creating apparatus is provided that includes image generation means for generating pixel information at the same position of an image to be multiplexed and synthesized.

  The second technical means reads storage information means for storing a plurality of images in the storage section, and reads out pixel information at the same position of each of the images stored in the storage section. Information generating means for generating one-dimensional information consisting of one-dimensional arbitrary information represented by the color information, cluster dividing means for dividing each pixel information into a plurality of clusters based on the one-dimensional information, Cluster average value acquisition means for obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and pixel information at the same position of an image to be averaged and multiplexed and synthesized A multiple image creating apparatus including an image generating unit that generates the image as a feature of

  The third technical means reads out the storage processing means for storing a plurality of images in the storage unit, and reads out each color information of the pixel at the same position of each of the images stored in the storage unit, and based on each of the color information, Cluster dividing means for dividing the color information of each pixel into a plurality of clusters, cluster average value obtaining means for obtaining an average value of color information of each pixel as a cluster average value for each cluster, and the cluster average value It is characterized in that it is a multiplex image creating apparatus provided with image generating means for taking each average and generating as each color information of the pixel at the same position of the image to be multiplexed and synthesized.

  A fourth technical means is a storage processing means for storing a plurality of images in a storage unit, and reads out pixel information at the same position of each of the images stored in the storage unit, and distance information indicating a gap between the pixel information A cluster dividing unit that divides each pixel information into a plurality of clusters based on: a cluster average value obtaining unit that obtains an average value of each element forming each pixel information as a cluster average value for each cluster; The image forming means includes an image generation means for taking the sum total weighted with the cluster average value and generating as the pixel information at the same position of the image to be multiplexed and synthesized.

  According to a fifth technical means, in the multiple image creating apparatus according to the fourth technical means, each of the plurality of cluster average values used when the image generating means takes a sum total weighted with the cluster average values. A weighting coefficient setting unit capable of presetting a weighting coefficient for weighting to an arbitrary value is provided.

  A sixth technical means is a storage processing step for storing a plurality of images in a storage unit, and reads out pixel information at the same position of each of the images stored in the storage unit, and distance information indicating a gap between the pixel information A cluster dividing step of dividing each pixel information into a plurality of clusters based on the above, a cluster average value obtaining step for obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and An average image of cluster average values is taken, and an image generation step for generating the pixel information at the same position of the image to be multiplexed and synthesized is provided.

  The seventh technical means reads a storage processing step of storing a plurality of images in the storage unit, reads out pixel information at the same position of each of the images stored in the storage unit, and determines luminance information or any one based on the pixel information. An information generating step for generating one-dimensional information consisting of one-dimensional arbitrary information represented by the color information, and a cluster dividing step for dividing each pixel information into a plurality of clusters based on the one-dimensional information, A cluster average value obtaining step for obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and pixel information at the same position of an image to be multiplexed and averaged by the cluster average value And a method for generating a multiple image having an image generation step of generating as follows.

  The eighth technical means stores a plurality of images in a storage unit, reads out each color information of a pixel at the same position of each of the images stored in the storage unit, and reads each of the color information based on the color information. A cluster dividing step of dividing pixel color information into a plurality of clusters, a cluster average value obtaining step for obtaining an average value of color information of each pixel as a cluster average value for each cluster, and each of the cluster average values And an image generation step for generating each color information of the pixels at the same position of the image to be multiplexed and synthesized.

  A ninth technical means is a storage processing step for storing a plurality of images in a storage unit, and reads out pixel information at the same position of each of the images stored in the storage unit, and distance information indicating a gap between the pixel information A cluster dividing step of dividing each pixel information into a plurality of clusters based on the above, a cluster average value obtaining step for obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and a plurality of steps An image generation step of generating a weighted sum of the cluster average values and generating as pixel information at the same position of the image to be multiplexed and synthesized.

  According to a tenth technical means, in the multiple image creation method according to the ninth technical means, each of the plurality of cluster average values used when the image generation step takes a sum total weighted with the cluster average values. It has a weighting coefficient setting step in which a weighting coefficient for weighting can be set in advance to an arbitrary value.

  According to an eleventh technical means, a multiple image creation method according to any one of the sixth to tenth technical means is formed into a program logic to form a multiple image creation program configured as a program executed by a computer. And

  A twelfth technical means is a program recording medium for recording the multiple image creation program described in the eleventh technical means on a computer-readable recording medium.

In the present invention constituted by each technical means as described above, the following effects can be obtained.
That is, according to the present invention, each pixel information located at the same position of a plurality of original images is extracted, distance information indicating a gap between the pixel information, luminance information and color information generated from the pixel information, etc. Is divided into several groups (clusters) based on the one-dimensional information, and for each divided cluster, the average value of each element forming each pixel information, for example, R, G, B, is set as the cluster average value Ai. Calculating and calculating the average of the calculated cluster average value Ai, or generating the pixel information at the same position of the image to be multiplexed and synthesized by calculating the weighted sum of the calculated cluster average value Ai. Even devices that do not have a large memory or a high-performance processing unit (CPU) such as a digital camera or a mobile phone with a camera can Images can be created at high speed by simple calculation, and even if the number of original images increases, it is easy to obtain a clean multiple still image without depending on the number of original images. Can do.

  In addition, when taking the sum total weighted with the cluster average value Ai, it is possible to set in advance a weighting coefficient for weighting each cluster average value Ai, so that the cluster average value Ai desired by the user (for example, human skin) It is also possible to create a multiple still image in which a specific cluster (for example, color) having a cluster average value Ai) indicating more is emphasized.

  The present invention stores a plurality of original images in a storage unit by storage processing means (or storage processing step), and pixels at the same position of each original image stored in the storage unit by cluster dividing means (or cluster division step). The information is read, and each pixel information is divided into a plurality of clusters based on the distance information indicating the distance between each pixel information, and the cluster is obtained by the cluster average value acquisition means (or cluster average value acquisition step). For each of the elements forming each pixel information, for example, an average value for each color information of R, G, B is obtained as a cluster average value, and thereafter, a plurality of cluster average values are obtained by the image generation means (or image generation step). And is generated as the pixel information at the same position of the image to be multiplexed and synthesized without requiring a large memory. The Do calculation, and makes it possible to create a multiple still images.

  Further, the cluster dividing means (or cluster dividing step) is configured to generate information (or information) based on each pixel information instead of dividing each pixel information into a plurality of clusters based on distance information indicating a gap between pixel information. Each pixel information is divided into a plurality of clusters based on the one-dimensional information typified by luminance information or any color information generated by the generation step), thereby simplifying the calculation when dividing into the clusters. Alternatively, each pixel information may be divided into a plurality of clusters according to the color information constituting each pixel information, so that the calculation for dividing the information into clusters is further simplified. May be.

  Further, the image generation means (or image generation step) takes a sum total weighted by the plurality of cluster average values instead of generating pixel information of an image to be multiplexed and synthesized by taking an average of the cluster average values. A multiple still image in which pixel information of an image to be multiplexed and synthesized is generated, and a specific cluster (for example, color) having a cluster average value (for example, a cluster average value indicating human skin) desired by the user is more emphasized. May be created easily.

  Hereinafter, embodiments of a multiple image creating apparatus, a multiple image creating method, a multiple image creating program, and a program recording medium according to the present invention will be described with reference to the drawings taking four cases as examples.

(First embodiment)
First, the first embodiment will be described. FIG. 1 is an operation flowchart for explaining a first embodiment of a multiple image creation apparatus and multiple image creation method according to the present invention.
First, the storage process is started, a plurality of original image 10a ₁ by the operation of the storage processing step, ..., 10a _n are stored in the storage unit 10 (step S11). In this state, multi-image creation processing is activated, the desired plurality of original images 10a ₁ stored in the storage unit 10, ..., to create a multiple still images superposed 10a _n, as an output image 10b The operation to output is started. When multiple image creation processing is started, first, a desired plurality of original images 10a _1, ..., performs initial setting of the positional information 10a _n (i, j) (step S101).

Next, is started cluster dividing step of dividing each image information into a plurality of clusters (groups), first, the position information (i, j) each of the plurality of original images 10a ₁ specified by, ..., of 10a _n Pixel information P ₁ ,..., P _n at the same position is acquired (step S102). Here, as each element forming the pixel information P _k (k = 1,..., N), that is, color expression, any expression method such as RGB, YUV, YCC, etc. may be used in the present invention. In FIG. 2, the case where RGB color expression is used will be described.

Next, the original image 10a ₁ has been acquired, ..., pixel information P ₁ of 10a _n, ..., and P _n, based on the distance information indicating the distance between the original image each image information, the appropriate number Divide into clusters (step S103). Here, the algorithm for dividing into clusters based on the distance information between the image information can be realized using various algorithms as long as an appropriate distance function and threshold value are given. Is published.

For example, distance information between each pixel information of the pixels P0 and P1 is one of color models for quantifying the color information. The lightness L (Lightness) indicating the degree of white with respect to black and the range from green to red. As a distance function between the pixel information of the pixels P0 and P1, when expressed by converting to a Lab system composed of the element (a) and two color elements ranging from blue to yellow (b) Color difference: ΔE = sqrt {(L0−L1) ^ 2 + (a0−a1) ^ 2
+ (B0−b1) ^ 2}
It is conceivable to perform clustering using. In addition to this expression, it is also conceivable to perform clustering by calculating a distance function based on, for example, one of the following expressions using R, G, B information.
ΔE = sqrt {(R0−R1) ^ 2 + (G0−G1) ^ 2
+ (B0-B1) ^ 2}
Or
ΔE = | R0−R1 | + | G0−G1 | + | B0−B1 |

  For each cluster obtained by applying any of the cluster division techniques as described above, the average value of each element forming the pixel information is set as the cluster average value Ai for each cluster in the cluster average value acquisition step. The obtained cluster average value is calculated (step S104). Here, as described above, the cluster average value calculation is to obtain the average value of each element RGB when RGB color representation is used as each pixel, and the cluster average value Ai of the cluster Cs is These are the average for the element R of each pixel belonging to the cluster Cs, the average for the element G of each pixel belonging to the cluster Cs, and the average for the element B of each pixel belonging to the cluster Cs.

  Next, image generation means for generating multiple still images is activated, and first, an average As of the calculated cluster average value Ai is obtained, that is, each of R, G, B calculated for each cluster is obtained. The average As for the cluster average value Ai is obtained and written as pixel information in the corresponding position (i, j) of the output image 10b, which is a multiple still image to be multiplexed and synthesized (step S105).

Finally, it is checked whether the processing has been completed for all positions (i, j) (step S106). If the processing has not been completed for all positions (i, j) (NO in step S106), the position (i, j) is updated to the next position, for example position as the information about the (i + 1, j) (step S107), the image 10a ₁ at the next location (i + 1, j), ..., pixel information P of 10a _n Return to step S102 to obtain ₁ ,..., P _n . On the other hand, (YES in step S106) all at position (i, j) if the processing for the ends and the desired plurality of original images 10a _1, ..., multi still image or the output image 10b 10a _n is superposed Is completed, the multiple image creation process is terminated.

Among the processes described above, the calculation method of the average As of the cluster average value Ai in step S105 is taken as an example shown in the schematic diagram of FIG. 5, taking the R element as one element forming each pixel information as an example. This is as follows.
Here, in order to simplify the story, when dividing into a plurality of clusters in the cluster dividing step of step S103, as shown in FIG. 5, backgrounds 54 ₀ , 54 ₁ , Cluster division is performed so as to divide into two clusters of 54 ₂ and objects 55 ₀ , 55 ₁ , and 55 _{2. In} the cluster average value acquisition step in step S 104, for example, one piece of pixel information is formed. The cluster average value Ai of the background cluster 54 (any of 54 ₀ , 54 ₁ , 54 ₂ ) is set to a and the cluster average value of the object cluster 55 (any _{one of} 55 ₀ , 55 ₁ , 55 ₂ ) regarding the R element as the element. Let Ai be b.

  In the image generation means in step S105, when generating the multiple still image 53 to be the output image 10b, in the portion 56 where the background 54 is overlapped, the number of R elements related to the background cluster 54 is “3”, and the object cluster Since the number of R elements related to 55 is “0”, the R element (color) of the average As of the cluster average value Ai regarding the pixel information at the corresponding position in the output image 10 b (the multiple still image 53) is the average of the background cluster 54. It becomes the value a.

  On the other hand, in the portion 57 where two backgrounds 54 and one object 55 overlap each other, the number of R elements related to the background cluster 54 is “2” and the number of R elements of the object cluster 55 is “1”. The average {(a + b) / 2} of a and the average value b of the object cluster 55 is the R element (color) of the average As of the cluster average value Ai regarding the pixel information at the corresponding position in the output image 10b (multiple still image 53). )

Similarly, in the portion 58 where one background 54 overlaps two objects 55, {(a + b) / 2} is the cluster average value Ai relating to the pixel information of the corresponding position in the output image 10b (multiple still image 53). The average As R element (color).
In the portion 59 where the three objects 55 overlap, the average value b of the object cluster 55 is an R element (color) of the average As of the cluster average value Ai related to pixel information at the corresponding position in the output image 10b (multiple still image 53). )

  That is, even if the number of original images is increased to 3 or more, the average As color of the cluster average value Ai in the output image 10b is the background cluster 54 in the image portion where all the original images are the background 54. The average As color of the cluster average value Ai in the output image 10b becomes the color of the average value b of the object cluster 55 in the image portion where all the original images become the object 55. In the image portion at the position where the background 54 and the object 55 are mixed, the average {(a + b) / 2} of the average value a of the background cluster 54 and the average value b of the object cluster 55 is the color in the output image 10b. The average As color of the cluster average value Ai is obtained, and a beautiful multiple still image can be obtained by a simple calculation without depending on the number of original images.

  If multiple still images are created by performing multiple image creation processing as described above, a large memory is not necessary and high arithmetic processing performance is not necessary, such as a digital camera or a mobile phone with a camera. Even a device that does not have a memory and has low processing performance (CPU performance) can easily create multiple still images.

(Second Embodiment)
Next, a second embodiment will be described. FIG. 2 is an operation flowchart for explaining a second embodiment of the multiple image creating apparatus and the multiple image creating method according to the present invention.
First, the storage process is started, a plurality of original image 20a ₁ by the operation of the storage processing step, ..., 20a _n is stored in the storage unit 20 (step S21). In this state, multi-image creation processing is activated, the storage unit desired plurality of original images 20a ₁ stored in the 20, ..., to create a multiple still images superposed 20a _n, as an output image 20b The operation to output is started. When multiple image producing process is started, first, a desired plurality of original images 20a _1, ..., performs initial setting of the positional information 20a _n (i, j) (step S201).

Next, is started cluster dividing step of dividing each image information into a plurality of clusters (groups), first, the position information (i, j) each of the plurality of original images 20a ₁ specified by, ..., of 20a _n Pixel information P ₁ ,..., P _n at the same position is acquired (step S202). Here, as each element forming the pixel information P _k (k = 1,..., N), that is, color expression, in the present invention, any expression method such as RGB, YUV, YCC may be used. But it is expressed as 3D information. In the following description, description will be made assuming that RGB color representation is used.

Then, each pixel data _{P K (k = 1, ...} , n) by the information generating step of generating a one-dimensional information about each original image 20a ₁ has been acquired, ..., pixel information P ₁ of 20a _n, ..., Information generation for generating one-dimensional information consisting of one-dimensional arbitrary information specified in advance represented by one of color information such as luminance information and green, based on the three-dimensional information of element RGB forming P _n Is performed (step S203).

Then, the cluster dividing step of dividing each image information into a plurality of clusters (groups), using the values of one-dimensional information generated in step S203, the original image 20a _1, ..., pixel information P of 20a _{n 1} ,..., P _n are divided into an appropriate number of clusters (step S204).
Next, in the cluster average value acquisition step, for each of the plurality of divided clusters, cluster average value calculation is performed to obtain the average value of each element forming the pixel information as the cluster average value Ai (step S205).

  Here, as described above, the cluster average value calculation is to obtain the average value of each element RGB when RGB color representation is used as each pixel, and the cluster average value Ai of the cluster Cs As in the case of the first embodiment, the average for the element R of each pixel belonging to the cluster Cs, the average for the element G of each pixel belonging to the cluster Cs, and the average for the element B of each pixel belonging to the cluster Cs. .

  Next, image generation means for generating multiple still images is activated, and the average As of the calculated cluster average value Ai is first obtained by the same method as in the first embodiment, that is, for each cluster. The average As for the cluster average values Ai of R, G, and B calculated in the above is obtained, and is written as pixel information in the corresponding position (i, j) of the output image 20b that is the multiple still image to be multiplexed and synthesized (step S206). ).

Finally, it is checked whether the processing has been completed for all positions (i, j) (step S207). If the processing has not been completed for all positions (i, j) (NO in step S207), the position (i, j) is updated to the next position, for example position as the information about the (i + 1, j) (step S208), the image 20a ₁ at the next location (i + 1, j), ..., pixel information P of 20a _n Return to step S202 to obtain ₁ ,..., P _n . On the other hand, (YES in step S207) all at position (i, j) if the processing for the ends and the desired plurality of original images 20a _1, ..., multi still image or the output image 20b 20a _n is superposed Is completed, the multiple image creation process is terminated.

If a multiple still image is created by performing the multiple image creation process as described above, a large memory is not required and a high arithmetic processing performance is unnecessary as in the first embodiment. Even a device that does not have a large memory and has low arithmetic processing performance (CPU performance) such as a camera-equipped mobile phone can easily create multiple still images.
Furthermore, since only the one-dimensional information generated in the information generation step of step S203 is used as the distance function when dividing into a plurality of clusters, the distance function can be calculated as compared with the first embodiment. Even in an environment where the processing becomes simpler and the processing performance (CPU performance) is lower, it is possible to expect a higher processing speed.

(Third embodiment)
Next, a third embodiment will be described. FIG. 3 is an operation flowchart for explaining a third embodiment of the multiple image creating apparatus and the multiple image creating method according to the present invention.
First, the storage process is started, a plurality of original image 30a ₁ by the operation of the storage processing step, ..., 30a _n is stored in the storage unit 30 (step S31). In this state, multi-image creation processing is activated, the memory unit 30 desired plurality of original images 30a stored in _1, ..., to create a multiple still images superposed 30a _n, as an output image 30b The operation to output is started. When multiple image producing process is started, first, a desired plurality of original images 30a _1, ..., performs initial setting of the positional information 30a _n (i, j) (step S301).

Next, is started cluster dividing step of dividing each image information into a plurality of clusters (groups), first, the position information (i, j) each of the plurality of original images 30a ₁ specified by, ..., of 30a _n Pixel information P ₁ ,..., P _n at the same position is acquired (step S302). Here, as each element forming the pixel information P _k (k = 1,..., N), that is, color expression, in the present invention, any expression method such as RGB, YUV, YCC may be used. But it is expressed as 3D information. In the following description, as an explanation for the case of using the RGB color representation, pixel information P _1, ..., the elements R for forming the P _n, G, for each color of B, submitted below Each process is configured to be performed independently.

Next, any color information is selected from among the elements forming the acquired pixel information P ₁ ,..., P _n, that is, color information R, G, B, and any of the selected color information is selected. with each original image 30a _1, ..., pixel information P ₁ of 30a _n, ..., the color information forming the P _n, divided into appropriate number of clusters (step S303).
Next, in the cluster average value acquisition step, for each of the plurality of divided clusters, cluster average value calculation is performed to obtain an average value related to the color information as the cluster average value Ai (step S304).

  Next, image generation means for generating multiple still images is activated, and the average As of the cluster average value Ai relating to the color information calculated for each cluster is first calculated by the same method as in the first embodiment. Then, it is written as the color information for forming pixel information at the corresponding position (i, j) of the output image 30b to be a multiple still image to be multiplexed and synthesized (step S305).

  Next, it is checked whether or not all color information has been processed (step S306). If color information that has not yet been processed remains (NO in step S306), the color information is changed to the next color. The information is updated (step S308), the process returns to step S302, and the above-described processing is repeated for the next color information.

On the other hand, if the processing has been completed for all the color information (YES in step S306), it is finally checked whether the processing has been completed for all positions (i, j) (step S307). If the process has not been completed for (i, j) (NO in step S307), the information on the position (i, j) is updated to the next position, for example, the position (i + 1, j) (step S309). position information (i + 1, j) each image 30a ₁ at a, ..., pixel information P ₁ of 30a _n, ..., the flow returns to step S302 to acquire the P _n. On the other hand, (YES in step S307) all at position (i, j) if the processing for the ends and the desired plurality of original images 30a _1, ..., multi still image or the output image 30b 30a _n is superposed Is completed, the multiple image creation process is terminated.

If a multiple still image is created by performing the multiple image creation process as described above, a large memory is not required and a high arithmetic processing performance is unnecessary as in the first embodiment. Even a device that does not have a large memory and has low arithmetic processing performance (CPU performance) such as a camera-equipped mobile phone can easily create multiple still images.
Furthermore, since only one of the color information is used as the distance function when dividing into a plurality of clusters, as in the case of the second embodiment described above, compared to the first embodiment. The calculation of the distance function becomes easier, and even in an environment where the arithmetic processing performance (CPU performance) is lower, it is possible to expect a higher processing speed.
In particular, each of the original image 30a _1, ..., position information (i, j) of 30a _n based on whether larger or smaller than the average value for the corresponding color information, if it is divided only into two clusters, more The calculation is simplified and further speedup can be expected.

(Fourth embodiment)
Next, a fourth embodiment will be described. FIG. 4 is an operation flowchart for explaining a fourth embodiment of the multiple image creating apparatus and the multiple image creating method according to the present invention.
First, the storage process is started, a plurality of original image 40a ₁ by the operation of the storage processing step, ..., 40a _n are stored in the storage unit 40 (step S41). In this state, multi-image creation processing is activated, the desired plurality of original images 40a ₁ stored in the storage unit 40, ..., to create a multiple still images superposed 40a _n, as an output image 40b The operation to output is started. When multiple image producing process is started, first, a desired plurality of original images 40a _1, ..., performs initial setting of the positional information 40a _n (i, j) (step S401).

Next, is started cluster dividing step of dividing each image information into a plurality of clusters (groups), first, the position information (i, j) each of the plurality of original images 40a ₁ specified by, ..., of 40a _n Pixel information P ₁ ,..., P _n at the same position is acquired (step S402). Here, as each element forming the pixel information P _k (k = 1,..., N), that is, color expression, any expression method such as RGB, YUV, YCC, etc. may be used in the present invention. In FIG. 2, the case where RGB color expression is used will be described.

Next, the original image 40a ₁ has been acquired, ..., pixel information P ₁ of 40a _n, ..., and P _n, based on the distance information indicating the distance between the original image each image information, the appropriate number Divide into clusters (step S403). Here, the algorithm for dividing into clusters based on the distance information between the image information can be realized using various algorithms as described above as long as an appropriate distance function and threshold value are given. Many technologies are disclosed as technologies.

For example, as in the case of the first embodiment described above, when the distance information between the pixel information of the pixels P0 and P1 is expressed by being converted to the Lab system, the distance between the pixel information of the pixels P0 and P1 is expressed. Color difference as a function: ΔE = sqrt {(R0−R1) ^ 2 + (a0−a1) ^ 2
+ (B0−b1) ^ 2}
It is conceivable to perform clustering using. In addition to this expression, it is also conceivable to perform clustering by calculating a distance function based on, for example, one of the following expressions using R, G, B information.
ΔE = sqrt {(R0−R1) ^ 2 + (G0−G1) ^ 2
+ (B0-B1) ^ 2}
Or
ΔE = | R0−R1 | + | G0−G1 | + | B0−B1 |

  For each cluster obtained by applying any of the cluster division techniques as described above, the average value of each element forming the pixel information is set as the cluster average value Ai for each cluster in the cluster average value acquisition step. The obtained cluster average value is calculated (step S404). Here, as described above, the cluster average value calculation is to obtain the average value of each element RGB when RGB color representation is used as each pixel, and the cluster average value Ai of the cluster Cs is These are the average for the element R of each pixel belonging to the cluster Cs, the average for the element G of each pixel belonging to the cluster Cs, and the average for the element B of each pixel belonging to the cluster Cs.

  Next, unlike the method of obtaining the average As of the cluster average value Ai shown in the first embodiment by starting the image generation means for generating the multiple still images, first, the calculated cluster average value Ai is calculated. Are weighted with respect to the elements constituting each cluster average value Ai, and a weighted sum is obtained, and written as pixel information in the corresponding position (i, j) of the output image 10b to be a multiple still image to be multiplexed and synthesized. (Step S405).

Finally, it is checked whether the processing has been completed for all positions (i, j) (step S406). If the processing has not been completed for all positions (i, j) (NO in step S406), the position (i, j) is updated to the next position, for example position as the information about the (i + 1, j) (step S407), the image 40a ₁ at the next location (i + 1, j), ..., pixel information P of 40a _n Return to step S402 to obtain ₁ ,..., P _n . On the other hand, (YES in step S406) all at position (i, j) if the processing for the ends and the desired plurality of original images 40a _1, ..., multi still image or the output image 40b 40a _n is superposed Is completed, the multiple image creation process is terminated.

  In addition, as a weighting coefficient applied to weighting, with respect to each element forming the average value Ai, the user uses a weighting coefficient setting step not shown in FIG. For example, when there is a cluster having a cluster average value Ai close to a specific color or luminance set in advance as desired color information or luminance information, the cluster average of the cluster It is possible to increase the weighting coefficient of the value Ai and set the weighting coefficient of the cluster average value Ai of other clusters not applicable in advance to be small. Therefore, when there is no cluster having a cluster average value Ai close to a specific color or brightness, the weight of each cluster average value Ai is equal to all the clusters, and the above-mentioned first This is exactly the same as the case of using the average As of the cluster average value Ai in the case of the first embodiment. Here, the sum of the weighting coefficients for weighting each of the plurality of cluster average values Ai is set to “1”.

  Thus, for example, when creating a multiple still image in which a plurality of original images showing human movement are overlapped, a specific color for which a weighting coefficient for human skin color (close color) should be set large Is selected in advance, and the weighting coefficient of the cluster average value Ai of the cluster in which human skin color (close color) exists is set large in advance, thereby generating an output image 40b of a multiple still image in which human motion is emphasized. It is possible.

  In each of the embodiments described above, a multiplex image creation method that can easily create a multiplex still image based on the operation flowchart has been described. However, the processing contents of each step shown in the operation flowchart are realized. It may be realized as a multiple image playback device provided with dedicated hardware means, or may be realized as a multiple image creation program that can be executed by a computer by performing such program logic, The multiple image creation program may be recorded on a computer-readable program recording medium.

3 is an operation flowchart for explaining a first embodiment of a multiple image creating apparatus and a multiple image creating method according to the present invention. It is an operation | movement flowchart for demonstrating the 2nd Example of the multiple image production apparatus and multiple image production method which concern on this invention. It is an operation | movement flowchart for demonstrating the 3rd Example of the multiple image production apparatus and multiple image production method which concern on this invention. It is an operation | movement flowchart for demonstrating the 4th Example of the multiple image production apparatus and multiple image production method which concern on this invention. It is a schematic diagram for demonstrating the process example which overlaps 3 images and produces one still image.

Explanation of symbols

10, 20, 30, 40 ... storage _{unit, 10a 1, ~, 10a n} , 20a 1, ~, 20a n, 30a 1, ~, 30a n, 40a 1, ..., 40a n ... original image, 10b, 20b, 30b, 40b ... output image, 50 ... first original image, 51 ... second original image, 52 ... third original image, 53 ... multiple still images, 54, 54 ₀ , 54 ₁ , 54 ₂ ... background (background cluster), 55, 55 ₀ , 55 ₁ , 55 ₂ ... object (object cluster), 56 ... part where three backgrounds overlap, 57 ... part where two backgrounds and one object overlap, 58 ... background Is a part where two objects overlap, 59 ... a part where three objects overlap.

Claims (12)

  Storage processing means for storing a plurality of images in the storage unit, pixel information at the same position of each of the images stored in the storage unit is read, and each pixel information based on distance information indicating a gap between the pixel information A cluster dividing means for dividing the cluster information into a plurality of clusters, a cluster average value obtaining means for obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and taking an average of the cluster average values. And an image generation means for generating pixel information at the same position of the image to be multiplexed and synthesized.   Storage processing means for storing a plurality of images in the storage unit, pixel information at the same position of each of the images stored in the storage unit is read, and represented by luminance information or any color information based on the pixel information. Information generating means for generating one-dimensional information consisting of arbitrary one-dimensional information, cluster dividing means for dividing each pixel information into a plurality of clusters based on the one-dimensional information, and for each cluster, Cluster average value acquisition means for obtaining an average value of each element forming the pixel information as a cluster average value, and image generation means for taking the average of the cluster average values and generating them as pixel information at the same position of an image to be multiplexed and combined A multi-image creating apparatus comprising:   Storage processing means for storing a plurality of images in the storage unit, each color information of the pixel at the same position of each of the images stored in the storage unit is read, and the color information of each of the pixels based on each of the color information, respectively Cluster dividing means for dividing into a plurality of clusters, cluster average value obtaining means for obtaining an average value of color information of each pixel as a cluster average value for each cluster, and averaging each of the cluster average values An apparatus for generating multiple images, comprising: image generation means for generating each color information of pixels at the same position of an image to be combined.   Storage processing means for storing a plurality of images in the storage unit, pixel information at the same position of each of the images stored in the storage unit is read, and each pixel information based on distance information indicating a gap between the pixel information A cluster dividing means for dividing the cluster information into a plurality of clusters, a cluster average value obtaining means for obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and weighting the plurality of cluster average values And an image generation means for generating pixel information at the same position of the image to be multiplexed and synthesized.   5. The multiple image creation device according to claim 4, wherein the image generation unit uses a weighting coefficient for weighting each of the plurality of cluster average values, which is used when taking a sum total obtained by weighting the cluster average values. A multi-image creating apparatus comprising weighting coefficient setting means capable of presetting a value.   A storage processing step for storing a plurality of images in a storage unit, and reading out pixel information at the same position of each of the images stored in the storage unit, and each pixel information based on distance information indicating a gap between the pixel information A cluster dividing step of dividing the cluster information into a plurality of clusters, a cluster average value obtaining step of obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and taking an average of the cluster average values And an image generation step of generating as pixel information at the same position of the image to be multiplexed and synthesized.   A storage processing step for storing a plurality of images in a storage unit, pixel information at the same position of each of the images stored in the storage unit is read out, and is represented by luminance information or any color information based on the pixel information. An information generating step for generating one-dimensional information consisting of arbitrary one-dimensional information, a cluster dividing step for dividing each pixel information into a plurality of clusters based on the one-dimensional information, and for each cluster, A cluster average value obtaining step for obtaining an average value of each element forming the pixel information as a cluster average value, and an image generating step for taking the average of the cluster average values and generating them as pixel information at the same position of the image to be multiplexed and combined A method of creating multiple images, comprising:   A storage processing step of storing a plurality of images in a storage unit, and reading out each color information of a pixel at the same position of each of the images stored in the storage unit, and a plurality of color information of each pixel based on each color information A cluster dividing step for dividing the cluster into clusters, a cluster average value obtaining step for obtaining an average value of color information of each pixel as a cluster average value for each cluster, and taking an average of each of the cluster average values to perform multiple synthesis And an image generation step of generating as each color information of the pixel at the same position of the image to be performed.   A storage processing step for storing a plurality of images in a storage unit, and reading out pixel information at the same position of each of the images stored in the storage unit, and each pixel information based on distance information indicating a gap between the pixel information A cluster dividing step of dividing the cluster information into a plurality of clusters, a cluster average value obtaining step for obtaining an average value of each element forming each pixel information as a cluster average value for each cluster, and weighting the plurality of cluster average values And an image generation step of generating as the pixel information at the same position of the image to be multiplexed and synthesized.   10. The multiple image creation method according to claim 9, wherein the image generation step uses a weighting coefficient for weighting each of the plurality of cluster average values, which is used when taking the sum total obtained by weighting the cluster average values. A method for creating multiple images, comprising a weighting coefficient setting step capable of presetting a value.   11. A multiplex image creation program comprising the program logic of the multiplex image creation method according to claim 6 configured as a program executed by a computer.   A program recording medium, wherein the multiple image creating program according to claim 11 is recorded on a computer-readable recording medium.

Images