WO2023079702A1 - 情報処理装置、情報処理方法及び情報処理プログラム - Google Patents

情報処理装置、情報処理方法及び情報処理プログラム Download PDF

Info

Publication number
WO2023079702A1
WO2023079702A1 PCT/JP2021/040851 JP2021040851W WO2023079702A1 WO 2023079702 A1 WO2023079702 A1 WO 2023079702A1 JP 2021040851 W JP2021040851 W JP 2021040851W WO 2023079702 A1 WO2023079702 A1 WO 2023079702A1
Authority
WO
WIPO (PCT)
Prior art keywords
images
image
evaluation value
information processing
evaluation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/040851
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
満 中澤
ビヨン シュテンガー
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rakuten Group Inc
Original Assignee
Rakuten Group Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rakuten Group Inc filed Critical Rakuten Group Inc
Priority to JP2022557884A priority Critical patent/JP7216255B1/ja
Priority to EP21950362.0A priority patent/EP4203451B1/en
Priority to PCT/JP2021/040851 priority patent/WO2023079702A1/ja
Priority to US18/005,056 priority patent/US12159374B2/en
Priority to JP2023006661A priority patent/JP7303953B2/ja
Publication of WO2023079702A1 publication Critical patent/WO2023079702A1/ja
Priority to JP2023103153A priority patent/JP7526318B2/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
    • H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
    • H04N1/00185—Image output
    • H04N1/00196—Creation of a photo-montage, e.g. photoalbum
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/00127—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture
    • H04N1/00132—Connection or combination of a still picture apparatus with another apparatus, e.g. for storage, processing or transmission of still picture signals or of information associated with a still picture in a digital photofinishing system, i.e. a system where digital photographic images undergo typical photofinishing processing, e.g. printing ordering
    • H04N1/00185—Image output
    • H04N1/00198—Creation of a soft photo presentation, e.g. digital slide-show
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/387—Composing, repositioning or otherwise geometrically modifying originals
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present invention relates to an information processing device, an information processing method, and an information processing program.
  • composite images For the purpose of presenting content containing multiple images or guiding users to such content, there is a demand for composite images in which multiple images in content are arranged in a predetermined arrangement, such as a tile arrangement. Examples include web page design for companies and stores, advertising posters for sightseeing spots and travel packages, introductions to movies and games, banners to introduce products related to electronic commerce, and the like. Such composite images are created by a designer selecting a required number of images and arranging them at predetermined positions.
  • the present invention has been made in view of such circumstances, and its purpose is to contribute to the creation of a high-quality composite image based on a plurality of images.
  • a single image selection unit that selects M images (M ⁇ N) from N images (N>1), and a single image selection unit that selects the M images selected in predetermined M frames, respectively.
  • a composition unit that arranges and creates a composite image; a selection evaluation value that is a linear sum of single image evaluation values of the M selected images; and a composition evaluation that is a single image evaluation value of the composite image.
  • an evaluation unit that associates and determines a comprehensive evaluation value including at least a linear sum of values with the generated synthesized image.
  • the evaluation unit normalizes the selected evaluation value based on M, which is the number of the selected images, and determines the overall evaluation value for each of the plurality of synthesized images with different Ms. information processing device.
  • the evaluation unit calculates the single image evaluation value of each of the selected M images and the weight coefficient corresponding to each of the predetermined M frames.
  • An information processing device that determines the selected evaluation value based on.
  • the evaluation unit normalizes the similarity evaluation value based on M, which is the number of the selected images, and calculates the total evaluation value of each of the plurality of synthesized images with different M An information processing device that determines.
  • ⁇ 1 , ⁇ 2 , and ⁇ 3 are arbitrary weighting factors
  • w i is the weighting factor corresponding to the i-th frame
  • I i is the i-th image
  • I whole is the composite image
  • Score(I) is A single image evaluation value of image I
  • Similarity(I i , I j ) is the degree of similarity between images I i and I j .
  • the evaluation unit inputs the selected image or the synthesized image to a machine learning model to correspond to the input image or the synthesized image
  • An information processing device that acquires a single image evaluation value.
  • the computer comprises: a single image selection unit that selects M images (M ⁇ N) from N images (N>1); A synthesizing unit that arranges images and creates a synthetic image; a selection evaluation value that is a linear sum of single image evaluation values of the M selected images; and a single image evaluation value of the synthetic image.
  • An information processing program functioning as an information processing apparatus, comprising: an evaluation unit that determines a total evaluation value including at least a linear sum of a certain composite evaluation value in association with the generated composite image.
  • FIG. 1 is a functional conceptual diagram of an information processing apparatus according to a first embodiment of the present invention
  • FIG. FIG. 10 is a diagram schematically showing how a composite image is created for a plurality of M values from N images.
  • 1 is a configuration diagram showing a typical physical configuration of a general computer
  • FIG. FIG. 4 is a diagram showing an example of the flow of operations of a single image selection unit according to the first embodiment of the present invention
  • FIG. 10 is a diagram showing various examples of templates
  • FIG. 4 is a diagram showing an example of the flow of operations of a synthesizing unit according to the first embodiment of the present invention
  • FIG. 10 is a diagram showing a setting example of a weighting factor wi ; It is a figure which shows an example of the flow of operation
  • FIG. 4 is a diagram showing an example of the flow of operations of a composite image selection unit according to the first embodiment of the present invention;
  • FIG. 7 is a functional conceptual diagram of an information processing apparatus according to a second embodiment of the present invention;
  • FIG. 1 is a functional conceptual diagram of the information processing device 100 according to the first embodiment of the present invention.
  • the information processing apparatus 100 is realized by realizing the functions shown in the figure by appropriate physical means, for example, a computer executing appropriate computer programs.
  • the information processing device 100 includes a single image selection unit 10 , a synthesis unit 20 , an evaluation unit 30 and a synthesis image selection unit 40 .
  • the information processing apparatus 100 inputs N images (N images) to be processed and outputs a combined image. where N>1.
  • the N images are given images, such as an image library or a group of images included in arbitrary content. Then, what the information processing apparatus 100 tries to do is to select M images from the N images (M ⁇ N), arrange them in M frames determined in advance, and place them as high as possible. To contribute to the creation of composite images that are of good quality, that is, that are attractive to viewers and that receive positive evaluations. At this time, various composite images can be obtained depending on how to select M images from N images and in which of the M frames each of the selected M images is arranged. And the evaluation is also different.
  • the information processing apparatus 100 evaluates the synthesized image obtained mechanically (that is, by information processing by a computer) without relying on people, and it is reasonable to attract the viewer and obtain a positive evaluation. It is configured to contribute to the creation of a synthetic image that is reasonably estimated.
  • FIG. 2 is a diagram schematically showing how a composite image is created for a plurality of M values from N images.
  • M three values of M of 4, 6, and 9 are assumed.
  • the types of values of M and the number of frames may be larger, and the arrangement of the frames is also arbitrary.
  • the N images may include images that are similar to each other (for example, photographic images of the same dish or scenery taken from different angles). are divided into groups of similar images. The whole picture would be verbose and unappealing because it would include images that do.
  • composite image C can present more diverse images, whereas composite images A and B potentially have N images. I can't fully convey the charm I have.
  • the information processing apparatus 100 also evaluates composite images having different Ms and frames with different arrangements, and eventually attracts the viewer and obtains a positive evaluation from among them. It is assumed that it is possible to create a synthetic image that is estimated to be
  • the information processing device 100 may be physically realized using a general computer.
  • FIG. 3 is a configuration diagram showing a representative physical configuration of such a general computer 1. As shown in FIG.
  • a computer 1 includes a CPU (Central Processing Unit) 1a, a RAM (Random Access Memory) 1b, a static storage device 1c, a GC (Graphics Controller) 1d, an input device 1e and an I/O (Input/Output) 1f as a data bus 1g. are connected so that electrical signals can be exchanged with each other.
  • the static storage device 1c is a device capable of statically recording information, such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).
  • a signal from the GC 1d is output to a monitor 1h, such as a CRT (Cathode Ray Tube) or a so-called flat panel display, on which the user visually recognizes an image, and displayed as an image.
  • a monitor 1h such as a CRT (Cathode Ray Tube) or a so-called flat panel display, on which the user visually recognizes an image, and displayed as an image.
  • the input device 1e is a device such as a keyboard, mouse, touch panel, etc., for a user to input information
  • the I/O 1f is an interface for the computer 1 to exchange information with external devices.
  • a plurality of CPUs 1a may be prepared to perform parallel operations.
  • An information processing program containing a sequence of instructions for causing the computer 1 to function as the information processing device 100 is installed in the external storage device 1c, read out to the RAM 1b as necessary, and executed by the CPU 1a. Further, such a program may be provided by being recorded on an appropriate computer-readable information recording medium such as an appropriate optical disk, magneto-optical disk, or flash memory, or may be provided via an information communication line such as the Internet.
  • an interface related to the computer 1 itself may be implemented so that the user may directly operate the computer 1, or a web interface may be used on another computer. It may be a so-called cloud computing method in which general-purpose software such as a browser is used and functions are provided from the computer 1 via the I/O1f. interface), the computer 1 may operate as the information processing apparatus 100 in response to a request from another computer.
  • Each configuration of the information processing device 100 shown in FIG. may be any configuration of the information processing device 100 shown in FIG.
  • the single image selection unit 10 selects M images from the N images. M at this time is equal to the number of frames of the synthesized image obtained by synthesizing the images selected by the synthesizing unit 20 .
  • the single image selection unit 10 extracts possible combinations for selecting M images from N images. Therefore, if the number of such combinations is k, the single image selection unit 10 outputs k sets of M images (M images).
  • FIG. 4 is a diagram showing an example of the operation flow of the single image selection unit 10 according to this embodiment.
  • step S101 the single image selection unit 10 substitutes an initial value of 1 for the variable x, and in subsequent step S102, selects the x-th template. Since M frames are set in the x-th template, in step S103 the single image selection unit 10 selects M images from the N images so that the same combination is not selected. select.
  • step S104 it is determined whether or not all possible combinations of M image sets have been selected. If all combinations have not yet been selected, the process returns to step S103 to continue selecting M images. If all combinations have already been selected, the process advances to step S105 to determine whether all of the multiple templates have been selected.
  • step S106 If all templates have not been selected yet, proceed to step S106, add 1 to the variable x, and return to step S102. If all templates have already been selected, all possible combinations of M image sets have been selected for all M possible combinations, and the process ends.
  • FIG. 5 is a diagram showing various examples of templates.
  • (a) and (b) are examples in which rectangular areas are arranged in tiles as an image frame f.
  • the template shown in (a) has 6 frames f of 3 vertical ⁇ 2 horizontal, and the template shown in (b) has 9 frames f of 3 vertical ⁇ 3 horizontal ( Note that only one symbol f is shown as a representative).
  • the template of (c) seven rectangular frames f are arranged, and a pre-arranged fixed image p is included in the portion other than the frames f. Therefore, in the composite image created using the template (c), the M selected images are arranged in M frames and the fixed image p is included.
  • the template of (d) three circular or elliptical frames f of different shapes and sizes are arranged, and a fixed image p is arranged below them.
  • the arrangement, number, shape and size of the frames f are arbitrary, and are limited to those in which the rectangular frames f are regularly arranged in tiles as shown in (a) and (b). not.
  • any fixed image p can be placed in the template outside the frame f.
  • a composite image including decorative frames and other images required by the design can be created, and the quality of the composite image can be evaluated as a whole including the fixed image p.
  • the synthesis unit 20 receives a set of M images from the single image selection unit 10, and arranges the selected M images in M predetermined frames in each template. , to create Combined Images. At this time, M! Since there is a permutation of streets, M! As many different composite images will be created. Note that the synthesizing unit 20 may determine a combination of a set of M images received from the single image selecting unit 10 and the layout of each image on the template as synthesizing information.
  • FIG. 6 is a diagram showing an example of the operation flow of the synthesizing unit 20 according to this embodiment.
  • the synthesizing unit 20 substitutes the initial value 1 for the variable x, and in subsequent step S202, selects the x-th template. M frames are set in the x-th template, and the single image selection unit 10 selects M! Only pairs are selected. Therefore, in order to sequentially select a set of these images, in step S203, an initial value of 1 is substituted for the variable y, and in subsequent step S204, the y-th set of M images is selected.
  • step S205 the synthesizing unit 20 performs M! Arrange the selected M images into M frames according to the permutation of the street, and M! create a composite image.
  • step S206 it is determined whether or not all image sets have been selected for the selected x-th template. If all image pairs have not been selected yet, y is incremented by 1 in step S207, and the process returns to step S204 to repeat the process.
  • step S208 determines whether all of the multiple templates have been selected. If all templates have not yet been selected, the process proceeds to step S209, adds 1 to the variable x, and returns to step S202. If all the templates have already been selected, then all permutations of the composite images have been created for all image sets for all possible templates, and the process ends.
  • an image when an image is treated as one independent image, such an image may be referred to as a "single image".
  • Each image included in the N images is treated as a "single image”
  • the synthesized image created by the synthesizing unit 20 is treated as a single independent image separated from its original image or template. In that case, the composite image will be treated as a "single image”.
  • the evaluation unit 30 receives the synthesized images from the synthesizing unit 20, and for each synthesized image, selects a selection evaluation value that is a linear sum of the single image evaluation values and a synthesis evaluation value that is a single image evaluation value of the synthesized image.
  • a comprehensive evaluation value (Reward) including at least a linear sum of the values is determined in association with the created synthetic image.
  • the evaluation unit 30 may receive the synthesis information from the synthesis unit 20 and associate the comprehensive evaluation value with the synthesis information.
  • the single image evaluation value is the evaluation value obtained when evaluating an image by viewing it as a single image. Therefore, the selection evaluation value included in the total evaluation value is a linear sum of M evaluation values obtained by evaluating each of the M images that are the sources of the composite image as single images. is the meaning.
  • the composite evaluation value is an evaluation value obtained by evaluating the composite image itself as one single image.
  • the evaluation unit 30 gives a comprehensive evaluation value to each composite image created by the composition unit 20 as its evaluation. As a result, based on this comprehensive evaluation value, it becomes possible to determine a composite image that is more attractive to the viewer.
  • the overall evaluation value must reasonably reflect the appeal of the synthetic image to those who view it.
  • the method for determining the comprehensive evaluation value in the evaluation unit 30 will be described more specifically.
  • the evaluation unit 30 determines the comprehensive evaluation value Reward by the following Equation 1.
  • ⁇ 1 , ⁇ 2 , and ⁇ 3 are arbitrary weighting factors
  • w i is the weighting factor corresponding to the i-th frame
  • I i is the i-th image
  • I whole is the synthesized image
  • Score(I) is the image
  • Similarity(I i , I j ) is the degree of similarity between images I i and I j .
  • the first term on the right side of Equation 1 indicates the selected evaluation value. That is, the selected M images are numbered from 1 to M to distinguish them, and the sum of the single image evaluation value Score (I i ) of the i-th image I i multiplied by an arbitrary weighting factor w i , That is, the linear sum is used as the selection evaluation value.
  • the weighting factor ⁇ 1 designates the weighting of the selected evaluation value in the entire comprehensive evaluation value Reward, and 1/M is a normalization coefficient.
  • the selected evaluation value indicates a higher value as the individual single image evaluation value Score(I i ) for the image I i increases. That is, the higher the number of images with high evaluation as single images that are selected, the higher the selection evaluation value.
  • the magnitude of the selection evaluation value depends on the number M of images selected. That is, the selection evaluation value becomes larger for a synthesized image that uses more images as synthesis sources.
  • the selection evaluation value is made independent of the value of M by multiplying by the normalization coefficient.
  • the normalization coefficient is a function of M and is given as g that satisfies Equation 2 below.
  • f(I) is an arbitrary evaluation function that gives a non-zero evaluation value to the image I
  • a is an arbitrary non-zero real number.
  • the evaluation unit 30 normalizes the selection evaluation value based on M, which is the number of selected images.
  • M is the number of selected images.
  • a/M satisfies Equation 2
  • the weighting factor wi is determined for each frame of the template. That is, in a given template, a larger value may be set for a frame at a position considered more important to the viewer, and a smaller value may be set for a frame that is relatively unimportant.
  • FIG. 7 is a diagram showing a setting example of the weighting factor wi .
  • the template of (e) in FIG. 7 includes nine rectangular frames arranged in 3 ⁇ 3 tiles, and the darker the color assigned to each frame, the larger the value of the weighting factor wi . is shown.
  • This template assumes a usage scene in which the observer looks at each image arranged in the frame in the obtained composite image in order from the upper left to the lower right. A larger weighting factor wi value is given to an object closer to the upper left frame, and a smaller weighting factor wi value is given to an object closer to the lower right frame.
  • the template of (f) in FIG. 7 also contains nine rectangular frames that are tiled 3 ⁇ 3. This template assumes a usage scene in which the viewer sees the composite image as a single image . is given, and the peripherally arranged frames are given relatively small values of weighting factors wi .
  • the evaluation unit 30 based on the single image evaluation value Score(I i ) of each of the selected M images, and the weighting factor w i corresponding to each of the M frames predetermined in the template, determines the selection evaluation value.
  • Equation 1 The second term on the right side of Equation 1 indicates the composite evaluation value. That is, the evaluation value Score (I whole ) when the composite image I whole is viewed as a single image is multiplied by a weighting factor ⁇ 2 that designates the weight of the selected evaluation value in the entire comprehensive evaluation value Reward. .
  • the meaning of the composite evaluation value is a numerical evaluation of the attractiveness to the viewer when viewing the composite image as a single image, independent of the individual images that make up the composite image. Therefore, apart from the contents of the individual images, it is considered that the composition evaluation value is higher as the composition image as a whole has a better balance of colors and details.
  • Both the aesthetic evaluation value and the CTR prediction value can be obtained by inputting the image I into the trained machine learning model.
  • Learning data for obtaining a machine learning model that outputs an aesthetic evaluation value already exists on the Internet for research or practical use for free or for a fee, so using such learning data For example, by training a machine learning model by CNN (convolutional neural network), a learned machine learning model can be easily obtained.
  • a machine learning model that outputs a CTR prediction value can be obtained by similarly training a machine learning model by, for example, a CNN, using a set of an image and the CTR obtained for the image as learning data. .
  • Learning data for training a machine learning model to output a CTR prediction value is, for example, for various images used for EC (electronic commerce) sites, the number of times (impressions) displayed to the user, It can be obtained by calculating the ratio of the number of times the image is clicked (selected).
  • the evaluation unit 30 can acquire the single image evaluation value corresponding to the input image I, here the selected image or the composite image.
  • the esthetic evaluation value obtained for image I is Score Aesthetic (I)
  • the predicted CTR value is linearly summed with Score CTR (I) to obtain a single image evaluation value. That is, the single image evaluation value Score(I) is obtained from the following equation (3).
  • W_Asthetic and W_CTR are arbitrary weighting factors.
  • the aesthetic evaluation value or the CTR prediction value may be used alone, or other evaluation values may be used.
  • the third term on the right side of Equation 1 indicates the similarity evaluation value.
  • a similarity evaluation value indicates the similarity between the selected images. In this example, when the selected M images are numbered from 1 to M and distinguished, the more similar images are included in the M images, the higher the value. there is
  • the similarity evaluation value is the similarity between the i-th image Ii and the j-th image Ij , i.e., the sum of the degree of approximation Similarity ( Ii , Ij ) indicating the degree of approximation, a weighting factor ⁇ 3 , It is multiplied by a normalization factor of 1/ MC2 .
  • the weighting factor ⁇ 3 designates the weight that the similarity evaluation value occupies in the overall comprehensive evaluation value Reward.
  • the degree of similarity (I i , I j ) in the similarity evaluation value is calculated equal to the number of combinations for selecting any two images from the M images. Since that number is M C 2 , the normalization factor used in this example is the reciprocal of this number of combinations, 1/ M C 2 . This normalization factor also satisfies Equation 2 above.
  • a technique used in any known image processing technique may be used. Examples thereof include a method using a DNN (deep neural network) such as CNN and other machine learning models, a method using the distance between image feature amount vectors, a combination of these methods, and the like. In this embodiment, a CNN-based machine learning model is used to obtain the similarity.
  • a DNN deep neural network
  • the comprehensive evaluation value Reward is a linear sum of the above-described selected evaluation value, combined evaluation value, and similarity evaluation value.
  • the sign of each term is linearly combined so that the selected evaluation value and the composite evaluation value are positive, and the similarity evaluation value is negative.
  • the higher the similarity evaluation value the lower the comprehensive evaluation value Reward. That is, the higher the evaluation value of the individual images selected to form the composite image, or the higher the evaluation value when the entire composite image is viewed as a single image, the higher the comprehensive evaluation value Reward is evaluated.
  • the more similar images are included in the images forming the image, and the more similar the images are, the lower the overall evaluation value Reward is.
  • the comprehensive evaluation value Reward By designing the comprehensive evaluation value Reward in this way, the attractiveness of the obtained synthesized image to the viewer can be evaluated rationally and quantitatively.
  • a specific formula for obtaining the comprehensive evaluation value Reward may differ from that shown in the present embodiment.
  • the evaluation value of a subset of images included in the synthesized image viewed as a single image may be considered, or the similarity evaluation value may not consider this.
  • the similarity evaluation value may be appropriately weighted so that the degree of approximation between images located closer to each other, such as adjacent images, has a greater influence.
  • FIG. 8 is a diagram showing an example of the operation flow of the evaluation unit 30 according to this embodiment.
  • the evaluation unit 30 substitutes the initial value 1 for the variable x, and in subsequent step S302, selects the x-th synthesized image.
  • step S303 based on Equation 1, the comprehensive evaluation value Reward of the selected x-th synthesized image is determined.
  • step S304 it is determined whether or not all synthesized images have been selected. If all composite images have not been selected yet, 1 is added to x in step S305, and the process returns to step S302 to repeat the process. If all synthesized images have already been selected, the overall evaluation value Reward has been determined for all synthesized images, and the process ends.
  • the synthesized image created by the synthesizing unit 20 is given a comprehensive evaluation value Reward by the evaluating unit 30 in association with it.
  • the composite image selection unit 40 selects at least one composite image based on the comprehensive evaluation value Reward. Note that the composite image selection unit 40 may select at least one piece of composite information based on the comprehensive evaluation value Reward, and create a composite image based on the selected composite information.
  • the simplest way is to select the composite image that shows the highest comprehensive evaluation value Reward.
  • FIG. 9 is a diagram showing an example of the operation flow of the composite image selection unit 40 according to this embodiment. Since the synthetic image selection unit 40 according to the present embodiment simply selects the synthetic image showing the highest comprehensive evaluation value Reward, it is sufficient to select the synthetic image in step S401 and terminate the operation.
  • each functional block shown in FIG. 1 is completed individually. That is, the operation of the single image selection unit 10 is completed only by the single image selection unit 10, and the same procedure has been described below. good.
  • the selected M images are immediately transferred to the synthesis unit 20, and a synthesized image of the transferred M images is obtained. may be created.
  • the created synthetic image is immediately transferred to the evaluating section 30, and the comprehensive evaluation value thereof may be determined.
  • all possible combinations of M images are selected from given N images, and each set of M images is , a synthetic image is obtained for all permutations of arranging the images in the template, and the total evaluation value is determined.
  • This method is excellent in that it ensures that a composite image with the largest possible overall score is obtained, but as the number of N increases, the number of composite images for which the overall score must be determined becomes exponential.
  • the computational load for information processing becomes heavy.
  • the information processing apparatus 200 in the information processing apparatus 200 according to the second embodiment of the present invention shown in FIG. 10, it is intended to select a synthesized image showing a reasonably high overall evaluation value with a smaller calculation load.
  • components common to or corresponding to the information processing 100 according to the first embodiment are denoted by the same reference numerals, and the following points differ from the information processing 100 according to the first embodiment. Only the first embodiment will be described, and the description of the first embodiment will be used for common points.
  • the information processing device 200 uses a specific algorithm to create and evaluate one or more known synthetic images based on known comprehensive evaluation values without determining overall evaluation values for all possible synthesized images. By successively determining the composite image that should be exploratory, it is configured to discover the composite image that gives the highest overall evaluation value or is thought to give the highest overall evaluation value.
  • the composite image selection unit 40 of the information processing apparatus 200 designates a set of M images to be selected next to the single image selection unit 10 based on the already obtained composite image and its comprehensive evaluation value. , and for the synthesizer 20, the permutation of the arrangement of the images to be selected next is designated.
  • a set of one or more M images arbitrarily selected and a synthetic image obtained by permutation of arrangement with respect to the template are obtained at first, and thereafter a higher comprehensive evaluation value is obtained based on this.
  • the information processing apparatus 200 can be used when the number N of given images is large, when the number of permutations of images is large, and when the number M of frames included in a template is large. Secondly, it is useful in trying to obtain a synthetic image that is attractive to the viewer at a reasonable computational time and computational load. On the other hand, when N and M are relatively small values, the use of the information processing apparatus 100 according to the first embodiment has the advantage of ensuring that an optimal composite image is obtained.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Processing Or Creating Images (AREA)
  • Editing Of Facsimile Originals (AREA)
  • Image Analysis (AREA)
PCT/JP2021/040851 2021-11-05 2021-11-05 情報処理装置、情報処理方法及び情報処理プログラム Ceased WO2023079702A1 (ja)

Priority Applications (6)

Application Number Priority Date Filing Date Title
JP2022557884A JP7216255B1 (ja) 2021-11-05 2021-11-05 情報処理装置、情報処理方法及び情報処理プログラム
EP21950362.0A EP4203451B1 (en) 2021-11-05 2021-11-05 Information processing device, information processing method, and information processing program
PCT/JP2021/040851 WO2023079702A1 (ja) 2021-11-05 2021-11-05 情報処理装置、情報処理方法及び情報処理プログラム
US18/005,056 US12159374B2 (en) 2021-11-05 2021-11-05 Information-processing device, information-processing method and non-transitory computer-readable information recording medium
JP2023006661A JP7303953B2 (ja) 2021-11-05 2023-01-19 情報処理装置、情報処理方法及び情報処理プログラム
JP2023103153A JP7526318B2 (ja) 2021-11-05 2023-06-23 情報処理装置、情報処理方法及び情報処理プログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/040851 WO2023079702A1 (ja) 2021-11-05 2021-11-05 情報処理装置、情報処理方法及び情報処理プログラム

Publications (1)

Publication Number Publication Date
WO2023079702A1 true WO2023079702A1 (ja) 2023-05-11

Family

ID=85111647

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/040851 Ceased WO2023079702A1 (ja) 2021-11-05 2021-11-05 情報処理装置、情報処理方法及び情報処理プログラム

Country Status (4)

Country Link
US (1) US12159374B2 (https=)
EP (1) EP4203451B1 (https=)
JP (1) JP7216255B1 (https=)
WO (1) WO2023079702A1 (https=)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7594075B1 (ja) 2023-11-17 2024-12-03 楽天グループ株式会社 画像生成装置、画像生成方法、および画像生成プログラム

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015162850A (ja) * 2014-02-28 2015-09-07 富士フイルム株式会社 画像合成装置，ならびにその方法，そのプログラム，およびそのプログラムを格納した記録媒体
JP2021027555A (ja) * 2019-08-08 2021-02-22 キヤノン株式会社 画像処理装置、制御方法及びプログラム
JP2021051595A (ja) * 2019-09-25 2021-04-01 大日本印刷株式会社 情報処理装置、情報処理方法及びプログラム

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104584081B (zh) * 2012-09-03 2018-04-27 索尼公司 信息处理设备、信息处理方法和程序
JP6282065B2 (ja) 2013-09-05 2018-02-21 キヤノン株式会社 画像処理装置、画像処理方法、及びプログラム
US11203425B2 (en) * 2016-06-30 2021-12-21 Skydio, Inc. Unmanned aerial vehicle inspection system
JP6946010B2 (ja) 2017-01-31 2021-10-06 キヤノン株式会社 情報処理装置、情報処理方法、及びプログラム
JP7493919B2 (ja) 2019-08-08 2024-06-03 キヤノン株式会社 画像処理装置、制御方法及びプログラム
DE102019131971A1 (de) * 2019-11-26 2021-05-27 Connaught Electronics Ltd. Ein Bildverarbeitungsmodul

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015162850A (ja) * 2014-02-28 2015-09-07 富士フイルム株式会社 画像合成装置，ならびにその方法，そのプログラム，およびそのプログラムを格納した記録媒体
JP2021027555A (ja) * 2019-08-08 2021-02-22 キヤノン株式会社 画像処理装置、制御方法及びプログラム
JP2021051595A (ja) * 2019-09-25 2021-04-01 大日本印刷株式会社 情報処理装置、情報処理方法及びプログラム

Also Published As

Publication number Publication date
EP4203451B1 (en) 2026-03-18
US20240249387A1 (en) 2024-07-25
JPWO2023079702A1 (https=) 2023-05-11
US12159374B2 (en) 2024-12-03
JP7216255B1 (ja) 2023-01-31
EP4203451A4 (en) 2023-07-26
EP4203451A1 (en) 2023-06-28

Similar Documents

Publication Publication Date Title
Skublewska-Paszkowska et al. 3D technologies for intangible cultural heritage preservation—literature review for selected databases
US8294710B2 (en) Extensible map with pluggable modes
CN109863537A (zh) 风格化输入图像
Deng et al. Research on Artificial Intelligence Interaction in Computer‐Aided Arts and Crafts
CN109242946A (zh) 根据单个图像估计深度
WO2009154482A1 (en) A method and system of graphically representing discrete data as a continuous surface
JPH10283414A (ja) 消費者行動モデルを作成する方法
Tatzgern Situated visualization in augmented reality
KR102345753B1 (ko) 서로 다른 복수의 인공신경망을 이용하여 데이터를 지능적으로 시각화하는 방법
CN109918162B (zh) 一种可学习的海量信息高维图形交互式展示方法
JP7216255B1 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
Kuang et al. Preserving architectural heritage in urban renewal: a stable diffusion model framework for automated historical facade generation
Huang et al. Application of style transfer algorithm in the integration of traditional garden and modern design elements
JP7303953B2 (ja) 情報処理装置、情報処理方法及び情報処理プログラム
CN116244045B (zh) 多任务模型的子任务概率计算方法及装置、介质及终端
Matsubara et al. Kansei Analysis of the Japanese Residential Garden and Development of a Low‐Cost Virtual Reality Kansei Engineering System for Gardens
Jin et al. [Retracted] Machinery Changes and Challenges of Architecture and Landscape Design in the Virtual Reality Perspective
JP7325775B2 (ja) 画像処理システム、方法、及びプログラム
Zhang Experimental Analysis of Style Transfer and Target Detection in Interactive Art on Smartphones
He Interior Design Layout Optimization Based On Convolutional Neural Network
Löffelmann et al. Fast visualization of 2D dynamical systems by the use of virtual ink droplets
CN119379946B (zh) 基于aigc技术的元宇宙场景自动化生成与优化平台
Ware et al. Optimizing for visual cognition in high performance scientific computing
Ni Research on somatic interactive exhibition design of lacquer artwork based on multi-source data analysis
Lei Applications Research of Virtual Reality Technology in Environmental Art Design

Legal Events

Date Code Title Description
WWE Wipo information: entry into national phase

Ref document number: 2022557884

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18005056

Country of ref document: US

ENP Entry into the national phase

Ref document number: 2021950362

Country of ref document: EP

Effective date: 20230123

NENP Non-entry into the national phase

Ref country code: DE

WWG Wipo information: grant in national office

Ref document number: 2021950362

Country of ref document: EP