WO2022180782A1 - Information processing device, information processing method, and information processing program - Google Patents

Information processing device, information processing method, and information processing program Download PDF

Info

Publication number
WO2022180782A1
WO2022180782A1 PCT/JP2021/007334 JP2021007334W WO2022180782A1 WO 2022180782 A1 WO2022180782 A1 WO 2022180782A1 JP 2021007334 W JP2021007334 W JP 2021007334W WO 2022180782 A1 WO2022180782 A1 WO 2022180782A1
Authority
WO
WIPO (PCT)
Prior art keywords
style
information processing
image
conversion
content image
Prior art date
Application number
PCT/JP2021/007334
Other languages
French (fr)
Japanese (ja)
Inventor
翔大 山田
弘員 柿沼
秀信 長田
浩太 日高
Original Assignee
日本電信電話株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 日本電信電話株式会社 filed Critical 日本電信電話株式会社
Priority to PCT/JP2021/007334 priority Critical patent/WO2022180782A1/en
Publication of WO2022180782A1 publication Critical patent/WO2022180782A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing

Definitions

  • the present invention relates to an information processing device, an information processing method, and an information processing program.
  • Non-Patent Document 1 There is a technique for converting the style of a content image (texture, painting style) with the style of a style image (Non-Patent Document 1).
  • style conversion if the granularity of the content image and the style image is different, the values of various parameters included in the style conversion algorithm are adjusted, such as adjusting the parameter values related to the size of the image and the granularity of the style.
  • the content image after style conversion is similar to the style of the reference range specified by the user on the style image side in terms of human perceptual scale, that is, the scale of the content image and style image match. , it is necessary to adjust the values of various parameters. Conventionally, however, the values of various parameters have been manually adjusted while visually confirming the state of style conversion.
  • the present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of improving the work efficiency of style conversion.
  • An information processing apparatus includes a conversion unit that converts the style of a content image with the style of the style image, a feature amount of a predetermined range of the content image after style conversion, and a feature of the predetermined range of the style image. and a search unit for searching for values of various parameters for style conversion that minimize the difference between the amount and the amount.
  • An information processing method is an information processing method performed by an information processing apparatus, comprising: converting a style of a content image into a style of the style image; and a step of searching for the value of various parameters for style conversion that minimizes the difference between the feature amount in the predetermined range of the style image.
  • An information processing program is an information processing program that causes a computer to function as the information processing apparatus.
  • FIG. 1 is a diagram showing a functional block configuration of an information processing apparatus.
  • FIG. 2 is a diagram illustrating a processing flow of the information processing apparatus;
  • FIG. 3 is a reference diagram for explaining the processing flow.
  • FIG. 4 is a diagram showing the hardware configuration of the information processing device.
  • the style A of the entire content image is converted by the style B of the style image, and the feature amount of the content image having the style A' after the style conversion and the feature amount of the style image of the predetermined range.
  • FIG. 1 is a diagram showing a functional block configuration of an information processing device 1 according to this embodiment.
  • the information processing device 1 is a style conversion device that converts the style of a content image using a style image.
  • the information processing device 1 includes an input unit 11 , a conversion unit 12 , a search unit 13 , an output unit 14 , a storage unit 15 and a display unit 16 .
  • the input unit 11 is a functional unit that inputs the range specified by the user for the style A content image displayed on the display unit 16 as the reference range R1.
  • the input unit 11 is a functional unit that inputs a range specified by the user for the style image of style B displayed on the display unit 16 as a reference range R2.
  • the conversion unit 12 is a functional unit that converts the style A of the content image into the style B of the style image.
  • the conversion unit 12 is a functional unit that converts the style A of the content image using the values of various parameters for style conversion searched by the search unit 13 .
  • the searching unit 13 selects various parameters for style conversion that minimize the difference between the feature amount of the reference range R1 of the content image of style A′ after style conversion and the feature amount of the reference range R2 of the style image of style B. It is a functional part that searches for the value of
  • the output unit 14 is a functional unit that outputs to the display unit 16 a content image whose style has been converted using the values of various parameters for style conversion searched by the search unit 13 .
  • the storage unit 15 is a functional unit that stores the values of various parameters for style conversion searched by the search unit 13 .
  • the display unit 16 is a functional unit that displays the style-converted content image output by the output unit 14 .
  • the display unit 16 is, for example, a touch panel display.
  • the display unit 16 also displays a content image before style conversion, a style image, and the like, and provides a GUI such as finger touch.
  • the functional division of the above functional units is an example.
  • the conversion unit 12, search unit 13, and output unit 14 may be combined into one processing unit.
  • FIG. 2 is a diagram showing a processing flow of the information processing device 1.
  • a style A content image of a domestic cat and a style B style image of a polygonal pattern are displayed on the left and right sides.
  • a user wishes to convert a cat image into a polygonal image.
  • ROI Region of Interest
  • Step S2 the input unit 11 determines whether or not the user has specified a range within the content image.
  • the determination method is, for example, a method of displaying a selection screen for whether or not to specify a range within the content image and making a determination based on the selection result of the user, or a method of determining that no range is specified after a certain period of time has elapsed after execution of step S1.
  • Step S3 When the user specifies a range within the content image, the input unit 11 inputs the range specified by the user within the content image as the reference range R1.
  • Step S4 If the user does not specify a range within the content image, the input unit 11 inputs a range randomly sampled (selected) from the content image as the reference range R1.
  • the sampling range is the range of all or part of the content image.
  • Step S5 Next, the conversion unit 12 temporarily converts the style A of the content image into the style B of the style image (see FIG. 3).
  • Step S6 the searching unit 13 selects a style conversion style image that minimizes the difference between the feature amount of the reference range R1 of the content image of style A′ after style conversion and the feature amount of the reference range R2 of the style image of style B.
  • Various parameters include patch size, patch stride, style image size, and the like.
  • the reference range in the image is defined as a feature quantity encoded by VGG 19 (a convolutional neural network with a depth of 19 layers) as a feature quantity that humans perceptually recognize.
  • VGG 19 a convolutional neural network with a depth of 19 layers
  • the distance dist ⁇ VGG19(R1), VGG19(R2) ⁇ is searched for the parameter value that makes the shortest. Since the searching unit 13 searches for the values of various parameters for such style conversion, it is possible to simplify the work of adjusting the sense of scale between the content image and the style image.
  • search process can be realized using existing technologies such as the Grid search function that automatically optimizes the parameters of the machine learning model.
  • Step S7 Next, the conversion unit 12 converts the style A of the content image using the values of the searched various parameters for style conversion.
  • Step S8 Finally, the output unit 14 outputs the content image subjected to style conversion in step S7 to the display unit 16 for preview display. Further, the searching unit 13 saves the searched various parameter values of the style conversion in the storage unit 15 .
  • the style of the content image is converted by the style of the style image, and the difference between the feature amount of the predetermined range of the content image after style conversion and the feature amount of the predetermined range of the style image is minimized. Since the values of various parameters for style conversion are searched for, it is possible to simplify the task of adjusting the sense of scale between the content image and the style image, thereby reducing the operation cost of style conversion.
  • the range on the content image side to be style-converted is automatically selected, and the values of various parameters for style conversion are is automatically derived
  • style conversion can be performed intuitively simply by specifying the ROI from the style image. Since the user can intuitively perform the style conversion process while looking at the style without being conscious of the parameter values themselves, work efficiency is greatly improved.
  • the information processing apparatus 1 of the present embodiment described above includes, for example, a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in FIG. It can be realized using a general-purpose computer system.
  • Memory 902 and storage 903 are storage devices.
  • each function of the information processing apparatus 1 is realized by executing a predetermined program loaded on the memory 902 by the CPU 901 .
  • the information processing device 1 may be implemented by one computer.
  • the information processing device 1 may be implemented by a plurality of computers.
  • the information processing device 1 may be a virtual machine implemented in a computer.
  • Programs for the information processing device 1 can be stored in computer-readable recording media such as HDDs, SSDs, USB memories, CDs, and DVDs.
  • the program for information processing device 1 can also be distributed via a communication network.

Abstract

An information processing device 1 is provided with: a transformation unit 12 for transforming the style of a content image using the style of a style image; and a finding unit 13 for finding values of various style transformation parameters such that the difference between a feature quantity of a prescribed region of the content image, the style of which has been transformed, and a feature quantity of a prescribed region of the style image is minimized.

Description

情報処理装置、情報処理方法、及び、情報処理プログラムInformation processing device, information processing method, and information processing program
　本発明は、情報処理装置、情報処理方法、及び、情報処理プログラムに関する。 The present invention relates to an information processing device, an information processing method, and an information processing program.
　コンテンツ画像のスタイル（テクスチャ、画風）をスタイル画像のスタイルで変換する技術がある（非特許文献１）。スタイル変換では、コンテンツ画像とスタイル画像の粒度が異なる場合は画像の大きさやスタイルの粒度に係るパラメータの値を調整する等、スタイル変換のアルゴリズムに含まれる各種パラメータの値が調整される。 There is a technique for converting the style of a content image (texture, painting style) with the style of a style image (Non-Patent Document 1). In style conversion, if the granularity of the content image and the style image is different, the values of various parameters included in the style conversion algorithm are adjusted, such as adjusting the parameter values related to the size of the image and the granularity of the style.
　スタイル変換では、スタイル変換後のコンテンツ画像がスタイル画像側でユーザの指定した参照範囲のスタイルと人間の知覚的な尺度において類似するように、つまりコンテンツ画像とスタイル画像とのスケール感が合うように、各種パラメータの値を調整する必要がある。しかし、従来は、スタイル変換の状態を目視で確認しながら各種パラメータの値を手作業で調整していたため、スケール感を合わせる作業効率が悪いという課題があった。 In style conversion, the content image after style conversion is similar to the style of the reference range specified by the user on the style image side in terms of human perceptual scale, that is, the scale of the content image and style image match. , it is necessary to adjust the values of various parameters. Conventionally, however, the values of various parameters have been manually adjusted while visually confirming the state of style conversion.
　本発明は、上記事情に鑑みてなされたものであり、本発明の目的は、スタイル変換の作業効率を改善可能な技術を提供することである。 The present invention has been made in view of the above circumstances, and an object of the present invention is to provide a technique capable of improving the work efficiency of style conversion.
　本発明の一態様の情報処理装置は、コンテンツ画像のスタイルをスタイル画像のスタイルで変換する変換部と、スタイル変換後の前記コンテンツ画像の所定範囲の特徴量と、前記スタイル画像の所定範囲の特徴量と、の差が最も小さくなるスタイル変換用の各種パラメータの値を探索する探索部と、を備える。 An information processing apparatus according to an aspect of the present invention includes a conversion unit that converts the style of a content image with the style of the style image, a feature amount of a predetermined range of the content image after style conversion, and a feature of the predetermined range of the style image. and a search unit for searching for values of various parameters for style conversion that minimize the difference between the amount and the amount.
　本発明の一態様の情報処理方法は、情報処理装置で行う情報処理方法において、コンテンツ画像のスタイルをスタイル画像のスタイルで変換するステップと、スタイル変換後の前記コンテンツ画像の所定範囲の特徴量と、前記スタイル画像の所定範囲の特徴量と、の差が最も小さくなるスタイル変換用の各種パラメータの値を探索するステップと、を行う。 An information processing method according to one aspect of the present invention is an information processing method performed by an information processing apparatus, comprising: converting a style of a content image into a style of the style image; and a step of searching for the value of various parameters for style conversion that minimizes the difference between the feature amount in the predetermined range of the style image.
　本発明の一態様の情報処理プログラムは、上記情報処理装置としてコンピュータを機能させる情報処理プログラムである。 An information processing program according to one aspect of the present invention is an information processing program that causes a computer to function as the information processing apparatus.
　本発明によれば、スタイル変換の作業効率を向上可能な技術を提供できる。 According to the present invention, it is possible to provide a technology that can improve the work efficiency of style conversion.
図１は、情報処理装置の機能ブロック構成を示す図である。FIG. 1 is a diagram showing a functional block configuration of an information processing apparatus. 図２は、情報処理装置の処理フローを示す図である。FIG. 2 is a diagram illustrating a processing flow of the information processing apparatus; 図３は、処理フロー説明時の参照図である。FIG. 3 is a reference diagram for explaining the processing flow. 図４は、情報処理装置のハードウェア構成を示す図である。FIG. 4 is a diagram showing the hardware configuration of the information processing device.
　以下、図面を参照して、本発明の実施形態を説明する。図面の記載において同一部分には同一符号を付し説明を省略する。 Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the description of the drawings, the same parts are denoted by the same reference numerals, and the description thereof is omitted.
　［発明の概要］
　本発明は、コンテンツ画像全体のスタイルＡをスタイル画像のスタイルＢで変換し、スタイル変換後のスタイルＡ’を有するコンテンツ画像の所定範囲の特徴量と、スタイル画像の所定範囲の特徴量と、の差が最も小さくなる（類似度が最も高くなる）スタイル変換用の各種パラメータの値を自動探索する。このようなスタイル変換用の各種パラメータの値を自動探索するので、コンテンツ画像とスタイル画像とのスケール感を合わせる作業を簡略化でき、スタイル変換の運用コストを低減できる。その結果、スタイル変換の作業効率を向上可能な技術を提供できる。 [Summary of Invention]
In the present invention, the style A of the entire content image is converted by the style B of the style image, and the feature amount of the content image having the style A' after the style conversion and the feature amount of the style image of the predetermined range. Automatically search for the values of various parameters for style conversion that minimize the difference (maximize the similarity). Since the values of various parameters for style conversion are automatically searched for, it is possible to simplify the task of adjusting the sense of scale between the content image and the style image, thereby reducing the operation cost of style conversion. As a result, it is possible to provide a technique capable of improving the work efficiency of style conversion.
　［情報処理装置の構成］
　図１は、本実施形態に係る情報処理装置１の機能ブロック構成を示す図である。当該情報処理装置１は、スタイル画像を用いてコンテンツ画像をスタイル変換するスタイル変換装置である。当該情報処理装置１は、入力部１１と、変換部１２と、探索部１３と、出力部１４と、記憶部１５と、表示部１６と、を備える。 [Configuration of information processing device]
FIG. 1 is a diagram showing a functional block configuration of an information processing device 1 according to this embodiment. The information processing device 1 is a style conversion device that converts the style of a content image using a style image. The information processing device 1 includes an input unit 11 , a conversion unit 12 , a search unit 13 , an output unit 14 , a storage unit 15 and a display unit 16 .
　入力部１１は、表示部１６に表示されたスタイルＡのコンテンツ画像に対してユーザが指定した範囲を参照範囲Ｒ１として入力する機能部である。また、入力部１１は、表示部１６に表示されたスタイルＢのスタイル画像に対してユーザが指定した範囲を参照範囲Ｒ２として入力する機能部である。 The input unit 11 is a functional unit that inputs the range specified by the user for the style A content image displayed on the display unit 16 as the reference range R1. The input unit 11 is a functional unit that inputs a range specified by the user for the style image of style B displayed on the display unit 16 as a reference range R2.
　変換部１２は、コンテンツ画像のスタイルＡをスタイル画像のスタイルＢで変換する機能部である。また、変換部１２は、探索部１３が探索したスタイル変換用の各種パラメータの値でコンテンツ画像のスタイルＡを変換する機能部である。 The conversion unit 12 is a functional unit that converts the style A of the content image into the style B of the style image. The conversion unit 12 is a functional unit that converts the style A of the content image using the values of various parameters for style conversion searched by the search unit 13 .
　探索部１３は、スタイル変換後のスタイルＡ’のコンテンツ画像の参照範囲Ｒ１の特徴量と、スタイルＢのスタイル画像の参照範囲Ｒ２の特徴量と、の差が最も小さくなるスタイル変換用の各種パラメータの値を探索する機能部である。 The searching unit 13 selects various parameters for style conversion that minimize the difference between the feature amount of the reference range R1 of the content image of style A′ after style conversion and the feature amount of the reference range R2 of the style image of style B. It is a functional part that searches for the value of
　出力部１４は、探索部１３が探索したスタイル変換用の各種パラメータの値でスタイル変換を行ったコンテンツ画像を表示部１６に出力する機能部である。 The output unit 14 is a functional unit that outputs to the display unit 16 a content image whose style has been converted using the values of various parameters for style conversion searched by the search unit 13 .
　記憶部１５は、探索部１３が探索したスタイル変換用の各種パラメータの値を記憶する機能部である。 The storage unit 15 is a functional unit that stores the values of various parameters for style conversion searched by the search unit 13 .
　表示部１６は、出力部１４が出力したスタイル変換後のコンテンツ画像を表示する機能部である。表示部１６は、例えば、タッチパネル式のディスプレイである。表示部１６は、スタイル変換前のコンテンツ画像、スタイル画像等も表示し、指タッチ等のＧＵＩを提供する。 The display unit 16 is a functional unit that displays the style-converted content image output by the output unit 14 . The display unit 16 is, for example, a touch panel display. The display unit 16 also displays a content image before style conversion, a style image, and the like, and provides a GUI such as finger touch.
　なお、上記機能部の機能分担は例である。例えば、変換部１２と探索部１３と出力部１４を合わせて１つの処理部としてもよい。 It should be noted that the functional division of the above functional units is an example. For example, the conversion unit 12, search unit 13, and output unit 14 may be combined into one processing unit.
　［情報処理装置の動作］
　図２は、情報処理装置１の処理フローを示す図である。表示部１６には、例えば、飼い猫を撮影したスタイルＡのコンテンツ画像と、ポリゴン模様のスタイルＢのスタイル画像と、が左右に表示されている。ユーザは、猫の画像をポリゴン調に変換したいと考えている。 [Operation of information processing device]
FIG. 2 is a diagram showing a processing flow of the information processing device 1. As shown in FIG. On the display unit 16, for example, a style A content image of a domestic cat and a style B style image of a polygonal pattern are displayed on the left and right sides. A user wishes to convert a cat image into a polygonal image.
　ステップＳ１；
　まず、入力部１１は、スタイル画像内でユーザが指定した範囲（ＲＯＩ；Region of Interest）を参照範囲Ｒ２として入力する。 Step S1;
First, the input unit 11 inputs a range (ROI; Region of Interest) designated by the user in the style image as a reference range R2.
　ステップＳ２；
　次に、入力部１１は、コンテンツ画像内でユーザが範囲を指定したか否かを判定する。判定方法は、例えば、コンテンツ画像内で範囲を指定するか否かの選択画面を表示してユーザの選択結果に基づき判定する方法、ステップＳ１の実行後に一定時間経過後に範囲指定なしと決定する方法がある。 Step S2;
Next, the input unit 11 determines whether or not the user has specified a range within the content image. The determination method is, for example, a method of displaying a selection screen for whether or not to specify a range within the content image and making a determination based on the selection result of the user, or a method of determining that no range is specified after a certain period of time has elapsed after execution of step S1. There is
　ステップＳ３；
　コンテンツ画像内でユーザが範囲を指定した場合、入力部１１は、コンテンツ画像内でユーザが指定した範囲を参照範囲Ｒ１として入力する。 Step S3;
When the user specifies a range within the content image, the input unit 11 inputs the range specified by the user within the content image as the reference range R1.
　ステップＳ４；
　コンテンツ画像内でユーザが範囲を指定しなかった場合、入力部１１は、コンテンツ画像の中からランダムにサンプリング（選択）した範囲を参照範囲Ｒ１として入力する。サンプリング範囲は、コンテンツ画像の全部又は一部の範囲である。 Step S4;
If the user does not specify a range within the content image, the input unit 11 inputs a range randomly sampled (selected) from the content image as the reference range R1. The sampling range is the range of all or part of the content image.
　ステップＳ５；
　次に、変換部１２は、コンテンツ画像のスタイルＡをスタイル画像のスタイルＢで一旦変換する（図３参照）。 Step S5;
Next, the conversion unit 12 temporarily converts the style A of the content image into the style B of the style image (see FIG. 3).
　ステップＳ６；
　次に、探索部１３は、スタイル変換後のスタイルＡ’のコンテンツ画像の参照範囲Ｒ１の特徴量と、スタイルＢのスタイル画像の参照範囲Ｒ２の特徴量と、の差が最小となるスタイル変換用の各種パラメータの値を探索する（図３参照）。各種パラメータとは、パッチサイズ、パッチストライド、スタイル画像サイズ等である。 Step S6;
Next, the searching unit 13 selects a style conversion style image that minimizes the difference between the feature amount of the reference range R1 of the content image of style A′ after style conversion and the feature amount of the reference range R2 of the style image of style B. (see FIG. 3). Various parameters include patch size, patch stride, style image size, and the like.
　例えば、画像内の参照範囲をＶＧＧ１９（深さが１９層の畳み込みニューラルネットワーク）でエンコードした特徴量を人が知覚的に認識する特徴量と定義し、探索部１３は、コンテンツ画像の参照範囲Ｒ１をＶＧＧ１９でエンコードした特徴量と、スタイル画像の参照範囲Ｒ２をＶＧＧ１９でエンコードした特徴量と、の差が最小となるパラメータの値、つまり２つの特徴量間の距離ｄｉｓｔ＝｛ＶＧＧ１９（Ｒ１），ＶＧＧ１９（Ｒ２）｝が最も短くなるパラメータの値を探索する。このようなスタイル変換用の各種パラメータの値を探索部１３が探索するので、コンテンツ画像とスタイル画像とのスケール感を合わせる作業を簡略化できる。 For example, the reference range in the image is defined as a feature quantity encoded by VGG 19 (a convolutional neural network with a depth of 19 layers) as a feature quantity that humans perceptually recognize. is the minimum difference between the VGG19-encoded feature amount and the VGG19-encoded feature amount of the reference range R2 of the style image, that is, the distance dist={VGG19(R1), VGG19(R2)} is searched for the parameter value that makes the shortest. Since the searching unit 13 searches for the values of various parameters for such style conversion, it is possible to simplify the work of adjusting the sense of scale between the content image and the style image.
　なお、探索処理については、機械学習モデルのパラメータを自動的に最適化するグリッドサーチ（Grid search）機能等、既存技術を用いて実現可能である。 It should be noted that the search process can be realized using existing technologies such as the Grid search function that automatically optimizes the parameters of the machine learning model.
　ステップＳ７；
　次に、変換部１２は、探索したスタイル変換用の各種パラメータの値でコンテンツ画像のスタイルＡを変換する。 Step S7;
Next, the conversion unit 12 converts the style A of the content image using the values of the searched various parameters for style conversion.
　ステップＳ８；
　最後に、出力部１４は、ステップＳ７でスタイル変換を行ったコンテンツ画像を表示部１６に出力してプレビュー表示する。また、探索部１３は、探索したスタイル変換の各種パラメータの値を記憶部１５に保存する。 Step S8;
Finally, the output unit 14 outputs the content image subjected to style conversion in step S7 to the display unit 16 for preview display. Further, the searching unit 13 saves the searched various parameter values of the style conversion in the storage unit 15 .
　［効果］
　本実施形態によれば、コンテンツ画像のスタイルをスタイル画像のスタイルで変換し、スタイル変換後のコンテンツ画像の所定範囲の特徴量と、スタイル画像の所定範囲の特徴量と、の差が最も小さくなるスタイル変換用の各種パラメータの値を探索するので、コンテンツ画像とスタイル画像とのスケール感を合わせる作業を簡略化でき、スタイル変換の運用コストを低減できる。 [effect]
According to this embodiment, the style of the content image is converted by the style of the style image, and the difference between the feature amount of the predetermined range of the content image after style conversion and the feature amount of the predetermined range of the style image is minimized. Since the values of various parameters for style conversion are searched for, it is possible to simplify the task of adjusting the sense of scale between the content image and the style image, thereby reducing the operation cost of style conversion.
　また、本実施形態によれば、ユーザがスタイル画像の範囲（ＲＯＩ）を指定すると、スタイル変換の対象となるコンテンツ画像側の範囲を自動的に選択し、さらに、スタイル変換用の各種パラメータの値も自動的に導出するので、スタイル変換において、スタイル画像からＲＯＩを指定するだけで、直感的なスタイル変換が可能となる。スタイル変換処理を、ユーザがスタイルを見ながら直観的に実施でき、パラメータ値自体を意識せずに実施できるので、作業効率が大幅に向上する。 Further, according to this embodiment, when the user designates the range (ROI) of the style image, the range on the content image side to be style-converted is automatically selected, and the values of various parameters for style conversion are is automatically derived, style conversion can be performed intuitively simply by specifying the ROI from the style image. Since the user can intuitively perform the style conversion process while looking at the style without being conscious of the parameter values themselves, work efficiency is greatly improved.
　以上より、スタイル変換の作業効率を向上可能な技術を提供できる。 From the above, we can provide a technology that can improve the work efficiency of style conversion.
　［その他］
　本発明は、上記実施形態に限定されない。本発明は、本発明の要旨の範囲内で数々の変形が可能である。 [others]
The invention is not limited to the above embodiments. The present invention can be modified in many ways within the scope of the gist of the present invention.
　上記説明した本実施形態の情報処理装置１は、例えば、図４に示すように、ＣＰＵ９０１と、メモリ９０２と、ストレージ９０３と、通信装置９０４と、入力装置９０５と、出力装置９０６と、を備えた汎用的なコンピュータシステムを用いて実現できる。メモリ９０２及びストレージ９０３は、記憶装置である。当該コンピュータシステムにおいて、ＣＰＵ９０１がメモリ９０２上にロードされた所定のプログラムを実行することにより、情報処理装置１の各機能が実現される。 The information processing apparatus 1 of the present embodiment described above includes, for example, a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in FIG. It can be realized using a general-purpose computer system. Memory 902 and storage 903 are storage devices. In the computer system, each function of the information processing apparatus 1 is realized by executing a predetermined program loaded on the memory 902 by the CPU 901 .
　情報処理装置１は、１つのコンピュータで実装されてもよい。情報処理装置１は、複数のコンピュータで実装されてもよい。情報処理装置１は、コンピュータに実装される仮想マシンであってもよい。情報処理装置１用のプログラムは、ＨＤＤ、ＳＳＤ、ＵＳＢメモリ、ＣＤ、ＤＶＤ等のコンピュータ読取り可能な記録媒体に記憶できる。情報処理装置１用のプログラムは、通信ネットワークを介して配信することもできる。 The information processing device 1 may be implemented by one computer. The information processing device 1 may be implemented by a plurality of computers. The information processing device 1 may be a virtual machine implemented in a computer. Programs for the information processing device 1 can be stored in computer-readable recording media such as HDDs, SSDs, USB memories, CDs, and DVDs. The program for information processing device 1 can also be distributed via a communication network.
　１：情報処理装置
　１１：入力部
　１２：変換部
　１３：探索部
　１４：出力部
　１５：記憶部
　１６：表示部
　９０１：ＣＰＵ
　９０２：メモリ
　９０３：ストレージ
　９０４：通信装置
　９０５：入力装置
　９０６：出力装置 1: Information Processing Device 11: Input Unit 12: Conversion Unit 13: Search Unit 14: Output Unit 15: Storage Unit 16: Display Unit 901: CPU
902: Memory 903: Storage 904: Communication device 905: Input device 906: Output device

Claims (5)

  1. 　コンテンツ画像のスタイルをスタイル画像のスタイルで変換する変換部と、
    　スタイル変換後の前記コンテンツ画像の所定範囲の特徴量と、前記スタイル画像の所定範囲の特徴量と、の差が最も小さくなるスタイル変換用の各種パラメータの値を探索する探索部と、
    　を備える情報処理装置。     a conversion unit that converts the style of the content image with the style of the style image;
    a searching unit that searches for values of various parameters for style conversion that minimize the difference between the feature amount of the predetermined range of the content image after style conversion and the feature amount of the predetermined range of the style image;
    Information processing device.
  2. 　前記探索部は、
    　スタイル変換後の前記コンテンツ画像の所定範囲をＶＧＧ１９でエンコードした特徴量と、前記スタイル画像の所定範囲をＶＧＧ１９でエンコードした特徴量と、の差が最も小さくなるスタイル変換用の各種パラメータの値を探索する請求項１に記載の情報処理装置。     The search unit is
    A search is made for values of various parameters for style conversion that minimize the difference between the feature amount obtained by encoding the predetermined range of the content image after style conversion with VGG19 and the feature amount obtained by encoding the predetermined range of the style image with VGG19. The information processing apparatus according to claim 1.
  3. 　前記コンテンツ画像の所定範囲は、
    　前記コンテンツ画像の中からランダムに選択した範囲である請求項１又は２に記載の情報処理装置。     The predetermined range of the content image is
    3. The information processing apparatus according to claim 1, wherein the range is randomly selected from the content image.
  4. 　情報処理装置で行う情報処理方法において、
    　コンテンツ画像のスタイルをスタイル画像のスタイルで変換するステップと、
    　スタイル変換後の前記コンテンツ画像の所定範囲の特徴量と、前記スタイル画像の所定範囲の特徴量と、の差が最も小さくなるスタイル変換用の各種パラメータの値を探索するステップと、
    　を行う情報処理方法。     In the information processing method performed by the information processing device,
    transforming the style of the content image with the style of the style image;
    a step of searching for values of various parameters for style conversion that minimize the difference between the feature amount of the predetermined range of the content image after style conversion and the feature amount of the predetermined range of the style image;
    Information processing method that performs
  5. 　請求項１乃至３のいずれかに記載の情報処理装置としてコンピュータを機能させる情報処理プログラム。 An information processing program that causes a computer to function as the information processing apparatus according to any one of claims 1 to 3.
PCT/JP2021/007334 2021-02-26 2021-02-26 Information processing device, information processing method, and information processing program WO2022180782A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/007334 WO2022180782A1 (en) 2021-02-26 2021-02-26 Information processing device, information processing method, and information processing program

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2021/007334 WO2022180782A1 (en) 2021-02-26 2021-02-26 Information processing device, information processing method, and information processing program

Publications (1)

Publication Number Publication Date
WO2022180782A1 true WO2022180782A1 (en) 2022-09-01

Family

ID=83048996

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/007334 WO2022180782A1 (en) 2021-02-26 2021-02-26 Information processing device, information processing method, and information processing program

Country Status (1)

Country Link
WO (1) WO2022180782A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009163616A (en) * 2008-01-09 2009-07-23 Nippon Telegr & Teleph Corp <Ntt> Image interpolation device, image interpolation method, and image interpolation program
JP2011216027A (en) * 2010-04-01 2011-10-27 System Run:Kk Color drawing creation apparatus, color drawing creation method, and color drawing creation program
JP2012058773A (en) * 2010-09-03 2012-03-22 Toshiba Corp Image processing apparatus

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009163616A (en) * 2008-01-09 2009-07-23 Nippon Telegr & Teleph Corp <Ntt> Image interpolation device, image interpolation method, and image interpolation program
JP2011216027A (en) * 2010-04-01 2011-10-27 System Run:Kk Color drawing creation apparatus, color drawing creation method, and color drawing creation program
JP2012058773A (en) * 2010-09-03 2012-03-22 Toshiba Corp Image processing apparatus

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
LU, S. ET AL.: "Avatar-net: multi-scale zero-shot style transfer by feature decoratio n", IEEE /CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR, 18 June 2018 (2018-06-18), pages 8242 - 8250, XP033473747, DOI: 10.1109/CVPR.2018.00860 *
MIYAMOTO, RYU ET AL.: "Image restoration by energy minimization considering geometric transformation of texture and similar pattern position", INFORMATION PROCESSING SOCIETY OF JAPAN SYMPOSIUM IMAGE RECOGNITION AND UNDERSTANDING SYMPOSIUM (MIRU) 2011, 2. 2 EXTENSION OF ENERGY FUNCTION THAT ALLOWS CHANGES IN TEXTURE BRIGHTNESS AND GEOMETRIC TRANSFORMATIONS, 20 July 2011 (2011-07-20), pages 1096 - 1103, XP009539452 *
NORIHIKO KAWAI, TOMOKAZU SATO, NAOKAZU YOKOYA: "Real-time background image transformation for AR marker removal, 2 Overview of AR marker removal-3.2 Background image deformation due to motion interpolation", DENSHI JOUHOU TSUUSHIN GAKKAI GIJUTSU KENKYUU HOUKOKU - IEICE TECHNICAL REPORT , vol. 114, no. 410, 15 January 2015 (2015-01-15), JP , pages 25 - 30, XP009539451, ISSN: 0913-5685 *
TIAN, Q. C. ET AL.: "Fast patch-based style transfer of arbitrary style", COMPUTER VISION AND PATTERN RECOGNITION, 13 December 2016 (2016-12-13), pages 1 - 10, XP080744019, Retrieved from the Internet <URL:https://arxiv.org/abs/1612.04337> *

Similar Documents

Publication Publication Date Title
WO2017219900A1 (en) Video detection method, server and storage medium
US10810633B2 (en) Generating a shoppable video
AU2012227166B2 (en) Face feature vector construction
JP2019008778A (en) Captioning region of image
CN106293074B (en) Emotion recognition method and mobile terminal
US20150035827A1 (en) Information processing device, information processing method, and information processing system
US20150339348A1 (en) Search method and device
US8417026B2 (en) Gesture recognition methods and systems
JP2005157448A5 (en)
CN111160288A (en) Gesture key point detection method and device, computer equipment and storage medium
US9013485B2 (en) Systems and methods for synthesizing high fidelity stroke data for lower dimension input strokes
CN112464833A (en) Dynamic gesture recognition method, device, equipment and storage medium based on optical flow
CN111507239A (en) Local feature face recognition method based on image pyramid
US10558850B2 (en) Body information analysis apparatus and method of simulating face shape by using same
JP5094682B2 (en) Image processing apparatus, image processing method, and program
TW202004525A (en) Search method, electronic device and non-transitory computer-readable recording medium
CN112991151B (en) Image processing method, image generation method, apparatus, device, and medium
WO2022180782A1 (en) Information processing device, information processing method, and information processing program
US20160026613A1 (en) Processing image to identify object for insertion into document
US20240012966A1 (en) Method and system for providing a three-dimensional computer aided-design (cad) model in a cad environment
JP4916950B2 (en) Moving image comparison apparatus, moving image comparison method, and moving image comparison program
JP6987987B2 (en) Media feature comparison method and equipment
AU2019200269B2 (en) An interactive user interface and its corresponding engine for improving image completion quality
TW202004524A (en) Search method, electronic device and non-transitory computer-readable recording medium
WO2021176566A1 (en) Feature conversion device, image recognition system, feature conversion method, and non-transitory computer-readable medium

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21927884

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21927884

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP