JP2020057373A5

JP2020057373A5 -

Info

Publication number: JP2020057373A5
Application number: JP2019152000A
Authority: JP
Filing date: 2019-08-22
Publication date: 2022-08-17
Anticipated expiration: 2039-08-22

Description

本発明の一側面としての画像処理方法は、被写体の第一の画像を取得する工程と、前記第一の画像よりもノイズ量が多く、前記第一の画像よりも前記被写体の高周波成分を多く含む第二の画像を取得する工程と、前記第一の画像および前記第二の画像をニューラルネットワークに入力する工程と、前記ニューラルネットワークを用いて、前記第一の画像よりも前記被写体の高周波成分を多く含み前記第二の画像よりもノイズ量が少ない第三の画像を生成する工程とを有する。 An image processing method as one aspect of the present invention includes a step of obtaining a first image of a subject, and a noise amount larger than that of the first image, and a high frequency component of the subject larger than that of the first image. inputting the first image and the second image to a neural network; using the neural network to detect higher frequency components of the subject than the first image and generating a third image that contains more and has a lower amount of noise than the second image.

本発明の他の側面としての画像処理装置は、被写体の第一の画像と、前記第一の画像よりもノイズ量が多く、前記第一の画像よりも前記被写体の高周波成分を多く含む第二の画像を取得する取得部と、前記第一の画像および前記第二の画像をニューラルネットワークに入力して、前記ニューラルネットワークを用いて前記第一の画像よりも前記被写体の高周波成分を多く含み前記第二の画像よりもノイズ量が少ない第三の画像を生成する算出部とを有する。 An image processing apparatus as another aspect of the present invention comprises a first image of a subject, and a second image having a larger amount of noise than the first image and containing more high-frequency components of the subject than the first image . and an acquisition unit that acquires an image of the and a calculator for generating a third image with less noise than the second image.

本発明の他の側面としての撮像装置は、前記の画像処理装置と、前記第一の画像を撮像する第一の撮像部と、前記第二の画像を撮像する第二の撮像部とを有する。 An imaging device as another aspect of the present invention comprises the image processing device, a first imaging unit that captures the first image , and a second imaging unit that captures the second image . have

本発明の他の側面としての画像処理システムは、第一の装置と第二の装置を有する画像処理システムであって、前記第一の装置は、被写体の第一の画像と、前記第一の画像よりもノイズ量が多く、前記第一の画像よりも前記被写体の高周波成分を多く含む第二の画像を用いた画像処理を前記第二の装置に実行させるための要求を送信する送信手段を有し、前記第二の装置は、前記送信手段によって送信された前記要求を受信する受信手段と、前記第一の画像と前記第二の画像を取得する取得手段と、前記第一の画像および前記第二の画像をニューラルネットワークに入力して、前記ニューラルネットワークを用いて前記第一の画像よりも前記被写体の高周波成分を多く含み前記第二の画像よりもノイズ量が少ない第三の画像を生成する算出部とを有する。 An image processing system as another aspect of the present invention is an image processing system comprising a first device and a second device, wherein the first device comprises a first image of a subject and a transmitting means for transmitting a request to cause the second device to execute image processing using a second image that has a larger amount of noise than the image and contains more high-frequency components of the subject than the first image ; wherein the second device comprises: receiving means for receiving the request transmitted by the transmitting means; obtaining means for obtaining the first image and the second image; inputting the second image to a neural network, and using the neural network to generate a third image containing more high-frequency components of the subject than the first image and having less noise than the second image; and a calculation unit for generating.

本発明の他の側面としての学習済みモデルの製造方法は、被写体の第一の画像を取得する工程と、前記第一の画像よりもノイズ量が多く、前記第一の画像よりも前記被写体の高周波成分を多く含む第二の画像を取得する工程と、前記第一の画像よりも前記被写体の高周波成分を多く含み、前記第二の画像よりもノイズ量が少ない第三の画像を取得する工程と、前記第一の画像と前記第二の画像と前記第三の画像を用いてニューラルネットワークのパラメータを学習する工程とを有する。 According to another aspect of the present invention, there is provided a method for producing a trained model, comprising: acquiring a first image of a subject ; acquiring a second image containing many high-frequency components; and acquiring a third image containing more high-frequency components of the subject than the first image and having less noise than the second image. and learning parameters of a neural network using the first image, the second image, and the third image.

Claims

obtaining a first image of the subject ;
Acquiring a second image that has a larger amount of noise than the first image and contains more high-frequency components of the subject than the first image ;
inputting the first image and the second image into a neural network ;
using the neural network to generate a third image containing more high-frequency components of the subject than the first image and having less noise than the second image. Image processing method.

2. The image processing method according to claim 1, wherein said first image and said second image are at least parts of a first input image and a second input image, respectively.

3. The image processing method according to claim 2, wherein said first input image has a wider angle of view than said second input image.

4. The image processing method according to claim 3, wherein the angle of view of said third image is equal to or smaller than the angle of view of said second input image.

1. The second image is an image obtained by imaging using an imaging device having a pixel pitch smaller than that of the imaging device used to acquire the first image. 5. The image processing method according to any one of 4.

1. The second image is an image obtained by imaging using an imaging device having an imaging area smaller than that of the imaging device used to obtain the first image. 6. The image processing method according to any one of 5.

7. The image processing method according to claim 1, wherein said neural network has at least one convolutional layer.

further comprising acquiring learning information about the pre-trained neural network;
In the step of generating the third image, when N is an integer of 2 or more and n is an integer from 1 to N,
n-th linear transformation by each of a plurality of linear functions based on the learning information and n-th nonlinear transformation by a nonlinear function with respect to the first image and the second image, where n is from 1 to N a step of generating intermediate data by sequentially executing until
8. The image processing according to any one of claims 1 to 7, further comprising the step of executing an (N+1)th linear transformation on the intermediate data using at least one linear function based on the learning information. Method.

The learning information includes a plurality of learning images in which the same subject is present, and a first input learning image and a second input learning image having a larger amount of noise than the first input learning image and containing a large amount of high-frequency components of the subject. Information learned using two input learning images and an output learning image containing more high-frequency components of the subject than the first input learning image and having less noise than the second input learning image 9. The image processing method according to claim 8, wherein:

10. The image processing method according to claim 9, wherein at least one of said first input learning image and said second input learning image is an image generated by simulation.

obtaining a first partial image from the first image;
obtaining a second partial image from the second image;
a step of obtaining a third partial image by inputting the first partial image and the second partial image into the neural network;
11. The image processing method according to claim 1, wherein the third image is generated by synthesizing the third partial images.

a first image of a subject and an acquisition unit that acquires a second image that has a larger amount of noise than the first image and contains more high-frequency components of the subject than the first image ;
The first image and the second image are input to a neural network, and the neural network contains more high-frequency components of the subject than the first image and has a noise amount that is greater than that of the second image. and a calculator for generating a small number of third images.

13. The image processing apparatus according to claim 12 , further comprising a storage unit that stores learning information related to the neural network learned in advance.

The acquisition unit acquires a first partial image from the first image, acquires a second partial image from the second image,
The calculation unit obtains a third partial image by inputting the first partial image and the second partial image into the neural network, and synthesizes the third partial image to obtain the third partial image. 14. The image processing apparatus according to claim 12, wherein three images are generated.

An image processing device according to any one of claims 12 to 14;
a first imaging unit that captures the first image;
and a second imaging unit that captures the second image.

A program for causing a computer to execute the image processing method according to any one of claims 1 to 11 .

17. A storage medium storing the program according to claim 16 .

An image processing system having a first device and a second device,
The first device is
The second device performs image processing using a first image of a subject and a second image having a larger amount of noise than the first image and containing more high-frequency components of the subject than the first image. a sending means for sending a request to cause the
The second device is
receiving means for receiving the request transmitted by the transmitting means;
acquisition means for acquiring the first image and the second image;
The first image and the second image are input to a neural network, and the neural network contains more high-frequency components of the subject than the first image and has a noise amount that is greater than that of the second image. and a calculator for generating fewer third images.

the acquisition means acquires a first partial image from the first image and acquires a second partial image from the second image;
The calculation unit obtains a third partial image by inputting the first partial image and the second partial image into the neural network, and synthesizes the third partial image to obtain the third partial image. 19. The image processing system of claim 18, wherein three images are generated.

obtaining a first image of the subject ;
Acquiring a second image that has a larger amount of noise than the first image and contains more high-frequency components of the subject than the first image ;
acquiring a third image that contains more high-frequency components of the subject than the first image and has less noise than the second image;
and learning parameters of a neural network using the first image, the second image, and the third image.

obtaining a first partial image from the first image;
obtaining a second partial image from the second image;
a step of obtaining a third partial image by inputting the first partial image and the second partial image into the neural network;
21. The method of manufacturing a trained model according to claim 20, wherein said third image is generated by synthesizing said third partial images.