WO2023134417A1

WO2023134417A1 - Image processing circuit

Info

Publication number: WO2023134417A1
Application number: PCT/CN2022/140689
Authority: WO
Inventors: 森静香
Original assignee: 海信视像科技股份有限公司; 东芝视频解决方案株式会社
Priority date: 2022-01-13
Filing date: 2022-12-21
Publication date: 2023-07-20
Also published as: CN116802675A; JP2023103116A

Abstract

An image processing circuit (1000). The image processing circuit (1000) has: a first processing circuit, which executes noise reduction processing for reducing the noise of an image; a second processing circuit, which executes super-resolution processing for improving the resolution of the image; a calculation circuit, which calculates, on the basis of respective intensity settings of the noise reduction processing and the super-resolution processing, a synthesis ratio that represents the ratio of a plurality of images to be synthesized; a neural network circuit (1007), which executes the noise reduction processing or the super-resolution processing on the basis of the intensity setting; a first synthesis circuit (1008), which synthesizes, on the basis of the synthesis ratio, an image on which the noise reduction processing has been implemented by the first processing circuit and an image on which image processing has been implemented by the neural network circuit (1007); and a second synthesis circuit (1010), which synthesizes, on the basis of the synthesis ratio, an image on which the super-resolution processing has been implemented by the second processing circuit and the image on which the image processing has been implemented by the neural network circuit (1007).

Description

image processing circuit

Cross References to Related Applications

This application claims priority to Japanese Patent Application No. 2022-003991, titled "Image Processing Circuit," filed with the Japan Patent Office on January 13, 2022, the entire contents of which are incorporated herein by reference.

technical field

Embodiments of the present application relate to image processing circuits.

Background technique

There is a neural network circuit in which a neural network is circuitized. The neural network circuit, for example, performs noise reduction processing that reduces noise of an image. The neural network circuit reduces noise more effectively than conventional noise reduction processing circuits in which a noise reduction processing algorithm is circuitized.

Therefore, the use of the neural network circuit is not limited to noise reduction processing, but it is also intended to be used in various image processing such as super-resolution processing. Furthermore, when performing a plurality of image processing, it is preferable that the intensity of the image processing can be changed for each image processing.

prior art literature

patent documents

Patent Document 1: Japanese Patent Laid-Open No. 2020-191046

Contents of the invention

However, since the neural network circuit has a large circuit scale, it is difficult to install it for each image processing. In addition, since the neural network circuit has a large set of coefficients set for each neuron (neuron), it is difficult to prepare for intensity setting for each of a plurality of image processes.

The technical problem to be solved by the present application is to provide an image processing circuit capable of changing the intensity setting for each of a plurality of image processes while using a neural network circuit.

The image processing circuit of the embodiment includes: a first processing circuit that performs image processing that reduces image noise, that is, noise reduction processing; a second processing circuit that performs image processing that increases the resolution of an image, that is, super-resolution processing; and a calculation circuit. , which calculates a composite ratio representing a ratio of a plurality of images to be composited based on respective strength settings of the noise reduction processing and the super-resolution processing; a neural network circuit formed of a neural network, the neural network performing the noise reduction processing or the super-resolution processing based on the intensity setting; a first compositing circuit that composites an image on which the noise reduction processing is performed by the first processing circuit based on the compositing ratio and the image subjected to the image processing by the neural network circuit; a second synthesis circuit that synthesizes the image subjected to the super-resolution processing by the second processing circuit and the image processed by the second processing circuit based on the synthesis ratio A neural network circuit implements the image processing on the image.

Description of drawings

FIG. 1 is a diagram showing an example of a hardware configuration of a television device according to an embodiment;

2 is a block diagram showing an example of a circuit configuration of an image processing circuit;

3 is a diagram illustrating an example of a calculation method of a composite ratio by a composite ratio operation circuit;

FIG. 4 is a diagram illustrating an example of a calculation method of a composite ratio by a composite ratio operation circuit;

FIG. 5 is a diagram illustrating an example of a calculation method of a combination ratio by a combination ratio calculation circuit.

Explanation of reference signs

10...television device, 108...signal processing unit, 1000...image processing circuit, 1001...level output circuit, 1002...coefficient selection circuit, 1003...first coefficient group, 1004...second coefficient group, 1005...synthesis ratio calculation circuit, 1006...NR (noise reduction) circuit, 1007...neural network circuit, 1008...the first synthesis circuit, 1009...SR (Super Resolution) circuit, 1010...the second synthesis circuit, L1...the first arrow, L2...the second arrow, P1, P2...matching point, R1...first slash, R2...second slash, R3...third slash, R4...fourth slash, R5...fifth slash, R6...sixth slash, R7 ...the seventh slash.

Detailed ways

Hereinafter, embodiments will be described in detail with reference to the drawings.

FIG. 1 is a diagram showing an example of a hardware configuration of a television device 10 according to the embodiment. The television device 10 performs image processing on images input through broadcasting or the like. Then, the television device 10 displays the image on which the image processing has been performed.

As shown in FIG. 1 , television set 10 includes antenna 101, input terminals 102a to 102c, tuner 103, demodulator 104, demultiplexer 105, A/D (analog/digital) converter 106, selector 107, Signal processing unit 108, speaker 109, display panel 110, operation unit 111, light receiving unit 112, IP communication unit 113, CPU (Central Processing Unit: central processing unit) 114, memory 115, memory 116, microphone 117, and audio I/O F (interface) 118 .

The antenna 101 receives broadcast signals of digital broadcasts, and supplies the received broadcast signals to the tuner 103 via the input terminal 102a.

The tuner 103 selects a broadcast signal of a desired channel from the broadcast signals supplied from the antenna 101 , and supplies the selected broadcast signal to the demodulator 104 .

The demodulator 104 demodulates the broadcast signal supplied from the tuner 103 , and supplies the demodulated broadcast signal to the demultiplexer 105 .

The demultiplexer 105 separates the broadcast signal supplied from the demodulator 104 to generate a video signal and an audio signal, and supplies the generated video signal and audio signal to the selector 107 .

The selector 107 selects one of the signals supplied from the demultiplexer 105 , the A/D converter 106 , and the input terminal 102 c, and supplies the selected one signal to the signal processing unit 108 .

The signal processing unit 108 performs predetermined signal processing on the video signal supplied from the selector 107 , and supplies the processed video signal to the display panel 110 . Also, the signal processing unit 108 performs predetermined signal processing on the audio signal supplied from the selector 107 , and supplies the processed audio signal to the speaker 109 . The signal processing unit 108 has the image processing circuit 1000 shown in FIG. 2 .

The speaker 109 outputs voice or various sounds based on the voice signal supplied from the signal processing unit 108 . In addition, speaker 109 changes volumes of speech to be output or various sounds based on control by CPU 114 .

The display panel 110 as a display unit displays video such as still images and moving images, other images, text information, and the like based on video signals supplied from the signal processing unit 108 or controlled by the CPU 114 .

The input terminal 102b receives analog signals such as video signals and audio signals input from the outside. In addition, the input terminal 102c receives digital signals such as video signals and audio signals input from the outside. For example, the input terminal 102c can input a signal from a video recorder or the like equipped with a drive device that drives a recording medium for recording and playback such as BD (Blu-ray (registered trademark) Disc) to perform recording and playback.

The A/D converter 106 supplies a digital signal generated by performing A/D on the analog signal supplied from the input terminal 102 b to the selector 107 .

The operation unit 111 receives user's operation input.

The light receiving unit 112 receives infrared rays from the remote controller 119 .

The IP communication unit 113 is a communication interface for performing IP (Internet Protocol) communication via the network 40 .

CPU 114 controls the entire television set 10 .

The memory 115 is a ROM that stores various computer programs executed by the CPU 114 , a RAM that provides job partitions for the CPU 114 , and the like.

The memory 116 is HDD (Hard Disk Drive: Hard Disk Drive), SSD (Solid State Drive: Solid State Disk Drive), or the like. The memory 116 records, for example, the signal selected by the selector 107 as video data.

The microphone 117 serving as a voice input unit acquires the voice uttered by the user, and sends it to the audio I/F 118 .

Audio I/F 118 performs analog-to-digital conversion on audio received by microphone 117 and sends it to CPU 114 as an audio signal.

Next, the image processing circuit 1000 included in the signal processing unit 108 will be described.

FIG. 2 is a block diagram showing an example of a circuit configuration of the image processing circuit 1000 . As shown in FIG. 2 , the image processing circuit 1000 includes a level output circuit 1001, a coefficient selection circuit 1002, a synthesis ratio calculation circuit 1005, an NR (noise reduction: noise reduction) circuit 1006, a neural network circuit 1007, a first synthesis circuit 1008, an SR (Super Resolution: super resolution) circuit 1009, and the second synthesis circuit 1010.

The image processing circuit 1000 executes a plurality of kinds of image processing at each intensity level with respect to an input image input into the image processing circuit 1000 . The image processing circuit 1000 can set, for example, four levels of intensity. As an example of a plurality of image processing, the image processing circuit 1000 performs noise reduction processing and super-resolution processing. However, the image processing circuit 1000 may also perform image processing other than noise reduction processing or super-resolution processing.

The level output circuit 1001 is a circuit that outputs respective intensity levels of noise reduction processing and super-resolution processing. The strength level is a strength setting indicating the respective strengths of the noise reduction processing and the super-resolution processing. In addition, the intensity level may be a value determined by a user's setting, may be a value determined from various settings, may be a calculated value, or may be a value other than these. Furthermore, when the intensity level is input from the outside of the image processing circuit 1000 , the image processing circuit 1000 does not need to have the level output circuit 1001 .

The NR circuit 1006 is a circuit that performs image processing for reducing image noise, that is, noise reduction processing. The NR circuit 1006 is an example of a first processing circuit. More specifically, the NR circuit 1006 is a circuit in which an algorithm for noise reduction processing is circuitized. In addition, the NR circuit 1006 executes noise reduction processing corresponding to the intensity level of the noise reduction processing output from the level output circuit 1001 .

Specifically, the NR circuit 1006 executes noise reduction processing corresponding to the intensity level of the noise reduction processing on the input image input to the image processing circuit 1000 . Then, the NR circuit 1006 outputs an NR image, which is an image subjected to noise reduction processing, to the neural network circuit 1007 and the first combination circuit 1008 .

The SR circuit 1009 is a circuit that executes image processing for increasing the resolution of an image, that is, super-resolution processing. The SR circuit 1009 is an example of a second processing circuit. For example, the so-called super-resolution processing is processing for raising thin lines included in an image or emphasizing edges. More specifically, the SR circuit 1009 is a circuit in which a super-resolution processing algorithm is circuitized. In addition, the SR circuit 1009 executes super-resolution processing corresponding to the intensity level of the super-resolution processing output from the level output circuit 1001 .

Specifically, the SR circuit 1009 performs super-resolution processing corresponding to the intensity level of the super-resolution processing on the first synthesized image synthesized by the first synthesis circuit 1008. Then, the NR circuit 1006 outputs the SR image, which is the image subjected to super-resolution processing, to the second synthesis circuit 1010 .

The coefficient selection circuit 1002 selects the first coefficient group 1003 for causing the neural network circuit 1007 to perform noise reduction processing, or the second coefficient group 1004 for causing the neural network circuit 1007 to perform super-resolution processing, based on the intensity level. The coefficient selection circuit 1002 is an example of a selection circuit. The first coefficient group 1003 is a group of coefficients set for each neuron of the neural network circuit 1007 . For example, the first coefficient group 1003 is a group of coefficients for noise reduction processing with an intensity level of four. In addition, the first coefficient group 1003 is an example of first setting information. The second coefficient group 1004 is a group of coefficients set for each neuron of the neural network circuit 1007 . For example, the second coefficient group 1004 is a group of coefficients for super-resolution processing with an intensity level of four. In addition, the second coefficient group 1004 is an example of second setting information.

In more detail, the coefficient selection circuit 1002 compares the intensity level of the noise reduction process with the intensity level of the super-resolution process. When the strength level of the noise reduction processing is higher than that of the super-resolution processing, the coefficient selection circuit 1002 selects the first coefficient group 1003 which is a coefficient group for noise reduction processing. On the other hand, when the intensity level of the super-resolution process is higher than that of the noise reduction process, the coefficient selection circuit 1002 selects the second coefficient group 1004 which is a coefficient group for super-resolution processing. Then, the coefficient selection circuit 1002 outputs the selected first coefficient group 1003 or second coefficient group 1004 .

Also, in the case where the intensity level of the noise reduction process is the same as that of the super-resolution process, the coefficient selection circuit 1002 selects the first coefficient group 1003 that causes the neural network circuit 1007 to perform the noise reduction process with an intensity level of 4. It should be noted that the coefficient selection circuit 1002 may also select the second coefficient group 1004 for the neural network circuit 1007 to perform super-resolution processing, however, it is preferable to select the first coefficient group 1003 . Super-resolution processing raises fine lines contained in an image, or emphasizes edges. In addition, the neural network circuit 1007 has a stronger effect of image processing than the SR circuit 1009 or the NR circuit 1006 . Therefore, in the case where super-resolution processing acts more strongly than noise reduction processing, there is a possibility that noise is emphasized. Therefore, the coefficient selection circuit 1002 selects the first coefficient group 1003 in the case where the intensity level of the noise reduction process is the same as that of the super-resolution process.

Furthermore, the coefficient selection circuit 1002 shown in FIG. 2 stores the first coefficient group 1003 or the second coefficient group 1004 . However, the coefficient selection circuit 1002 may not store the first coefficient group 1003 or the second coefficient group 1004. For example, the coefficient selection circuit 1002 may also obtain the first coefficient group 1003 or the second coefficient group 1004 from a storage medium such as RAM, and output the obtained first coefficient group 1003 or the second coefficient group 1004 .

In addition, the coefficient selection circuit 1002 does not have 5 coefficient groups with intensity levels from 0 to 4, but has one coefficient group with intensity level 4. In this way, the coefficient selection circuit 1002 selects coefficient groups of some strength levels among the strength levels of a plurality of stages. Therefore, the storage medium does not need to store the coefficient groups of all combinations of intensity levels for each image processing. Here, the coefficient group is set for each neuron of the neural network circuit 1007, so the data capacity is large. Therefore, if it is desired to prepare coefficient groups for combinations of all intensity levels for each image processing, a very large-capacity storage medium is required. The coefficient selection circuit 1002 selects a coefficient group from the first coefficient group 1003 or the second coefficient group 1004, so that it does not need to have a very large-capacity storage medium.

The neural network circuit 1007 is a circuit formed of a neural network that executes noise reduction processing or super-resolution processing based on the strength levels of the noise reduction processing and super-resolution processing output from the level output circuit 1001 . The neural network circuit 1007 is an example of a neural network circuit. More specifically, the neural network circuit 1007 performs image processing of the first coefficient group 1003 or the second coefficient group 1004 selected by the coefficient selection circuit 1002 according to the strength level of the noise reduction processing and the super-resolution processing.

That is, when the first coefficient group 1003 is output from the coefficient selection circuit 1002 , the neural network circuit 1007 executes the noise reduction processing at the intensity level 4 indicated by the first coefficient group 1003 . In addition, when the second coefficient group 1004 is output from the coefficient selection circuit 1002 , the neural network circuit 1007 executes super-resolution processing with an intensity level of 4 indicated by the second coefficient group 1004 .

Also, the neural network circuit 1007 executes image processing according to the intensity level on the image subjected to the noise reduction processing by the NR circuit 1006 . That is, the neural network circuit 1007 executes noise reduction processing with an intensity level of 4, or super-resolution processing with an intensity level of 4. Then, the neural network circuit 1007 outputs the neural network image, which is an image subjected to image processing, to the first combining circuit 1008 and the second combining circuit 1010 .

The first combination circuit 1008 combines the NR image subjected to noise reduction processing by the NR circuit 1006 and the neural network image subjected to image processing by the neural network circuit 1007 based on the combination ratio calculated by the combination ratio calculation circuit 1005 . The first combining circuit 1008 is an example of a first combining circuit. In addition, the first combining circuit 1008 can use any method to combine images. For example, the first compositing circuit 1008 can composite images by performing multiplication with the composite ratio as a weight coefficient. Then, the first combining circuit 1008 outputs the first combined image generated by combining the NR image and the neural network image to the SR circuit 1009 .

The second combination circuit 1010 combines the SR image subjected to super-resolution processing by the SR circuit 1009 and the neural network image subjected to image processing by the neural network circuit 1007 based on the combination ratio calculated by the combination ratio calculation circuit 1005 . The second combining circuit 1010 is an example of a second combining circuit. In addition, the second compositing circuit 1010 may also use any method to composite images. For example, the second compositing circuit 1010 may composite the images by performing multiplication calculation using the composite ratio as a weight coefficient. Then, the second compositing circuit 1010 outputs an output image generated by compositing the SR image and the neural network image.

The synthesis ratio calculation circuit 1005 calculates a synthesis ratio indicating a ratio of a plurality of images to be synthesized based on respective intensity levels of the noise reduction processing and the super-resolution processing. The composite ratio calculation circuit 1005 is an example of a calculation circuit. More specifically, the synthesis ratio calculation circuit 1005 calculates the image processed by the NR circuit 1006 or the SR circuit 1009 based on the difference between the intensity level of the noise reduction process and the intensity level of the super-resolution process, and the image processed by the neural network circuit 1007. Synthesis ratio of neural network image synthesis with image processing implemented.

FIG. 3 is a diagram illustrating an example of a calculation method of a combination ratio performed by the combination ratio calculation circuit 1005 . The composite ratio calculation circuit 1005 calculates the composite ratio using the graph shown in FIG. 3 . The vertical axis of the graph represents the intensity level of the noise reduction processing. The horizontal axis of the graph represents the intensity level of the super-resolution processing. The first oblique line R1 , the second oblique line R2 , the third oblique line R3 , and the fourth oblique line R4 are oblique lines used to determine a combination ratio when the neural network circuit 1007 performs noise reduction processing. The fifth oblique line R5 , the sixth oblique line R6 , and the seventh oblique line R7 are oblique lines used to determine a combination ratio when the neural network circuit 1007 performs super-resolution processing.

In the graph shown in FIG. 3 , the combination ratio calculation circuit 1005 detects an orthogonal point at which the intensity level of the noise reduction process output from the level output circuit 1001 is orthogonal to the intensity level of the super-resolution process. When an orthogonal point exists on the first arrow L1 or on the second arrow L2, the combination ratio operation circuit 1005 plots the orthogonal point as matching points P1 and P2 (see FIGS. 4 and 5 ). . When there is no orthogonal point on the first arrow L1 or on the second arrow L2 , the combination ratio calculation circuit 1005 determines an oblique line passing through the orthogonal point. That is, the combination ratio operation circuit 1005 calculates the first oblique line R1, the second oblique line R2, the third oblique line R3, the fourth oblique line R4, the fifth oblique line R5, the sixth oblique line R6, and the seventh oblique line R7 Select a diagonal line through the orthogonal point in .

In addition, the combination ratio calculation circuit 1005 plots points where the first arrow L1 or the second arrow L2 cross the selected oblique line as matching points P1 and P2 (see FIG. 4 and FIG. 5 ). Then, the composite ratio calculation circuit 1005 determines the composite ratio based on the matching points P1 and P2 (see FIG. 4 and FIG. 5 ) in the first arrow L1 or the second arrow L2 .

Here, the calculation method of the combination ratio by the combination ratio calculation circuit 1005 is demonstrated using FIG. 4 and FIG. 5, giving a specific example. FIG. 4 is a diagram illustrating an example of a calculation method of a combination ratio performed by the combination ratio calculation circuit 1005 . FIG. 4 shows a state where the intensity level of the noise reduction process is 3 and the intensity level of the super-resolution process is 2.

The synthesis ratio calculation circuit 1005 detects an orthogonal point at which the intensity level of the noise reduction process is orthogonal to the intensity level of the super-resolution process. Since there is no detected orthogonal point on the first arrow L1 or the second arrow L2, the combined ratio operation circuit 1005 detects the third oblique line R3 passing through the orthogonal point. The combination ratio operation circuit 1005 detects a point where the first arrow L1 and the third oblique line R3 are perpendicular to each other as the matching point P1. Then, the composite ratio calculation circuit 1005 determines the composite ratio of the image based on the matching point P1. Specifically, the matching point P1 divides the first arrow L1 into 1:3. Therefore, the composite ratio calculation circuit 1005 determines the ratio of the NR image to be 75%, and the ratio of the neural network image to be 25%.

FIG. 5 is a diagram illustrating an example of a calculation method of a combination ratio by the combination ratio calculation circuit 1005 . FIG. 5 shows a state where the intensity level of noise reduction processing is 1 and the intensity level of super-resolution processing is 3. The synthesis ratio calculation circuit 1005 detects an orthogonal point at which the intensity level of the noise reduction process is orthogonal to the intensity level of the super-resolution process. Since there is an orthogonal point detected on the first arrow L1 or the second arrow L2, the combination ratio operation circuit 1005 detects the orthogonal point as the matching point P2. Then, the composite ratio calculation circuit 1005 determines the composite ratio of the image based on the matching point P2. Specifically, the matching point P2 divides the second arrow L2 into 1:1. Therefore, the combination ratio calculation circuit 1005 determines the ratio of the NR image to 50%, and the ratio of the neural network image to 50%.

As the matching points P1 and P2 (see FIG. 4 and FIG. 5 ) approach the fourth oblique line R4, the combination ratio calculation circuit 1005 reduces the ratio of the neural network image in the combination of images. In other words, as the difference between the intensity level of the noise reduction process and the intensity level of the super-resolution process becomes smaller, the combination ratio operation circuit 1005 reduces the difference between the NR image or the SR image subjected to the image processing by the NR circuit 1006 or the SR circuit 1009 The ratio of the neural network images subjected to image processing by the neural network circuit 1007 among the neural network images subjected to image processing by the neural network circuit 1007 .

In the graph shown in FIG. 3 , when the matching points P1 and P2 (refer to FIGS. 4 and 5 ) are plotted on the first arrow L1, as the matching points P1 and P2 (refer to FIGS. 4 and 5 ) Approaching the fourth oblique line R4 , the combination ratio operation circuit 1005 reduces the ratio of the neural network image subjected to the noise reduction processing by the neural network circuit 1007 . On the other hand, when the matching points P1 and P2 (see FIGS. 4 and 5 ) are plotted on the second arrow L2, as the matching points P1 and P2 (see FIGS. 4 and 5 ) approach the fourth oblique line R4, the combination ratio calculation circuit 1005 reduces the ratio of the neural network image subjected to super-resolution processing by the neural network circuit 1007 .

In this way, the combination ratio calculation circuit 1005 can maintain the continuity of the image processing effect when the intensity level is changed by reducing the proportion of the neural network image according to the difference between the intensity level of the noise reduction process and the super-resolution process. Here, the neural network circuit 1007 is more effective in image processing than the NR circuit 1006 and the SR circuit 1009 . Therefore, in the case where the content of the image processing performed by the neural network circuit 1007 is switched, the effect of the image processing changes.

For example, in the case where the neural network circuit 1007 has performed noise reduction processing, the first combining circuit 1008 generates a first combined image by combining the NR image and the neural network image. In a case where the image processing performed by the neural network circuit 1007 is switched from noise reduction processing to super-resolution processing, the first synthesis circuit 1008 generates a first synthesis image from the NR image. In this case, since the neural network image is not used for image synthesis, if the proportion of the neural network image in the first combined image is high, the effect of the noise reduction process will vary greatly. On the other hand, if the proportion of the neural network image in the first synthesized image is low, the effect of the noise reduction process will change little.

In addition, when the difference in the setting level of the image processing is small, the neural network circuit 1007 switches the content of the image processing to be executed with a slight change of the setting level. Therefore, the combination ratio calculation circuit 1005 can maintain the continuity of the effect of the image processing by reducing the ratio of the neural network image as the difference in the set level of the image processing becomes smaller.

In addition, the combination ratio calculation circuit 1005 is not limited to the graph shown in FIG. 3 and may calculate the combination ratio. For example, the combination ratio calculation circuit 1005 calculates the difference between the intensity level of the noise reduction process and the intensity level of the super-resolution process. Then, the composition ratio operation circuit 1005 calculates the composition ratio of the image based on the difference.

For example, when the difference is 0, the combination ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 0%, and the ratio of the image generated by the NR circuit 1006 or SR circuit 1009 to 100%. Also, when the difference is 1, the combination ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 25%, and the ratio of the image generated by the NR circuit 1006 or SR circuit 1009 to 75%. Also, when the difference is 2, the combination ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 50%, and the ratio of the image generated by the NR circuit 1006 or SR circuit 1009 to 50%. Also, when the difference is 3, the combination ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 75%, and the ratio of the image generated by the NR circuit 1006 or SR circuit 1009 to 25%. Also, when the difference is 4, the combination ratio calculation circuit 1005 sets the ratio of the image generated by the neural network circuit 1007 to 100%, and the ratio of the image generated by the NR circuit 1006 or SR circuit 1009 to 0%. In addition, the ratio of each image can also be changed arbitrarily.

As described above, the image processing circuit 1000 of the embodiment includes: the NR circuit 1006 that performs noise reduction processing; the SR circuit 1009 that performs super-resolution processing; and the neural network circuit that performs noise reduction processing or super-resolution processing based on intensity levels 1007. In addition, the synthesis ratio calculation circuit 1005 calculates a synthesis ratio indicating a ratio of a plurality of images to be synthesized based on respective strength levels of the noise reduction processing and the super-resolution processing. The first synthesis circuit 1008 synthesizes the NR image subjected to the noise reduction processing by the NR circuit 1006 and the neural network image subjected to the image processing by the neural network circuit 1007 based on the synthesis ratio. The second synthesis circuit 1010 synthesizes the SR image subjected to super-resolution processing by the SR circuit 1009 and the neural network image subjected to image processing by the neural network circuit 1007 based on the synthesis ratio. Then, the image processing circuit 1000 outputs the neural network image synthesized by the second synthesis circuit 1010 .

In this way, the image processing circuit 1000 causes the neural network circuit 1007 to perform noise reduction processing or super-resolution processing according to the intensity level. In addition, the image processing circuit 1000 calculates a combination ratio based on the intensity level. Then, the image processing circuit 1000 synthesizes the NR image and the neural network image according to the combining ratio, and synthesizes the SR image and the neural network image according to the combining ratio. Therefore, the image processing circuit 1000 can use the neural network circuit 1007 and can change the respective intensity levels of a plurality of image processing.

In addition, in the above-mentioned embodiment, it has been described that the signal processing unit 108 has the image processing circuit 1000 . However, components other than the signal processing unit 108 may include the image processing circuit 1000 .

In addition, in the above-described embodiments, it has been described that the television device 10 includes the image processing circuit 1000 . However, devices other than the television device 10 may have the image processing circuit 1000 . For example, a personal computer, a smartphone, a tablet terminal, a video recorder for recording images, and a display device may include the image processing circuit 1000 .

Although the embodiment of the present application has been described, the embodiment is shown as an example and is not intended to limit the scope of the application. This new embodiment can be implemented in other various forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the application, and are included in the technical solutions described in the claims and their equivalents.

Claims

An image processing circuit, comprising:

a first processing circuit that performs noise reduction processing that is image processing that reduces noise of an image;

a second processing circuit that performs super-resolution processing, which is image processing that increases the resolution of an image;

a calculation circuit that calculates a composite ratio representing a ratio of a plurality of images to be composited based on respective intensity settings of the noise reduction processing and the super-resolution processing;

a neural network circuit formed of a neural network that performs said noise reduction processing or said super-resolution processing based on said intensity setting;

a first combining circuit that combines the image subjected to the noise reduction processing by the first processing circuit and the image subjected to the image processing by the neural network circuit based on the combining ratio; and

A second combining circuit that combines the image subjected to the super-resolution processing by the second processing circuit and the image subjected to the image processing by the neural network circuit based on the combining ratio.
The image processing circuit according to claim 1, wherein,

The image processing circuit further includes a selection circuit for selecting first setting information or second setting information based on the strength setting, the first setting information being used to cause the neural network circuit to execute In the noise reduction processing, the second setting information is used to make the neural network circuit perform the super-resolution processing,

The neural network circuit performs the image processing of the first setting information or the second setting information selected by the selection circuit.
The image processing circuit according to claim 2, wherein,

The selection circuit selects the first setting information or the second setting information of a part of the intensity settings among the intensity settings of a plurality of stages.
The image processing circuit according to any one of claims 1 to 3, wherein,

The calculation circuit calculates the combination ratio for combining two images based on a difference between the intensity setting of the noise reduction process and the intensity setting of the super-resolution process, one of which is An image subjected to the image processing by the first processing circuit or the second processing circuit, or an image subjected to the image processing by the neural network circuit.
The image processing circuit according to claim 4, wherein,

The calculation circuit reduces the amount of noise generated by the first processing circuit or the second processing as the difference between the intensity setting of the noise reduction processing and the intensity setting of the super-resolution processing becomes smaller. The proportion of the image subjected to the image processing by the neural network circuit among the image subjected to the image processing by the circuit and the image subjected to the image processing by the neural network circuit.
The image processing circuit according to any one of claims 1 to 5, wherein,

the first processing circuit performs the noise reduction processing on the inputted input image,

the neural network circuit executes the image processing corresponding to the intensity setting on the image subjected to the noise reduction processing by the first processing circuit,

the first synthesis circuit synthesizes the image subjected to the noise reduction processing by the first processing circuit and the image subjected to the image processing by the neural network circuit,

the second processing circuit performs the super-resolution processing on the image synthesized by the first synthesis circuit,

The second synthesis circuit synthesizes the image subjected to the super-resolution processing by the second processing circuit and the image subjected to the image processing by the neural network circuit.