WO2022156248A1 - 影像处理设备、影像处理设备的工作方法以及非易失性存储介质 - Google Patents

影像处理设备、影像处理设备的工作方法以及非易失性存储介质 Download PDF

Info

Publication number
WO2022156248A1
WO2022156248A1 PCT/CN2021/118764 CN2021118764W WO2022156248A1 WO 2022156248 A1 WO2022156248 A1 WO 2022156248A1 CN 2021118764 W CN2021118764 W CN 2021118764W WO 2022156248 A1 WO2022156248 A1 WO 2022156248A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
determination unit
video
scene
processing
Prior art date
Application number
PCT/CN2021/118764
Other languages
English (en)
French (fr)
Inventor
柴田诚
Original Assignee
海信视像科技股份有限公司
东芝视频解决方案株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司, 东芝视频解决方案株式会社 filed Critical 海信视像科技股份有限公司
Priority to CN202180027166.0A priority Critical patent/CN115398880A/zh
Publication of WO2022156248A1 publication Critical patent/WO2022156248A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/44—Receiver circuitry for the reception of television signals according to analogue transmission standards
    • H04N5/57—Control of contrast or brightness

Definitions

  • the present disclosure relates to an image processing apparatus, a working method of the image processing apparatus, and a nonvolatile storage medium.
  • a video display device such as a television receiver, a smartphone, or the like is developed that has a function of automatically adjusting the video of a program to the picture quality and sound quality according to the genre (Genre).
  • the genre of the program is acquired from raw data such as the EPG program table.
  • one program includes a plurality of scenes (scenes) with different optimum image quality and the like.
  • a program whose genre is news includes a character scene, an indoor scene, a landscape scene, a sports scene, and the like.
  • AI Artificial Intelligence
  • peripheral equipment such as a television receiver
  • resources are small, so it is not easy to perform appropriate video processing using AI computation.
  • Patent Document 1 Japanese Patent Laid-Open No. 2008-28871
  • Patent Document 2 International Publication No. 2018/067962
  • An object of the present disclosure is to provide a video processing device that outputs the most appropriate video corresponding to a scene, an operation method of the video processing device that outputs the most appropriate video corresponding to the scene, and a video that stores the video outputting the most appropriate video corresponding to the scene A non-volatile storage medium for processing programs.
  • the image processing apparatus of the present disclosure includes: a first judgment unit for judging a change level of the image through a first AI calculation; a comparison unit for comparing the change level with a predetermined value; and a second judgment unit for only the change level at the change level
  • the second AI calculation is used to determine which scene of the plurality of scenes the video is classified into; the setting unit sets the image quality parameter based on the determined scene; and the adjustment unit uses the selected scene. Adjust the image by describing the quality parameters.
  • the working method of the image processing apparatus of the present disclosure includes: judging the change level of the image through the first AI operation; comparing the change level with a predetermined value; Two AI operations are performed to determine which scene of a plurality of scenes the image is classified into; to set an image quality parameter based on the determined scene; and to adjust the image using the image quality parameter.
  • the non-volatile storage medium of the present disclosure stores an image processing program, the image processing program causing the computer to execute the following steps: judging the change level of the image through the first AI operation; comparing the change level with a predetermined value; only When the change level exceeds the predetermined value, the second AI operation is used to determine which scene of a plurality of scenes the video is classified into; to set an image quality parameter based on the determined scene; and to use the image quality parameters to adjust the image.
  • FIG. 1 is a configuration diagram of a television receiver including an image processing apparatus according to an embodiment
  • FIG. 2 is a flowchart of an operation method of the image processing apparatus according to the first embodiment
  • FIG. 3 is a diagram for explaining an operation method of the image processing apparatus according to the first embodiment
  • FIG. 4 is a flowchart of an operation method of the image processing apparatus of the second embodiment
  • FIG. 5 is a diagram for explaining an operation method of the video processing apparatus according to the second embodiment.
  • the video processing device 1 of the present embodiment, the tuner 31 and the memory 32 constitute a reception device 30
  • the reception device 30 , the display 42 and the speaker 43 constitute a reception system 9
  • the receiver 30 may also be a television receiver integrated with the display 42 and the speaker 43 .
  • the display 42 is a liquid crystal, EL (Electroluminescence), plasma display, SED (Surface Electric Field Display), video projector, rear projection (rear projection type), or a picture tube (including flat type) or the like.
  • the terminal through which the user operates the receiving device 30, that is, the remote controller 44, may be a smartphone, a tablet terminal, an AI speaker, or the like.
  • the tuner 31 receives, for example, by selecting one channel from among a plurality of channels of terrestrial digital television broadcasting and satellite digital television broadcasting received by the receiving antenna 41 .
  • the tuner 31 can also receive Internet broadcasts input from the server 47 via the network line 46 .
  • Program images recorded in the video recorder 45 may also be input to the receiving device 30 .
  • the video processing device 1 processes the input video, and outputs an image signal and a sound signal.
  • the image signal is output to the display 42, and the sound signal is output to the speaker 43, so that the user watches the program.
  • the video processing apparatus 1 includes a CPU 10 as a processor and an AI computing unit 20 as a neural network.
  • the AI calculation unit 20 includes a first determination unit 21 and a second determination unit 22 .
  • the first determination unit 21 and the second determination unit 22 share the resources of the AI computation unit 20, and therefore cannot perform computation processing at the same time.
  • the AI computing unit 20 includes a semiconductor, and, for example, reads and operates a program stored in the memory 32 .
  • the first determination unit 21 performs a first AI calculation (AI calculation 1) for determining the change level D of the image of the video using a neural network.
  • the second determination unit 22 performs a second AI calculation (AI calculation 2 ) for determining which of a plurality of scenes the video is classified into using a neural network.
  • AI operations based on neural networks use deep learning based on deep learning algorithms to perform image analysis processing.
  • the deep learning algorithm is an algorithm including a well-known convolutional neural network (CNN: Convolutional Neural Network) method, a fully connected layer, and an output layer. Deep learning is called deep learning. Since image analysis processing based on AI computation using deep learning is a well-known technique, a detailed description thereof will be omitted.
  • CNN Convolutional Neural Network
  • the CPU 10 performs overall control of the reception device 30 .
  • the CPU 10 includes a semiconductor and, for example, reads and operates a program stored in the memory 32 .
  • the CPU 10 includes a comparison unit 11 , a setting unit 12 , and an adjustment unit 13 .
  • at least one of these functional units executed by the CPU 10 may be configured as a dedicated circuit independent of the CPU 10 .
  • one CPU unit may include the CPU 10 and the AI computing unit 20 . However, for high-speed processing, it is preferable that the AI computation is performed in an AI-dedicated processor.
  • the comparison unit 11 compares the change level D of the video image determined by the first determination unit 21 with a predetermined value K.
  • the second determination unit 22 performs a scene determination operation, that is, a second AI operation, only when the change level D determined by the first determination unit 21 by the first AI operation exceeds the predetermined value K.
  • the second AI calculation is performed.
  • the setting unit 12 sets the image quality parameter based on the scene determined by the second determination unit 22 .
  • the adjustment unit 13 adjusts the video using the image quality parameter.
  • the second AI operation is performed regardless of the output of the first AI operation.
  • the second AI operation may not be performed based on the output of the first AI operation. Therefore, even the video processing device, which is a peripheral device with small resources, outputs the most appropriate video according to the scene.
  • the video of the television broadcast has, for example, 30 frame images (still images) per second.
  • the frame image (first image) and the next frame image (second image) are input to the first determination unit 21 .
  • the first determination unit 21 performs the first AI computation in which the AI computation unit 20 determines the change level D between the first image and the second image. For example, in the first AI operation, extraction of a two-dimensional feature map or extraction of a one-dimensional feature vector is performed.
  • the comparison unit 11 compares the change level D determined by the first determination unit 21 with the predetermined value K. When the change level D is larger than the predetermined value K (Yes), the process of step S40 is performed. When the change level D is equal to or smaller than the predetermined value K (NO), the process of step S10 is performed.
  • the predetermined value K is set to an appropriate value, for example, exceeding 70%.
  • the predetermined value K may also be changeable by a user's operation.
  • the second judgment unit 22 performs a second AI calculation for judging which scene of the plurality of scenes the second image is in the AI calculation unit 20 shared with the first judgment unit 21 .
  • the scene is, for example, a character scene, a landscape scene, a night scene scene, and a sports scene.
  • object detection or segmentation using a two-dimensional feature map as an input, or image classification processing using a one-dimensional feature vector as an input is performed.
  • Step S50 Time measurement (elapse of TA)
  • the processing interval (time) TA of the first determination unit 21 repeated that is, the interval TA of the first AI calculation is longer than the first processing time T1 of the first AI calculation.
  • the interval TA is shorter than the total time of the first processing time T1 of the first AI calculation and the second processing time T2 (T2A+T2B) of the second AI calculation. Therefore, during the interval TA, the second AI calculation is not completed.
  • the video processing apparatus 1 When the processing interval TA is reached (Yes), the video processing apparatus 1 temporarily suspends the second AI calculation, and performs the processing from step S60.
  • step S10 two new frame images are input to the first determination unit 21 .
  • the first determination unit 21 performs the first AI calculation for determining the change level D in the same manner as in step S20.
  • the comparison unit 11 compares the change level D determined by the first determination unit 21 with the predetermined value K.
  • step S80 when the change level D is larger than the predetermined value K (Yes), a new second AI calculation is performed in step S40. The second operation in the middle of the process is forcibly terminated. In addition, the halfway result that has already been processed may be used instead as the second AI calculation result.
  • the change level D is equal to or less than the predetermined value K (NO)
  • the second AI calculation A which is performed halfway is restarted.
  • the second AI calculation by the second determination unit 22 is divided and performed while the first AI calculation by the first determination unit 21 is not performed.
  • the second AI operation is divided into two and performed by the second AI operations 2A and 2B, but the second AI operation may be divided into three or more.
  • the processing interval TA of the first AI calculation is longer than the frame interval Tf (for example, 1/30 second).
  • the first AI operation may also be performed on all frame images.
  • the third AI operation may be performed subsequent to the second AI operation. For example, after it is determined in the second AI calculation that the video scene is "sports", the specific game name "soccer" may be determined in the third AI calculation.
  • step S10 When the second AI calculation is completed (Yes), a series of processing from step S10 is performed again, and simultaneously the processing of step S100 is performed. Continue until the end of the second AI operation (NO).
  • the setting unit 12 sets the image quality parameter.
  • the adjustment unit 13 uses the image quality parameter to adjust the video, that is, the frame image after the frame image in which there is a change.
  • the image quality parameters are, for example, brightness, color shade, shadow, color temperature, sharpness, noise reduction level, contrast enhancement level, and detail enhancement level.
  • a clear video can be obtained.
  • the texture of the skin becomes natural by increasing the noise reduction level and detail enhancement level and reducing the color shading level.
  • the image quality parameters based on each scene are stored in the memory 32 in advance, for example.
  • the video processing device 1 is a peripheral device with few resources, but can also output the most appropriate video according to the scene.
  • the operating method of the video processing apparatus includes: step S20 of determining the change level of the video by the first AI calculation; step S30 of comparing the change level with a predetermined value; and only when the change level exceeds In the case of the predetermined value, the second AI calculation is used to determine which scene of a plurality of scenes the video is classified into; the step S100 of setting an image quality parameter based on the determined scene; and using the image quality parameter to adjust the image in step S100.
  • the video processing program causes the computer to execute the following steps: step S20 of judging the change level of the video by the first AI calculation, step S30 of comparing the change level with a predetermined value; and only when the change level exceeds the predetermined value In this case, the second AI operation is used to determine which scene of the plurality of scenes the video is classified into; the step S100 of setting an image quality parameter based on the judged scene; and the step of adjusting the video using the image quality parameter S100.
  • the video processing apparatus 1A of the present modification is similar to the video processing apparatus 1 , and therefore the same reference numerals are given to the constituent elements with the same functions, and the description thereof is omitted.
  • the video processing apparatus 1A acquires, for example, program data (eg, EPG: Electronic Programming Guide) added to a video signal of a television program.
  • EPG data is stored in the memory 32 .
  • the program data includes genre data in addition to the program name, performers, program outline, and the like. Types are, for example, "News/Report”, “Sports”, “Information/Slide Show”, “Drama”, “Music”, “Variety”, “Movie”, “Motion Picture/Special Shot”, “Documentary/Culture” ", “Theatre/Performance”, “Fun/educationion”, “Welfare”.
  • the setting unit 12 of the video processing apparatus 1A sets the image quality parameter based on the type and the scene determined by the second determination unit 22 .
  • the genre is a news video
  • the ratio of increasing each level of brightness level, color gradation, and shading is set to be smaller. image quality parameters. Therefore, in a video whose genre is news, for example, even if it is switched from a human scene to a landscape scene video, it does not greatly change. Conversely, in a video whose genre is movie, a more shocking image of a landscape scene than a video whose genre is news is output.
  • Image quality parameters based on each of a plurality of types of scenes are stored, for example, in the memory 32 in advance.
  • the video processing apparatus 1A can more appropriately adjust the video of the scene according to the type.
  • the video processing apparatus 1B of the present modification is similar to the video processing apparatus 1, the same reference numerals are assigned to the constituent elements with the same functions, and the description thereof will be omitted.
  • the setting unit 12 of the video processing device 1B sets not only the image quality parameter but also the sound quality parameter based on the scene.
  • the adjustment unit 13 not only adjusts the image quality of the video, but also adjusts the sound of the video using the sound quality parameter.
  • the sound quality parameter is, for example, an equalizer level and a noise reduction level based on a high-pass filter and a low-pass filter.
  • the equalizer level is set to be flat and the noise reduction level is set to be larger for easier hearing.
  • the video not only adjusts the image, but also appropriately adjusts the sound according to the scene.
  • the image quality parameter may be set based on the type and the scene determined by the second determination unit 22 .
  • the video processing apparatus 1C of the present embodiment is similar to the video processing apparatus 1 and the like, the same reference numerals are given to the components having the same functions, and the description thereof will be omitted.
  • the second AI operation starts, and processing is performed until the end. After the second AI calculation is completed, a series of processing from step S10 is performed again, and the processing of step S100 is simultaneously performed.
  • the first processing interval (time) TA of the first determination unit 21, which is repeatedly performed is longer than the first processing time T1 of the first AI calculation and the second AI calculation.
  • the total time of the second processing time T2 (T2A+T2B) is short.
  • the first determination section 21 of the image processing apparatus 1C does not restart the processing until the processing of the second determination section 22 ends. Therefore, the second processing interval TA2 of the first determination unit 21 when the second AI calculation is performed is longer than the first processing interval TA1 when the second AI calculation is not performed.
  • the video processing apparatus 1C may increase the processing interval of the first determination unit 21, it is feared that the video of the scene cannot be properly adjusted in a video in which the scene changes drastically.
  • the first determination unit 21 detects a scene change of the video, it is possible to output a video of a more appropriate image quality faster than the video processing device 1 .
  • the video processing device 1c may perform image quality adjustment based on the type and the scene determined by the second determination unit 22 as in the video processing device 1A, or perform sound quality adjustment based on the scene as in the video processing device 1B.
  • the present disclosure also provides a computer-readable non-volatile storage medium, where the storage medium stores computer instructions, and when the computer instructions are executed by a processor, the image quality processing in the foregoing embodiments is implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Databases & Information Systems (AREA)
  • Television Receiver Circuits (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Television Signal Processing For Recording (AREA)
  • Image Processing (AREA)

Abstract

提供一种影像处理设备、影像处理设备的工作方法以及非易失性存储介质。影像处理设备具备：第一判断部(21)，通过第一AI运算来判断影像的变化等级；比较部(11)，将变化等级与规定值进行比较；第二判断部(22)，仅在变化等级超过规定值的情况下，通过第二AI运算来判断影像被分类为多个场景的哪个场景；设定部(12)，基于被判断出的场景来设定画质参数；和调整部(13)，使用画质参数来调整影像。

Description

影像处理设备、影像处理设备的工作方法以及非易失性存储介质
本申请要求在2021年01月20日提交日本专利局、申请号为2021-006896、发明名称为“影像处理设备、影像处理设备的工作方法以及影像处理程序”的日本专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及影像处理设备、影像处理设备的工作方法以及非易失性存储介质。
背景技术
开发一种具有将节目的影像自动地调整为与种类(Genre)相应的画质、音质的功能的电视接收机、智能手机等的影像显示设备。节目的种类从EPG节目表等的原始数据获取。
但是，一个节目中包含最适当的画质等不同的多个场景(场面)。例如，在种类是新闻的节目中，包含人物场景、屋内场景、风景场景以及运动场景等。
因此，在仅基于节目的种类的调整中，不能提供最适合于各个场景的影像。此外，也存在不能参照EPG节目表等的原始数据的影像。
近年来，影像处理中使用AI(人工智能)运算。AI运算的运算量较多，因此需要较大的资源(计算资源)。正在进行高效地进行AI运算的方法的开发。
但是，在电视接收机等的周边设备中，资源较小，因此不容易使用AI运算来进行适当的影像处理。
在先技术文献
专利文献
专利文献1：日本特开2008-28871号公报
专利文献2：国际公开第2018/067962号
发明内容
本公开的目的在于，提供一种输出与场景相应的最适当影像的影像处理设备、输出与场景相应的最适当影像的影像处理设备的工作方法以及存储有输出与场景相应的最适当影像的影像处理程序的非易失性存储介质。
本公开的影像处理设备具备：第一判断部，通过第一AI运算来判断影像的变化等级；比较部，将所述变化等级与规定值进行比较；第二判断部，仅在所述变化等级超过所述规定值的情况下，通过第二AI运算来判断影像被分类为多个场景的哪个场景；设定部，基于被判断出的场景来设定画质参数；和调整部，使用所述画质参数来调整影像。
本公开的影像处理设备的工作方法包括：通过第一AI运算来判断影像的变化等级；将所述变化等级与规定值进行比较；仅在所述变化等级超过所述规定值的情况下通过第二AI运算来判断影像被分类为多个场景的哪个场景；基于被判断出的场景来设定画质参数；和使用所述画质参数来调整影像。
本公开的非易失性存储介质储存有影像处理程序，所述影像处理程序使计算机执行以下步骤：通过第一AI运算来判断影像的变化等级；将所述变化等级与规定值进行比较；仅在所述变化等级超过所述规定值的情况下通过第二AI运算来判断影像被分类为多个场景的哪个场景；基于被判断出的场景来设定画质参数；和使用所述画质参数来调整影像。
附图说明
图1是包含实施方式的影像处理设备的电视接收机的结构图；
图2是第一实施方式的影像处理设备的工作方法的流程图；
图3是用于对第一实施方式的影像处理设备的工作方法进行说明的图；
图4是第二实施方式的影像处理设备的工作方法的流程图；
图5是用于对第二实施方式的影像处理设备的工作方法进行说明的图。
附图标记说明
1、1A-1C…影像处理设备
9…接收系统
11…比较部
12…设定部
13…调整部
20…神经网络
21…第一判断部
22…第二判断部
30…接收设备
31…调谐器
32…存储器
41…接收天线
42…显示器
43…扬声器
44…遥控器
45…录像机
46…网络线路
47…服务器。
具体实施方式
第一实施方式
如图1所示，本实施方式的影像处理设备1与调谐器31以及存储器32构成接收设备30，接收设备30与显示器42以及扬声器43构成接收系统9。接收设备30也可以是与显示器42以及扬声器43一体的电视接收设备。
显示器42是液晶、EL(电致发光)、等离子体显示器、SED(表面电场显示器)、视频投影仪、背投(背面投影型)、或者显像管(包含平面型) 等。用户操作接收设备30的终端即遥控器44可以是智能手机、平板终端、AI音箱等。
调谐器31例如通过从由接收天线41接收的地面数字电视广播以及卫星数字电视广播的多个频道之中选择一个频道而接收。调谐器31也可以经由网络线路46来接收从服务器47输入的因特网广播。录像机45中记录的节目影像也可以被输入到接收设备30。
影像处理设备1处理被输入的影像，输出图像信号和声音信号。图像信号被输出到显示器42，声音信号被输出到扬声器43，从而用户收看节目。
影像处理设备1具有：作为处理器的CPU10、作为神经网络的AI运算部20。
AI运算部20具有第一判断部21和第二判断部22。第一判断部21与第二判断部22共享AI运算部20的资源，因此不能同时进行运算处理。AI运算部20包含半导体，例如，读取存储器32中存储的程序并进行动作。
如后述那样，第一判断部21进行使用神经网络对影像的图像的变化等级D进行判断的第一AI运算(AI运算1)。第二判断部22进行使用神经网络对影像被分类为多个场景的哪个场景进行判断的第二AI运算(AI运算2)。
基于神经网络的AI运算使用基于深层学习算法的深层学习，执行影像的解析处理。深层学习算法是包含公知的卷积神经网络(CNN：Convolutional Neural Network)的方法、全连接层、输出层的算法。深层学习被称为深度学习。由于基于使用了深层学习的AI运算的图像解析处理是公知技术，因此省略具体的说明。
CPU10进行接收设备30的整体的控制。CPU10包含半导体，例如，读取存储器32中存储的程序并进行动作。CPU10包含比较部11、设定部12、调整部13。另外，CPU10执行的这些功能部的至少任一者也可以构成为与CPU10独立的专用电路。此外，也可以一个CPU单元具有CPU10和AI运算部20。但是，为了高速处理，优选AI运算在AI专用处理器中进行。
比较部11将第一判断部21判断出的影像的变化等级D与规定值K进行 比较。第二判断部22仅在第一判断部21通过第一AI运算而判断出的变化等级D超过规定值K的情况下，进行场景的判断运算即第二AI运算。
例如，在规定值K为75％、影像变化的可能性即变化等级D为80％的情况下，由于变化等级D超过规定值K，因此进行第二AI运算。设定部12基于第二判断部22判断出的场景，设定画质参数。调整部13使用画质参数来调整影像。
在规定值K为75％、变化等级D为60％的情况下，由于变化等级D为规定值K以下，因此不进行第二AI运算。
在现有的AI运算中，是多个运算必须连续进行的流水线方式。即，与第一AI运算的输出无关地都进行第二AI运算。与此相对地，在影像处理设备1中，可能根据第一AI运算的输出，而不进行第二AI运算。因此，即使是资源较小的周边设备即影像处理设备，也输出与场景相应的最适当的影像。
影像处理设备的工作方法
按照图2的流程图，对影像处理设备1的工作方法进行说明。
＜步骤S10＞帧图像输入
如图3的上段所示，电视广播的影像例如在1秒期间具有30张帧图像(静止图像)。向第一判断部21输入帧图像(第一图像)和其下个帧图像(第二图像)。
＜步骤S20＞第一AI运算
第一判断部21进行在AI运算部20判断第一图像与第二图像的变化等级D的第一AI运算。例如，在第一AI运算中，进行二维的特征图的提取或者一维的特征向量的提取。
若基于影像的明亮度变化、每个像素的亮度的变化等来判断场景，则担心影像稍微放大或者相机转动的情况下，错误地判断为场景变化。但是，通过使用AI运算，能够准确地判断场景的变化。
＜步骤S30＞变化等级比较
比较部11对第一判断部21判断出的变化等级D与规定值K进行比较。 在变化等级D大于规定值K的情况(是)下，进行步骤S40的处理。在变化等级D为规定值K以下的情况(否)下，进行步骤S10的处理。
另外，若规定值K过小，则担心频繁地进行画质调整并成为不自然的影像。因此，规定值K被设定为适当的值、例如超过70％。规定值K也可以能够通过用户的操作来变更。
＜步骤S40＞第二AI运算开始
第二判断部22进行在与第一判断部21共享的AI运算部20中判断第二图像是多个场景的哪个场景的第二AI运算。
场景例如是人物场景、风景场景、夜景场景、运动场景。
例如，在第二AI运算中，进行将二维的特征图作为输入的物体检测或分割、或者将一维的特征向量作为输入的图像分类处理。
＜步骤S50＞时间测量(经过TA)
如图3所示，在影像处理设备1中，反复进行的第一判断部21的处理间隔(时间)TA、即第一AI运算的间隔TA比第一AI运算的第一处理时间T1长。但是，间隔TA比第一AI运算的第一处理时间T1与第二AI运算的第二处理时间T2(T2A+T2B)的合计时间短。因此，在间隔TA的期间，第二AI运算未结束。
若成为处理间隔TA(是)，则影像处理设备1暂时中断第二AI运算，进行从步骤S60开始的处理。
＜步骤S60＞帧图像输入
与步骤S10同样地，向第一判断部21输入新的两张帧图像。
＜步骤S70＞第一AI运算
与步骤S20同样地，第一判断部21进行判断变化等级D的第一AI运算。
＜步骤S80＞变化等级比较
与步骤S30同样地，比较部11对第一判断部21判断出的变化等级D与规定值K进行比较。在步骤S80中，在变化等级D大于规定值K的情况(是)下，在步骤S40中进行新的第二AI运算。处理到中途的第二运算被强制结束。 另外，也可以将已经处理结束的中途结果代替利用为第二AI运算结果。与此相对地，在变化等级D为规定值K以下的情况(否)下，重新开始进行到中途的第二AI运算A。
即，基于第二判断部22的第二AI运算在未进行基于第一判断部21的第一AI运算的期间被分割进行。在影像处理设备1中，第二AI运算被二分割为第二AI运算2A、2B而进行，但第二AI运算也可以被分割成三份以上。
另外，在影像处理设备1中，第一AI运算的处理间隔TA比帧间隔Tf(例如，1/30秒)长。但是，在AI运算速度较快的情况下，也可以对所有帧图像进行第一AI运算。
此外，也可以接着第二AI运算进行第3AI运算。例如，也可以在第二AI运算中判断为影像场景是“运动”之后，在第3AI运算中判断具体的比赛名称“足球”。
＜步骤S90＞第二AI运算结束
若第二AI运算结束(是)，则再次进行从步骤S10开始的一系列的处理，同时进行步骤S100的处理。继续直到第二AI运算结束(否)。
＜步骤S100＞
基于第二判断部22判断的场景，设定部12设定画质参数。调整部13使用画质参数来调整影像、即存在变化的帧图像以后的帧图像。
画质参数例如是明亮度、颜色浓淡、阴影、色温、锐度、降噪级别、对比度增强等级、细节增强等级。
例如，在风景场景的情况下，通过使明亮度等级、颜色浓淡以及阴影的各等级提高为超过标准参数，从而成为鲜明的影像。在人物场景的情况下，通过提高降噪级别以及细节增强等级并降低颜色浓淡等级，从而肌肤的质感变得自然。基于各个场景的画质参数例如预先存储于存储器32。
影像处理设备1是资源少的周边设备，但也能够输出与场景相应的最适当的影像。
如以上的说明那样，影像处理设备的工作方法具备：通过第一AI运算来 判断影像的变化等级的步骤S20；将所述变化等级与规定值进行比较的步骤S30；仅在所述变化等级超过所述规定值的情况下通过第二AI运算来判断影像被分类为多个场景的哪个场景的步骤S40；基于被判断的场景来设定画质参数的步骤S100；和使用所述画质参数来调整影像的步骤S100。
影像处理程序使计算机执行以下步骤：通过第一AI运算来判断影像的变化等级的步骤S20、将所述变化等级与规定值进行比较的步骤S30；仅在所述变化等级超过所述规定值的情况下通过第二AI运算来判断影像被分类为多个场景的哪个场景的步骤S40；基于被判断的场景来设定画质参数的步骤S100；和使用所述画质参数来调整影像的步骤S100。
＜第一实施方式的变形例1＞
本变形例的影像处理设备1A与影像处理设备1类似，因此针对相同功能的结构要素赋予相同的符号并省略说明。
影像处理设备1A例如获取附加于电视节目的影像信号的节目数据(例如，EPG：Electronic Programming Guide)。EPG数据被存储于存储器32。节目数据除了节目名、出演者、节目概要等，还具有种类数据。种类例如是“新闻/报道”、“运动”、“信息/幻灯片放映”、“电视剧”、“音乐”、“综艺”、“电影”、“动态图像片/特摄”、“纪录片/文化”、“剧场/公演”、“趣味/教育”、“福利”。
影像处理设备1A的设定部12基于种类以及第二判断部22判断的场景来设定画质参数。
即，即使是相同的风景场景，在种类是新闻的影像的情况下，也相比于种类是电影的影像的情况，设定提高明亮度等级、颜色浓淡以及阴影的各等级的比例较小的画质参数。因此，在种类是新闻的影像中，例如即使从人物场景切换为风景场景影像也不会较大地变化。相反地，在种类是电影的影像中，输出比种类是新闻的影像更具有震慑力的风景场景的影像。
基于多个种类的各个多个场景的画质参数例如预先存储于存储器32。影像处理设备1A能够根据种类，更加适当地调整场景的影像。
＜第一实施方式的变形例2＞
由于本变形例的影像处理设备1B与影像处理设备1类似，因此对相同功能的结构要素赋予相同的符号并省略说明。
影像处理设备1B的设定部12基于场景，不仅设定画质参数，还设定音质参数。调整部13不仅调整影像的画质，还使用音质参数来调整影像的声音。
音质参数例如是基于高通滤波器以及低通滤波器的均衡器等级、降噪级别。
例如，在人物的嘴活动的会话场景中，为了更容易听到，均衡器等级被设定为平坦，降噪级别被设定为较大。
在影像处理设备1B中，影像不仅调整图像，也根据场景来适当地调整声音。
在影像处理设备1B中，当然也可以如影像处理设备1A那样，基于种类以及第二判断部22判断的场景来设定画质参数。
＜第二实施方式＞
本实施方式的影像处理设备1C与影像处理设备1等类似，因此对相同功能的结构要素赋予相同的符号并省略说明。
按照图4的流程图，对影像处理设备1C的工作方法进行说明。
＜步骤S10-S30＞
与图2中说明的影像处理设备1相同。
＜步骤S41＞
第二AI运算开始，并进行处理直到结束。在第二AI运算结束后，再次进行从步骤S10开始的一系列的处理，同时进行步骤S100的处理。
＜步骤S100＞
与图2中说明的影像处理设备1相同。
在影像处理设备1C中，与影像处理设备1同样地，被反复进行的第一判断部21的第一处理间隔(时间)TA比第一AI运算的第一处理时间T1与第二AI运算的第二处理时间T2(T2A+T2B)的合计时间短。
如图5所示，影像处理设备1C的第一判断部21不重新开始处理直到第 二判断部22的处理结束。因此，进行了第二AI运算的情况下的第一判断部21的第二处理间隔TA2比未进行第二AI运算的情况下的第一处理间隔TA1长。
由于影像处理设备1C可能第一判断部21的处理间隔变长，因此在场景变化激烈的影像中担心不能适当地调整场景的影像。但是，在第一判断部21检测到影像的场景变化的情况下，能够比影像处理设备1更快地输出更加适当的画质的影像。
在影像处理设备1c中，当然也可以如影像处理设备1A那样基于种类以及第二判断部22判断的场景来进行画质调整，或者如影像处理设备1B那样基于场景来进行音质调整。
本公开还提供一种计算机可读的非易失性存储介质，所述存储介质存储有计算机指令，所述计算机指令被处理器执行时实现上述实施方式中的画质处理。
说明了本公开的几个实施方式，但这些实施方式作为示例而提示，并不意图限定本公开的范围。这些新的实施方式能够通过其它各种方式来实施，在不脱离本公开的主旨的范围内，能够进行各种省略、置换、变更。这些实施方式、其变形包含于本公开的范围、主旨，并且包含于权利要求书所述的本公开和其等同的范围。

Claims (8)

  1. 一种影像处理设备，其中，具备：
    第一判断部，通过第一AI运算，判断影像的变化等级；
    比较部，将所述变化等级与规定值进行比较；
    第二判断部，仅在所述变化等级超过所述规定值的情况下，通过第二AI运算来判断影像被分类为多个场景的哪个场景；
    设定部，基于被判断出的场景，设定画质参数；和
    调整部，使用所述画质参数来调整影像。
  2. 根据权利要求1所述的影像处理设备，其中，
    所述第一判断部的处理间隔比所述第一判断部的第一处理时间长，所述第一判断部的处理间隔比所述第一处理时间与所述第二判断部的第二处理时间的合计时间短，
    与所述第一判断部共享资源的所述第二判断部在不进行所述第一判断部的处理的期间分割来进行处理。
  3. 根据权利要求1所述的影像处理设备，其中，
    所述第一判断部的第一处理间隔比所述第一判断部的第一处理时间长，所述第一判断部的第一处理间隔比所述第一处理时间与所述第二判断部的第二处理时间的合计时间短，
    所述第一判断部不重新开始处理直到共享资源的所述第二判断部的处理结束为止。
  4. 根据权利要求1至3中的任一项所述的影像处理设备，其中，
    所述影像处理设备与调谐器、显示器以及扬声器一起，构成电视接收设备。
  5. 根据权利要求1至4中的任一项所述的影像处理设备，其中，
    所述影像是确定出种类的广播节目的影像，
    所述设定部基于所述种类以及所述第二判断部判断出的所述场景来设定 所述画质参数。
  6. 根据权利要求1至5中的任一项所述的影像处理设备，其中，
    所述设定部基于所述场景来设定音质参数，
    所述调整部使用所述音质参数来调整所述影像的声音。
  7. 一种影像处理设备的工作方法，其中，具备：
    通过第一AI运算来判断影像的变化等级；
    将所述变化等级与规定值进行比较；
    仅在所述变化等级超过所述规定值的情况下，通过第二AI运算来判断影像被分类为多个场景的哪个场景；
    基于被判断出的场景来设定画质参数；和
    使用所述画质参数来调整影像。
  8. 一种计算机可读的非易失性存储介质，其存储有程序或计算机指令，该程序或计算机指令在计算机上被执行，其中，使计算机执行以下步骤：
    通过第一AI运算来判断影像的变化等级；
    将所述变化等级与规定值进行比较；
    仅在所述变化等级超过所述规定值的情况下，通过第二AI运算来判断影像被分类为多个场景的哪个场景；
    基于被判断出的场景来设定画质参数；和
    使用所述画质参数来调整影像。
PCT/CN2021/118764 2021-01-20 2021-09-16 影像处理设备、影像处理设备的工作方法以及非易失性存储介质 WO2022156248A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202180027166.0A CN115398880A (zh) 2021-01-20 2021-09-16 影像处理设备、影像处理设备的工作方法以及非易失性存储介质

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021006896A JP7488002B2 (ja) 2021-01-20 2021-01-20 映像処理装置、映像処理装置の作動方法および映像処理プログラム
JP2021-006896 2021-01-20

Publications (1)

Publication Number Publication Date
WO2022156248A1 true WO2022156248A1 (zh) 2022-07-28

Family

ID=82548480

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/118764 WO2022156248A1 (zh) 2021-01-20 2021-09-16 影像处理设备、影像处理设备的工作方法以及非易失性存储介质

Country Status (3)

Country Link
JP (1) JP7488002B2 (zh)
CN (1) CN115398880A (zh)
WO (1) WO2022156248A1 (zh)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007121953A (ja) * 2005-10-31 2007-05-17 Nanao Corp 表示方法及び表示装置
US20080186413A1 (en) * 2007-02-02 2008-08-07 Mitsubishi Electric Corporation Video display apparatus
CN105898364A (zh) * 2016-05-26 2016-08-24 北京小米移动软件有限公司 一种视频播放处理方法、装置、终端及系统
CN108810649A (zh) * 2018-07-12 2018-11-13 深圳创维－Rgb电子有限公司 画质调节方法、智能电视机及存储介质
CN110933490A (zh) * 2019-11-20 2020-03-27 深圳创维－Rgb电子有限公司 一种画质和音质的自动调整方法、智能电视机及存储介质
CN110996153A (zh) * 2019-12-06 2020-04-10 深圳创维－Rgb电子有限公司 基于场景识别的音画品质增强方法、系统和显示器
US20200160062A1 (en) * 2018-11-16 2020-05-21 Samsung Electronics Co., Ltd. Image processing apparatus and method thereof

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009194550A (ja) 2008-02-13 2009-08-27 Sony Corp 画質調整装置、画質調整方法及びプログラム
JP2011166315A (ja) 2010-02-05 2011-08-25 Sharp Corp 表示装置、表示装置の制御方法、プログラム及び記録媒体
JP2014241457A (ja) 2011-10-04 2014-12-25 シャープ株式会社 シーンチェンジ検出装置、表示装置、シーンチェンジ検出方法およびシーンチェンジ検出プログラム
JP7113474B2 (ja) 2018-08-07 2022-08-05 国立研究開発法人情報通信研究機構 データ・セグメンテーション装置
US11115217B2 (en) 2018-11-21 2021-09-07 Avaya Inc. Systems and methods for detecting device location and usage
KR102662951B1 (ko) 2018-12-27 2024-05-02 엘지전자 주식회사 신호 처리 장치 및 이를 구비하는 영상표시장치

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2007121953A (ja) * 2005-10-31 2007-05-17 Nanao Corp 表示方法及び表示装置
US20080186413A1 (en) * 2007-02-02 2008-08-07 Mitsubishi Electric Corporation Video display apparatus
CN105898364A (zh) * 2016-05-26 2016-08-24 北京小米移动软件有限公司 一种视频播放处理方法、装置、终端及系统
CN108810649A (zh) * 2018-07-12 2018-11-13 深圳创维－Rgb电子有限公司 画质调节方法、智能电视机及存储介质
US20200160062A1 (en) * 2018-11-16 2020-05-21 Samsung Electronics Co., Ltd. Image processing apparatus and method thereof
CN110933490A (zh) * 2019-11-20 2020-03-27 深圳创维－Rgb电子有限公司 一种画质和音质的自动调整方法、智能电视机及存储介质
CN110996153A (zh) * 2019-12-06 2020-04-10 深圳创维－Rgb电子有限公司 基于场景识别的音画品质增强方法、系统和显示器

Also Published As

Publication number Publication date
JP2022111459A (ja) 2022-08-01
CN115398880A (zh) 2022-11-25
JP7488002B2 (ja) 2024-05-21

Similar Documents

Publication Publication Date Title
CN108810649B (zh) 画质调节方法、智能电视机及存储介质
US11601699B2 (en) Predictive content delivery for video streaming services
US20180359410A1 (en) Method and system of camera control and image processing with a multi-frame-based window for image data statistics
KR20210129583A (ko) 미디어 재생 디바이스에서의 콘텐츠 필터링
US9832373B2 (en) Systems and methods for automatically capturing digital images based on adaptive image-capturing templates
US20170332060A1 (en) Image processing method for locally adjusting image data of real-time image
CN111405339B (zh) 一种分屏显示方法、电子设备及存储介质
JP5142837B2 (ja) 表示制御装置および表示制御方法
JP2015149691A (ja) 画像補正装置、画像補正方法、及び、撮像装置
WO2021057412A1 (zh) 一种视频录制方法、装置、终端以及计算机可读存储介质
WO2017185584A1 (zh) 播放优化方法和装置
US8718376B2 (en) Apparatus and method for enhancing image in portable terminal
US11138437B2 (en) Image processing apparatus and method thereof
US20090304300A1 (en) Display control apparatus and display control method
WO2022151852A1 (zh) 图像处理方法、装置、系统、电子设备以及存储介质
CN111050212A (zh) 视频播放方法、设备及存储介质
US20190306462A1 (en) Image processing apparatus, videoconference system, image processing method, and recording medium
WO2022156248A1 (zh) 影像处理设备、影像处理设备的工作方法以及非易失性存储介质
TWI604413B (zh) 影像處理方法及影像處理裝置
JP2011166315A (ja) 表示装置、表示装置の制御方法、プログラム及び記録媒体
US20200106821A1 (en) Video processing apparatus, video conference system, and video processing method
JP2010183460A (ja) 撮像装置およびその制御方法
CN113542909A (zh) 视频处理方法、装置、电子设备及计算机存储介质
WO2019145835A1 (en) Enhancing image data with appearance controls
US20150130959A1 (en) Image processing device and exposure control method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21920626

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21920626

Country of ref document: EP

Kind code of ref document: A1