CN105184773A - Method and device for inserting pictures in scene switching - Google Patents

Method and device for inserting pictures in scene switching Download PDF

Info

Publication number
CN105184773A
CN105184773A CN201510497429.0A CN201510497429A CN105184773A CN 105184773 A CN105184773 A CN 105184773A CN 201510497429 A CN201510497429 A CN 201510497429A CN 105184773 A CN105184773 A CN 105184773A
Authority
CN
China
Prior art keywords
picture
information
pictures
dominant colors
time point
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201510497429.0A
Other languages
Chinese (zh)
Other versions
CN105184773B (en
Inventor
高同庆
王新志
李志强
丁宜辰
余春雨
宋虎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense Group Co Ltd
Original Assignee
Hisense Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hisense Group Co Ltd filed Critical Hisense Group Co Ltd
Priority to CN201810085403.9A priority Critical patent/CN108335292B/en
Priority to CN201510497429.0A priority patent/CN105184773B/en
Publication of CN105184773A publication Critical patent/CN105184773A/en
Application granted granted Critical
Publication of CN105184773B publication Critical patent/CN105184773B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10024Color image

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The embodiments of the invention relate to the field of an intelligent electronic device, and especially relates to a method and device for inserting pictures in scene switching, for solving the problem of incapability of realizing smooth switching of front and rear scene images in the aspect of color in scene switching in the prior art. The method provided by one embodiment of the invention comprises the following steps: determining color difference information between the last picture in a current scene image and the first picture in a target scene image; according to a time interval of picture insertion and the color difference information, determining color change information; and for time points when the pictures need to be inserted in a scene switching process, according to dominant color information of the last picture in the current scene image and the color change information, selecting the pictures inserted at the time points, and inserting the pictures when the time points come. According to the embodiments of the invention, since colors of the pictures inserted in the scene switching process are gradually changed, the problem of incapability of realizing smooth switching of the front and rear scene images in the aspect of color in the scene switching in the prior art is solved.

Description

The method and apparatus of insert pictures during a kind of scene switches
Technical field
The present invention relates to intelligent electronic device field, particularly a kind of scene switch in the method and apparatus of insert pictures.
Background technology
Along with the widespread use of intelligent electronic device, intelligent electronic device becomes and becomes increasingly complex, and except supporting basic function, has intelligent use and the game of various complexity simultaneously, in use, an inevitable problem is exactly switch application scene to intelligent electronic device.
In the prior art, the scene for intelligent electronic device switches, and mostly concentrates on the switch speed and switching time of optimizing insert pictures, and optimizes the mode inserted screenshot picture or insert outer setting picture.And for having the intelligent electronic device of audio-visual function, image quality is one of its important feature, if be absent from the scene, scape is not analyzed the picture of front and back scene before switching, when the picture color switching front and back scene is widely different, user has visually and impacts sense, can not realize taking over seamlessly in color when scene switches.
Summary of the invention
The invention provides the method and apparatus of insert pictures in the switching of a kind of scene, if widely different in order to the picture color solving the scene before switching in prior art and the scene after switching, the problem that take over seamlessly of front and back scene picture in color can not be realized.
Based on the problems referred to above, a kind of method of insert pictures during the embodiment of the present invention provides scene to switch, comprising:
Determine the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture;
According to time interval of insert pictures between the first pictures in last picture in current scene picture and target scenic picture, and described heterochromia information, determine color change information, the wherein said time interval is the duration between the adjacent time point needing insert pictures;
For the time point needing insert pictures in scene handoff procedure, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture that described time point needs to insert, and insert described picture after described time point arrives; Or according to the second dominant colors information of the first pictures in target scenic picture and described color change information, choose the picture that described time point needs to insert, and insert described picture after described time point arrives.
Due in the embodiment of the present invention in scene handoff procedure insert pictures, and according to the heterochromia information of the first pictures in last picture in current scene picture and target scenic picture, and the time interval of insert pictures, determine the color change information of the adjacent picture inserted, color information according to determining chooses picture, the color of picture in scene handoff procedure is gradually changed, if the picture color of the scene solved before switching at present and the scene after switching is widely different, then can not realize the problem that take over seamlessly of front and back scene picture in color.
Optionally, the described heterochromia information determining the first pictures in last picture and target scenic picture in current scene picture, comprising:
Determine the second dominant colors information of the first pictures in the first dominant colors information of last picture in described current scene picture and described target scenic picture respectively;
According to described first dominant colors information and described second dominant colors information, determine described heterochromia information.
Due to the heterochromia information of the first pictures in last picture in embodiment of the present invention determination current scene picture and target scenic picture, the picture inserted in scene handoff procedure is selected according to heterochromia information, the dominant colors information will choosing picture can be determined more exactly, thus make color change in scene handoff procedure more level and smooth.
Optionally, described the first dominant colors information determining last picture in current scene picture, comprising:
All pixels in last picture described are divided into multiple set, and each class color parameter value corresponding to the pixel wherein in each set is in same numerical range;
Determine in multiple set, to comprise the maximum set of number of pixels, and the same class color parameter value of each pixel in the set determined is averaged, obtain each class color parameter mean value;
Using the first dominant colors information of each class color parameter mean value as last picture in current scene picture.
Owing to embodiments providing the method for the first dominant colors information calculating last picture in current scene picture, the method is also applicable to calculate the dominant colors information wanting insert pictures chosen, use the method accurately can obtain the dominant colors information of picture, thus make the picture color chosen more accurate.
Optionally, described the second dominant colors information determining the first pictures in target scenic picture, comprising:
If there is the dominant colors information of the first pictures in the last described target scenic picture stored, then using the second dominant colors information of the dominant colors information of the first pictures in the described target scenic picture of the last time storage as the first pictures in described target scenic picture;
If there is no the dominant colors information of the first pictures in the last described target scenic picture stored, then using the second dominant colors information of default information as the first pictures in described target scenic picture.
Because the embodiment of the present invention second dominant colors information directly adopts the dominant colors information of the first pictures in the last this target scenic picture stored, if target scene is for entering for the first time, there is not the dominant colors information of the first pictures of storage, then the second dominant colors information directly adopts that default information is this determines mode simple and fast, can obtain the second dominant colors information of the first pictures in target scenic picture fast.
Optionally, described first dominant colors information comprises the first brightness value and the first intensity value; Described second dominant colors information comprises the second brightness value and the second intensity value;
Described according to the first dominant colors information and described second dominant colors information, determine described heterochromia information, specifically comprise:
According to the first brightness value of described first dominant colors information and the second brightness value determination luminance difference of described second dominant colors information, and according to the first intensity value of described first dominant colors information and the second intensity value determination saturation degree difference of described second dominant colors information;
If the absolute value of described saturation degree difference is greater than setting first threshold, and the absolute value of luminance difference is less than setting Second Threshold, then using described saturation degree difference as heterochromia information;
If the absolute value of described saturation degree difference is less than setting first threshold, and the absolute value of luminance difference is greater than setting Second Threshold, then using described luminance difference as heterochromia information;
If the absolute value of described saturation degree difference is greater than setting first threshold and the absolute value of described luminance difference is greater than setting Second Threshold, or the absolute value of described saturation degree difference is less than setting first threshold and the absolute value of described luminance difference is less than setting Second Threshold, then using comprehensive difference as described heterochromia information, wherein comprehensive difference equals described saturation degree difference and is multiplied by the first weights and described luminance difference are multiplied by the second weights and value.
Because the absolute value of saturation degree difference compares with first threshold by the embodiment of the present invention respectively, the absolute value of luminance difference compares with Second Threshold, determine the heterochromia information of the foundation when choosing picture, thus make the picture color chosen in scene handoff procedure more accurate, the color in scene handoff procedure is switched more level and smooth.
Optionally, described color change information is determined according to following formula:
I delta=I scope*Δt/T scope
Wherein, I deltafor color change information; I scopefor color difference information; Δ t is the time interval of insert pictures; T scopefor scene switches duration.
Owing to embodiments providing the Method for Accurate Calculation of color change information, thus color change information accurately can be obtained.
Optionally, choose the picture of insertion according to the first dominant colors information of last picture in described current scene picture and described color change information, comprising:
If described heterochromia information is saturation degree difference, then need the time point of insert pictures for n-th, the intensity value scope of the picture chosen is [S orig+ (n-1) * I delta, S orig+ n*I delta], and described in the brightness value of picture chosen be less than the 3rd threshold value, in the brightness value of described picture and current scene picture, the absolute value of the first brightness value difference of last picture is not more than the 4th threshold value, described in the brightness value of picture chosen and target scenic picture the absolute value of the second brightness value difference of the first pictures be not more than the 4th threshold value;
If described heterochromia information is luminance difference, then need the time point of insert pictures for n-th, choose the range of luminance values of picture for [L orig+ (n-1) * I delta, L orig+ n*I delta]; And the intensity value of described picture is less than the 5th threshold value, in the intensity value of described picture and current scene picture, the absolute value of the first intensity value difference of last picture is not more than the 6th threshold value, and in the intensity value of described picture and target scenic picture, the absolute value of the second intensity value difference of the first pictures is not more than the 6th threshold value;
If described heterochromia information is comprehensive difference, then need the time point of insert pictures for n-th, choose the desired value I of picture xscope is [I orig+ (n-1) * I delta, I orig+ n*I delta], and I x=S x* A+L x* B;
Wherein, I orig=S orig* A+L orig* B; S xfor the intensity value of picture chosen; L xfor the brightness value of picture chosen; I deltafor color change information; S origfor the first intensity value of last picture in current scene picture; L origfor the first brightness value of last picture in current scene picture; I origfor the first desired value of last picture in current scene picture; A is the first weights; B is the second weights.
Because the embodiment of the present invention is according to the difference of heterochromia information, three kinds of situation definition are divided into choose the principle of picture, all provide the computing method of the dominant colors value scope of each time point insert pictures at each occurrence, dominant colors value scope according to each time point insert pictures chooses picture, thus the picture color chosen is met the requirements more, realize front and back scene picture taking over seamlessly in color.
Optionally, if do not select the picture that described time point inserts, and there is the time point of a upper insert pictures, then for the time point of a upper insert pictures, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture of insertion; The picture chosen is needed on the described time point not selecting picture the picture inserted, the picture wherein chosen is different from the picture that a upper time point inserts;
If do not select the picture that described time point inserts, and there is not the time point of a upper insert pictures, then the picture of setting is needed on the described time point not selecting picture the picture inserted.
Because the embodiment of the present invention is for the time point not selecting insert pictures, provide other modes choosing picture at this time point, ensure can choose picture at the time point of each insert pictures, thus improve the reliability choosing picture.
On the other hand, the equipment of insert pictures during a kind of scene that the embodiment of the present invention provides switches, comprising:
Determination module, for determining the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture;
Processing module, for according to time interval of insert pictures between the first pictures in last picture in current scene picture and target scenic picture, and described heterochromia information, determine color change information, the wherein said time interval is the duration between the adjacent time point needing insert pictures;
Choose module, for for the time point needing insert pictures in scene handoff procedure, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture that described time point needs to insert, and insert described picture after described time point arrives; Or according to the second dominant colors information of the first pictures in target scenic picture and described color change information, choose the picture that described time point needs to insert, and insert described picture after described time point arrives; Or according to the second dominant colors information of the first pictures in target scenic picture and described color change information, choose the picture that described time point needs to insert, and insert described picture after described time point arrives.
Optionally, described determination module, specifically for:
Determine the second dominant colors information of the first pictures in the first dominant colors information of last picture in described current scene picture and described target scenic picture respectively; According to described first dominant colors information and described second dominant colors information, determine described heterochromia information.
Accompanying drawing explanation
Fig. 1 for a kind of scene that the embodiment of the present invention provides switch in the method flow schematic diagram one of insert pictures;
Fig. 2 is pixel classifications schematic diagram in embodiment of the present invention picture;
Fig. 3 is the method schematic diagram of embodiment of the present invention insert pictures;
Fig. 4 for a kind of scene that the embodiment of the present invention provides switch in the method flow schematic diagram two of insert pictures;
Fig. 5 for a kind of scene that the embodiment of the present invention provides switch in the device structure schematic diagram of insert pictures.
Embodiment
The heterochromia information of the first pictures in last picture and target scenic picture in embodiment of the present invention determination current scene picture; According to time interval of insert pictures between the first pictures in last picture in current scene picture and target scenic picture, and described heterochromia information, determine color change information, the wherein said time interval is the duration between the adjacent time point needing insert pictures; For the time point needing insert pictures in scene handoff procedure, according to the first dominant colors information and the color change information of last picture in current scene picture, choose the picture that described time point needs to insert, and insert described picture after described time point arrives; Or according to the second dominant colors information of the first pictures in target scenic picture and described color change information, choose the picture that described time point needs to insert, and insert described picture after described time point arrives.Because the embodiment of the present invention is according to the heterochromia information of the first pictures in last picture in current scene picture and target scenic picture, determine the color change information of each time point insert pictures, and according to the dominant colors information of selected picture, select the picture inserted in scene handoff procedure, thus realization front and back scene picture in scene handoff procedure takes over seamlessly in color.
Below in conjunction with Figure of description, the embodiment of the present application is described in further detail.
As shown in Figure 1, in a kind of scene switching of the embodiment of the present invention, the method for insert pictures comprises:
Step 101, determines the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture;
Step 102, according to time interval of insert pictures between the first pictures in last picture in current scene picture and target scenic picture, and described heterochromia information, determine color change information, the wherein said time interval is the duration between the adjacent time point needing insert pictures;
Step 103, for the time point needing insert pictures in scene handoff procedure, according to the first dominant colors information and the color change information of last picture in current scene picture, choose the picture that this time point needs to insert, and insert described picture after this time point arrives; Or according to the second dominant colors information of the first pictures in target scenic picture and described color change information, choose the picture that described time point needs to insert, and insert described picture after described time point arrives.
Wherein, embodiment of the present invention executive agent includes but not limited to: TV, computer, mobile phone etc.
Dominant colors information in the embodiment of the present invention represents the color parameter value of effect diagram sheet color, is the color parameter value played a decisive role to the color of picture.Wherein, what dominant colors information included but not limited in following parameters value is part or all of:
Tone value, intensity value and brightness value.
Optionally, when determining the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture, determine the second dominant colors information of the first pictures in the first dominant colors information of last picture in current scene picture and target scenic picture respectively, according to the first dominant colors information and the second dominant colors information determination heterochromia information.
In the embodiment of the present invention, current scene is the scene that using of user before switching, in current scene picture last picture for user determine handoff scenario time, the picture that equipment is showing; Target scene is the scene that user needs to enter, and in target scenic picture, the first pictures is enter the picture that first this target scene device show.
Optionally, according to the first dominant colors information of last picture in following manner determination current scene picture.
Pixels all in last picture are divided into multiple set, and each class color parameter value corresponding to the pixel wherein in each set is in same numerical range; Determine in multiple set, to comprise the maximum set of number of pixels, and the same class color parameter value of each pixel in the set determined is averaged, obtain each class color parameter mean value; Using the first dominant colors information of each class color parameter mean value as last picture in current scene picture.
Below using tone, saturation degree and brightness as color parameter, illustrate the mode determining the first dominant colors information.
First, based on the ultimate resolution that display screen and displaying contents can be supported, intercept last picture in current scene, for the rgb value of pixel each in picture, obtain its corresponding HSL (tone, saturation degree, brightness) value, color is divided into six large class C1 ~ C6 by the scope according to saturation degree and brightness, as shown in Figure 2, the scope of saturation degree and brightness is 0 ~ 1.0, because brightness is larger on the impact of whole colour field, brightness is divided three classes, low-light level (0 ~ 0.33), middle brightness (0.33 ~ 0.66), high brightness (0.66 ~ 1.0); Saturation degree is divided into two classes, low saturation (0 ~ 0.5), high saturation (0.5 ~ 1.0), wherein C 1represent high brightness high saturation, C 2brightness high saturation in expression, C 3represent low-light level high saturation, C 4represent high brightness low saturation, C 5brightness low saturation in expression, C 6represent low-light level low saturation.
The concrete account form of the first dominant colors information is as follows:
1, obtain intensity value (S) and brightness value (L) that in picture, each pixel RGB is corresponding, and according to intensity value and brightness value, this pixel is referred to C 1~ C 6in;
2, C is determined 1~ C 6the class that middle number of pixels is maximum, as the dominant colors class C of described picture dom;
3, dominant colors class C is calculated domin the color parameter mean value of all pixels and will as the dominant colors information of described picture.
Calculate dominant colors class C domin the color parameter mean value of all pixels comprise calculating with suppose C domthere is N number of pixel, then have:
H ‾ = ( H 0 + H 1 + ... + H n ) / N ;
S ‾ = ( S 0 + S 1 + ... + S n ) / N ;
L ‾ = ( L 0 + L 1 + ... + L n ) / N ;
H S L ‾ = H ‾ , S ‾ , L ‾ ;
Color parameter mean value be the first dominant colors information of last picture in current scene picture.
Above-mentioned account form just illustrates, in force, if do not need some color parameter, can not calculate in computation process, if than in the process choosing picture, the color parameter of foundation only has saturation degree and brightness, then can not calculate the value relevant to colorimetric parameter when calculating dominant colors information.
The embodiment of the present invention, in the process choosing picture, the dominant colors information of the picture chosen also obtains according to above-mentioned account form.
In enforcement, the first dominant colors information of last picture in the current scene picture at every turn calculated is stored and upgrades in time.
Optionally, when determining the second dominant colors information of the first pictures in target scenic picture, if target scene is not enter first time, then there is the dominant colors information of the first pictures in the last this target scenic picture stored, if target scene enters first time, then there is no the dominant colors information of the first pictures in the last this target scenic picture stored.
Whether basis has the dominant colors information of the first pictures in the last target scenic picture stored below, the second dominant colors information of the first pictures in point two kinds of different situations determination target scenic picture.
Situation one: have the dominant colors information that in the last this target scenic picture stored, the first pictures is corresponding, now, using the second dominant colors information of dominant colors information corresponding for this picture as the first pictures in target scenic picture.
Situation two: do not have the dominant colors information that in the last this target scenic picture stored, the first pictures is corresponding, then using the second dominant colors information of default information as the first pictures in this target scenic picture.
In the embodiment of the present invention, the information preset is by the first dominant colors information of last picture in the current scene picture that calculates, as the second dominant colors information of the first pictures in this target scenic picture, ensure that the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture is 0.
Optionally, first dominant colors information comprises the first brightness value and the first intensity value, second dominant colors information comprises the second brightness value and the second intensity value, determining the first brightness value and first intensity value of last picture in current scene picture, and in target scenic picture after the second brightness value of the first pictures and the second intensity value, calculate saturation degree difference and the luminance difference of the first pictures in last picture and target scenic picture in current scene picture respectively; Size according to the absolute value of saturation degree difference and the absolute value of luminance difference can obtain different heterochromia information, and the rule that heterochromia information difference chooses picture is not identical yet.
Wherein, according to the size of the absolute value of saturation degree difference and the absolute value of luminance difference, can according to following manner determination heterochromia information:
If the absolute value of saturation degree difference is greater than setting first threshold, and the absolute value of luminance difference is less than setting Second Threshold, then using saturation degree difference as heterochromia information; If the absolute value of saturation degree difference is less than setting first threshold, and the absolute value of luminance difference is greater than setting Second Threshold, then using luminance difference as heterochromia information; If the absolute value of saturation degree difference is greater than setting first threshold and the absolute value of luminance difference is greater than setting Second Threshold, or the absolute value of saturation degree difference is less than setting first threshold and the absolute value of luminance difference is less than setting Second Threshold, then using comprehensive difference as described heterochromia information; Wherein, comprehensive difference equals saturation degree difference and is multiplied by the first weights and luminance difference are multiplied by the second weights and value.
Below according to different heterochromia information, point three kinds of different situations choose picture.
Situation one: heterochromia information is saturation degree difference.
According to following formula determination color change information:
I delta=I scope* Δ t/T scope; Wherein, I deltafor color change information; I scopefor heterochromia information; Δ t is the time interval of insert pictures; T scopefor scene switches duration;
Need the time point of insert pictures for n-th, the intensity value scope of the picture chosen is [S orig+ (n-1) * I delta, S orig+ n*I delta], and the brightness value of the picture chosen is less than the 3rd threshold value, the absolute value choosing the first brightness value difference of last picture in the brightness value of picture and current scene picture is not more than the 4th threshold value, and the absolute value choosing the second brightness value difference of the first pictures in the brightness value of picture and target scenic picture is not more than the 4th threshold value.
In the embodiment of the present invention, heterochromia information is designated as I scope, the intensity value of the picture that choose is S x, brightness value is L x, in current scene picture, the first intensity value of last picture is S orig, in current scene picture, the first brightness of last picture is L orig, in target scenic picture, the second intensity value of the first pictures is S dest, in target scenic picture, the second brightness value of the first pictures is L dest.
Be 0.6 with first threshold, Second Threshold, the 3rd threshold value and the 4th threshold value below, the time interval Δ t of insert pictures is 2s is example, illustrates when heterochromia information is saturation degree difference, chooses the method for picture.
First threshold, Second Threshold, the 3rd threshold value and the 4th threshold value are 0.6, i.e. ︱ S dest-S orig︱ >0.6, and ︱ L dest-L orig︱ <0.6, now, heterochromia information is:
I scope=S dest-S orig, and ︱ L x-L dest︱ <0.6, ︱ L x-L orig︱ <0.6.
I scopefor the saturation degree difference of the first pictures in last picture to target scenic picture from current scene picture, it is T that scene switches duration scope, insert a pictures every the 2s time, in the 2s time, saturation degree change information is I delta, then:
I delta=I scope*2/T scope
Insert pictures schematic diagram in scene handoff procedure as shown in Figure 3, is the time point of first insert pictures at the end of current scene, inserts the first pictures at this time point, the saturation degree S of the first pictures xat [S orig, S orig+ I delta] in scope, and the brightness L of the first pictures x<0.6; Interval time, 2s, inserted the second pictures at second time point, now the saturation degree S of the second pictures xwill at [S orig+ I delta, S orig+ 2I delta] in scope, and the brightness L of the second pictures x<0.6; Then insert the n-th pictures at the n-th time point, now the saturation degree S of the n-th pictures xwill at [S orig+ (n-1) * I delta, S orig+ n*I delta] in scope, and the brightness L of the n-th pictures x<0.6.
Situation two: heterochromia information is luminance difference.
According to following formula determination color change information:
I delta=I scope* Δ t/T scope; Wherein, I deltafor color change information; I scopefor heterochromia information; Δ t is the time interval of insert pictures; T scopefor scene switches duration;
Need the time point of insert pictures for n-th, choose the range of luminance values of picture for [L orig+ (n-1) * I delta, L orig+ n*I delta]; And the intensity value of described picture is less than the 5th threshold value, in the intensity value of described picture and current scene picture, the absolute value of the first intensity value difference of last picture is not more than the 6th threshold value, and in the intensity value of described picture and target scenic picture, the absolute value of the second intensity value difference of the first pictures is not more than the 6th threshold value;
In the embodiment of the present invention, heterochromia information is designated as I scope, the intensity value of the picture that choose is S x, brightness value is L x, in current scene picture, the first intensity value of last picture is S orig, in current scene picture, the first brightness of last picture is L orig, in target scenic picture, the second intensity value of the first pictures is S dest, in target scenic picture, the second brightness value of the first pictures is L dest.
Be 0.6 with first threshold, Second Threshold, the 5th threshold value and the 6th threshold value below, the time interval Δ t of insert pictures is 2s is example, illustrates when heterochromia information is luminance difference, chooses the method for picture.
First threshold, Second Threshold, the 5th threshold value and the 6th threshold value are 0.6, i.e. ︱ S dest-S orig︱ <0.6, and ︱ L dest-L orig︱ >0.6, now, heterochromia information is:
I scope=L dest-L orig, and ︱ S x-S dest︱ <0.6, ︱ S x-S orig︱ <0.6.
I scopefor the luminance difference of the first pictures in last picture to target scenic picture from current scene picture, it is T that scene switches duration scope, insert a pictures every the 2s time, in the 2s time, brightness change information is I delta, then:
I delta=I scope*2/T scope
Insert pictures schematic diagram in scene handoff procedure as shown in Figure 3, is the time point of first insert pictures at the end of current scene, inserts the first pictures at this time point, the brightness L of the first pictures xat [L orig, L orig+ I delta] in scope, and the saturation degree S of the first pictures x<0.6; Interval time, 2s, inserted the second pictures at second time point, the now brightness L of the second pictures xwill at [L orig+ I delta, L orig+ 2I delta] in scope, and the saturation degree S of the second pictures x<0.6; Then insert the n-th pictures at the n-th time point, now the brightness L of the n-th pictures xwill at [L orig+ (n-1) * I delta, L orig+ n*I delta] in scope, and the saturation degree S of the n-th pictures x<0.6.
Situation three: heterochromia information is comprehensive difference.
According to following formula determination color change information:
I delta=I scope* Δ t/T scope; Wherein, I deltafor color change information; I scopefor heterochromia information; Δ t is the time interval of insert pictures; T scopefor scene switches duration;
Need the time point of insert pictures for n-th, choose the desired value I of picture xscope is [I orig+ (n-1) * I delta, I orig+ n*I delta], and I x=S x* A+L x* B;
Wherein, I orig=S orig* A+L orig* B; S xfor the intensity value of picture chosen; L xfor the brightness value of picture chosen; I deltafor color change information; S origfor the first intensity value of last picture in current scene picture; L origfor the first brightness value of last picture in current scene picture; I origfor the first desired value of last picture in current scene picture; A is the first weights; B is the second weights.
Be 0.6 with first threshold and Second Threshold below, the first weights are the 0.4, second weights is 0.6, and the time interval Δ t of insert pictures is 2s is example, illustrates when heterochromia information is comprehensive difference, chooses the method for picture.
First threshold and Second Threshold are 0.6, and the first weights are the 0.4, second weights is 0.6, i.e. ︱ S dest-S orig︱ >0.6 and ︱ L dest-L orig︱ >0.6, or, ︱ S dest-S orig︱ <0.6 and ︱ L dest-L orig︱ <0.6, now, heterochromia information is:
I scope=(S dest-S orig)*0.4+(L dest-L orig)*0.6;
I scopefor the saturation degree of the first pictures in last picture to target scenic picture from current scene picture and the comprehensive difference of brightness, it is T that scene switches duration scope, insert a pictures every the 2s time, in the 2s time, brightness change information is I delta, then:
I delta=I scope*2/T scope
In enforcement, for each picture that will choose, according to its desired value I of following formulae discovery x:
I x=S x*0.4+L x*0.6;
Meanwhile, the desired value I of last picture in current scene picture orig=S orig* 0.4+L orig* 0.6.
Insert pictures schematic diagram in scene handoff procedure as shown in Figure 3, is the time point of first insert pictures at the end of current scene, inserts the first pictures at this time point, the desired value I of the first pictures xat [I orig, I orig+ I delta] in scope, interval time, 2s, inserted the second pictures at second time point, now the desired value I of the second pictures xat [I orig+ I delta, I orig+ 2I delta] in scope, then insert the n-th pictures at the n-th time point, now the desired value I of the n-th pictures xwill at [I orig+ (n-1) * I delta, I orig+ n*I delta] in scope.
In the embodiment of the present invention, if do not select the picture of described time point insertion according to the above-mentioned mode choosing picture, the picture that two kinds of these time points of way selection below insert can be adopted.
Mode one: if on this time point exists the time point of an insert pictures, then for the time point of a upper insert pictures, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture of insertion; The picture chosen is needed on the described time point not selecting picture the picture inserted, the picture wherein chosen is different from the picture that a upper time point inserts.
Such as, the picture met the demands is not chosen as shown in Figure 3 at the time point of second insert pictures, the principle of picture is then chosen according to the time point of first insert pictures, be that the time point of second insert pictures chooses picture, and the picture choosing this time point is different from the picture that first time point is chosen.
Mode two: if on this time point does not exist the time point of an insert pictures, then the picture of setting is needed on the described time point not selecting picture the picture inserted.
Such as, as shown in Figure 3, do not choose the picture met the demands at the time point of first insert pictures, then by default picture as the picture that this first time point inserts.
The embodiment of the present invention is when choosing picture, and in order to improve the efficiency choosing picture, preferentially choose from the picture stored, in the picture stored, do not choose picture, the scope choosing picture can also be the external unit etc. of network or insertion.
Optionally, when to each picture-storage, according to the account form of picture dominant colors information, calculate the dominant colors information HSL of each picture x(H x, S x, L x), and the desired value I that picture is corresponding x=S x* A+L x* B, and by the result that obtains with S x, L x, I xfor key value, set up three Hash tables.
The embodiment of the present invention, to set up the mode of Hash table, records the S of each picture x, L x, I xvalue, thus be convenient to choose picture, improve the efficiency choosing picture.
The mode of embodiment of the present invention insert pictures includes but not limited to following two kinds of modes.
Mode one: first choose the picture that all time points will insert, after described time point arrives, inserts corresponding picture.
In the embodiment of the present invention, the starting stage is switched in scene, according to the dominant colors information of the first pictures in last picture in current scene picture and target scenic picture, and the time interval determined, choosing allly in scene handoff procedure needs the picture that the time point of insert pictures is corresponding, in handoff procedure, after the time point arrival of each insert pictures, insert the corresponding picture chosen, such as, as shown in Figure 3, at the end of current scene, according to the above-mentioned principle choosing picture, choose very first time point, second time point, the picture of the 3rd time point and the 4th time point, then in scene handoff procedure, picture corresponding to this time point is inserted every 2s.
Mode two: after described time point arrives, choose the picture that this time point will insert, and insert pictures.
In the embodiment of the present invention, for each time point, according to the above-mentioned principle choosing picture, choose and insert pictures, such as, as shown in Figure 3, at the end of current scene, for first time point, choose and insert pictures, after the 2s of interval, then according to second time point, choose and insert pictures.
It should be noted that, the embodiment of the present invention, for the time point of an insert pictures, can insert one or a picture group sheet at this time point, when insertion one picture group sheet, is reduced by this picture group sheet, and this picture group sheet is presented on same picture.If select according to the above-mentioned mode choosing picture the picture that time point described in multiple inserts, then random selecting one or one group from plurality of pictures, as the picture that this time point inserts.
As shown in Figure 4, be the overall flow figure of the method for insert pictures in a kind of scene handoff procedure of the embodiment of the present invention.
Step 401, calculates the first dominant colors information of last picture in current scene picture;
Step 402, judges whether the dominant colors information of the first pictures in the target scenic picture that existence stores, if so, performs step 403, if not, performs step 404;
Step 403, using the second dominant colors information of the dominant colors information of the first pictures in the target scenic picture of the last time storage as the first pictures in target scenic picture;
Step 404, using the second dominant colors information of default information as the first pictures in target scenic picture;
Step 405, according to the first dominant colors information and the second dominant colors information determination heterochromia information;
Step 406, according to heterochromia information, the time interval of insert pictures and scene switch duration determination color change information;
Step 407, for the time point of insert pictures, according to the picture that the first dominant colors information and color change information select this time point to insert;
Step 408, judges whether to choose the picture satisfied condition, and if so, performs step 412, if not, performs step 409;
Step 409, judges that whether this time point is first time point of insert pictures, if so, performs step 410, if not, perform step 411;
Step 410, the picture that the picture choosing setting inserts as this time point;
Step 411, chooses picture for a upper time point, and using the picture that the picture chosen will insert as this time point, the picture wherein chosen is different from the picture that a time point inserts;
Step 412, insert pictures after this time point arrives.
Based on same inventive concept, additionally provide in the embodiment of the present application a kind of scene switch in insert pictures equipment, during the principle of dealing with problems due to this equipment switches to the embodiment of the present application scene, the method for insert pictures is similar, therefore the enforcement of this equipment see the enforcement of method, can repeat part and repeats no more.
As shown in Figure 5, the equipment of insert pictures during a kind of scene switches, comprising:
Determination module 501, for determining the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture;
Processing module 502, for according to time interval of insert pictures between the first pictures in last picture in current scene picture and target scenic picture, and described heterochromia information, determine color change information;
Choose module 503, for for the time point needing insert pictures in scene handoff procedure, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture that described time point needs to insert, and insert described picture after described time point arrives; Or according to the second dominant colors information of the first pictures in target scenic picture and described color change information, choose the picture that described time point needs to insert, and insert described picture after described time point arrives.
Optionally, determination module 501, specifically for:
Determine the second dominant colors information of the first pictures in the first dominant colors information of last picture in described current scene picture and described target scenic picture respectively; According to described first dominant colors information and described second dominant colors information, determine described heterochromia information.
Above with reference to illustrating that the block diagram of method, device (system) and/or computer program according to the embodiment of the present application and/or process flow diagram describe the application.Should be understood that the combination that can be realized a block of block diagram and/or flowchart illustration and the block of block diagram and/or flowchart illustration by computer program instructions.These computer program instructions can be supplied to multi-purpose computer, the processor of special purpose computer and/or other programmable data treating apparatus, to produce machine, make the instruction establishment performed via computer processor and/or other programmable data treating apparatus for realizing the method for function/action specified in block diagram and/or flow chart block.
Correspondingly, the application can also be implemented with hardware and/or software (comprising firmware, resident software, microcode etc.).Further, the application can take computing machine to use or the form of computer program on computer-readable recording medium, it has the computing machine realized in media as well and can use or computer readable program code, to be used by instruction execution system or combined command executive system and using.In the present context, computing machine can use or computer-readable medium can be arbitrary medium, it can comprise, store, communicate, transmit or convey program, with by instruction execution system, device or equipment use, or combined command executive system, device or equipment use.
Obviously, those skilled in the art can carry out various change and modification to the present invention and not depart from the spirit and scope of the present invention.Like this, if these amendments of the present invention and modification belong within the scope of the claims in the present invention and equivalent technologies thereof, then the present invention is also intended to comprise these change and modification.

Claims (10)

1. scene switch in the method for insert pictures, it is characterized in that, the method comprises:
Determine the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture;
According to time interval of insert pictures between the first pictures in last picture in current scene picture and target scenic picture, and described heterochromia information, determine color change information, the wherein said time interval is the duration between the adjacent time point needing insert pictures;
For the time point needing insert pictures in scene handoff procedure, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture that described time point needs to insert, and insert described picture after described time point arrives; Or according to the second dominant colors information of the first pictures in target scenic picture and described color change information, choose the picture that described time point needs to insert, and insert described picture after described time point arrives.
2. the method for claim 1, is characterized in that, the described heterochromia information determining the first pictures in last picture and target scenic picture in current scene picture, comprising:
Determine the second dominant colors information of the first pictures in the first dominant colors information of last picture in described current scene picture and described target scenic picture respectively;
According to described first dominant colors information and described second dominant colors information, determine described heterochromia information.
3. method as claimed in claim 2, it is characterized in that, described the first dominant colors information determining last picture in current scene picture, comprising:
All pixels in last picture described are divided into multiple set, and each class color parameter value corresponding to the pixel wherein in each set is in same numerical range;
Determine in multiple set, to comprise the maximum set of number of pixels, and the same class color parameter value of each pixel in the set determined is averaged, obtain each class color parameter mean value;
Using the first dominant colors information of each class color parameter mean value as last picture in current scene picture.
4. method as claimed in claim 2, it is characterized in that, described the second dominant colors information determining the first pictures in target scenic picture, comprising:
If there is the dominant colors information of the first pictures in the last described target scenic picture stored, then using the second dominant colors information of the dominant colors information of the first pictures in the described target scenic picture of the last time storage as the first pictures in described target scenic picture;
If there is no the dominant colors information of the first pictures in the last described target scenic picture stored, then using the second dominant colors information of default information as the first pictures in described target scenic picture.
5. method as claimed in claim 2, it is characterized in that, described first dominant colors information comprises the first brightness value and the first intensity value; Described second dominant colors information comprises the second brightness value and the second intensity value;
Described according to the first dominant colors information and described second dominant colors information, determine described heterochromia information, specifically comprise:
According to the first brightness value of described first dominant colors information and the second brightness value determination luminance difference of described second dominant colors information, and according to the first intensity value of described first dominant colors information and the second intensity value determination saturation degree difference of described second dominant colors information;
If the absolute value of described saturation degree difference is greater than setting first threshold, and the absolute value of luminance difference is less than setting Second Threshold, then using described saturation degree difference as heterochromia information;
If the absolute value of described saturation degree difference is less than setting first threshold, and the absolute value of luminance difference is greater than setting Second Threshold, then using described luminance difference as heterochromia information;
If the absolute value of described saturation degree difference is greater than setting first threshold and the absolute value of described luminance difference is greater than setting Second Threshold, or the absolute value of described saturation degree difference is less than setting first threshold and the absolute value of described luminance difference is less than setting Second Threshold, then using comprehensive difference as described heterochromia information, wherein comprehensive difference equals described saturation degree difference and is multiplied by the first weights and described luminance difference are multiplied by the second weights and value.
6. method as claimed in claim 5, is characterized in that, determine described color change information according to following formula:
I delta=I scope*Δt/T scope
Wherein, I deltafor color change information;
I scopefor heterochromia information;
Δ t is the time interval of insert pictures;
T scopefor scene switches duration.
7. method as claimed in claim 6, is characterized in that, chooses the picture of insertion, comprising according to the first dominant colors information of last picture in described current scene picture and described color change information:
If described heterochromia information is saturation degree difference, then need the time point of insert pictures for n-th, the intensity value scope of the picture chosen is [S orig+ (n-1) * I delta, S orig+ n*I delta], and described in the brightness value of picture chosen be less than the 3rd threshold value, in the brightness value of described picture and current scene picture, the absolute value of the first brightness value difference of last picture is not more than the 4th threshold value, described in the brightness value of picture chosen and target scenic picture the absolute value of the second brightness value difference of the first pictures be not more than the 4th threshold value;
If described heterochromia information is luminance difference, then need the time point of insert pictures for n-th, choose the range of luminance values of picture for [L orig+ (n-1) * I delta, L orig+ n*I delta]; And the intensity value of described picture is less than the 5th threshold value, in the intensity value of described picture and current scene picture, the absolute value of the first intensity value difference of last picture is not more than the 6th threshold value, and in the intensity value of described picture and target scenic picture, the absolute value of the second intensity value difference of the first pictures is not more than the 6th threshold value;
If described heterochromia information is comprehensive difference, then need the time point of insert pictures for n-th, choose the desired value I of picture xscope is [I orig+ (n-1) * I delta, I orig+ n*I delta], and I x=S x* A+L x* B;
Wherein, I orig=S orig* A+L orig* B;
S xfor the intensity value of picture chosen;
L xfor the brightness value of picture chosen;
I deltafor color change information;
S origfor the first intensity value of last picture in current scene picture;
L origfor the first brightness value of last picture in current scene picture;
I origfor the first desired value of last picture in current scene picture;
A is the first weights;
B is the second weights.
8. method as claimed in claim 7, it is characterized in that, the method also comprises:
If do not select the picture that described time point inserts, and there is the time point of a upper insert pictures, then for the time point of a upper insert pictures, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture of insertion; The picture chosen is needed on the described time point not selecting picture the picture inserted, the picture wherein chosen is different from the picture that a upper time point inserts;
If do not select the picture that described time point inserts, and there is not the time point of a upper insert pictures, then the picture of setting is needed on the described time point not selecting picture the picture inserted.
9. scene switch in the equipment of insert pictures, it is characterized in that, this equipment comprises:
Determination module, for determining the heterochromia information of the first pictures in last picture and target scenic picture in current scene picture;
Processing module, for according to time interval of insert pictures between the first pictures in last picture in current scene picture and target scenic picture, and described heterochromia information, determine color change information, the wherein said time interval is the duration between the adjacent time point needing insert pictures;
Choose module, for for the time point needing insert pictures in scene handoff procedure, according to the first dominant colors information and the described color change information of last picture in current scene picture, choose the picture that described time point needs to insert, and insert described picture after described time point arrives; Or according to the heterochromia information of the first pictures in last picture in current scene picture and target scenic picture, and the time interval of insert pictures, determine the color change information of the adjacent picture inserted, the color information according to determining chooses picture.
10. equipment as claimed in claim 9, is characterized in that, described determination module, specifically for:
Determine the second dominant colors information of the first pictures in the first dominant colors information of last picture in described current scene picture and described target scenic picture respectively; According to described first dominant colors information and described second dominant colors information, determine described heterochromia information.
CN201510497429.0A 2015-08-13 2015-08-13 The method and apparatus of insert pictures in a kind of scene switching Expired - Fee Related CN105184773B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201810085403.9A CN108335292B (en) 2015-08-13 2015-08-13 Method for inserting picture in scene switching
CN201510497429.0A CN105184773B (en) 2015-08-13 2015-08-13 The method and apparatus of insert pictures in a kind of scene switching

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510497429.0A CN105184773B (en) 2015-08-13 2015-08-13 The method and apparatus of insert pictures in a kind of scene switching

Related Child Applications (1)

Application Number Title Priority Date Filing Date
CN201810085403.9A Division CN108335292B (en) 2015-08-13 2015-08-13 Method for inserting picture in scene switching

Publications (2)

Publication Number Publication Date
CN105184773A true CN105184773A (en) 2015-12-23
CN105184773B CN105184773B (en) 2018-05-01

Family

ID=54906826

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201810085403.9A Active CN108335292B (en) 2015-08-13 2015-08-13 Method for inserting picture in scene switching
CN201510497429.0A Expired - Fee Related CN105184773B (en) 2015-08-13 2015-08-13 The method and apparatus of insert pictures in a kind of scene switching

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201810085403.9A Active CN108335292B (en) 2015-08-13 2015-08-13 Method for inserting picture in scene switching

Country Status (1)

Country Link
CN (2) CN108335292B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110706666B (en) * 2019-09-16 2021-08-24 深圳市华星光电半导体显示技术有限公司 Picture transition method, device, controller and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040189878A1 (en) * 2003-03-25 2004-09-30 Yamaha Corporation Apparatus and program for setting video processing parameters
US20080019610A1 (en) * 2004-03-17 2008-01-24 Kenji Matsuzaka Image processing device and image processing method
CN101594471A (en) * 2008-05-30 2009-12-02 新奥特(北京)视频技术有限公司 A kind of method for changing scenes
CN102067616A (en) * 2009-04-24 2011-05-18 索尼公司 Image information processing device, image pick-up device, image information processing method, and program
CN102387371A (en) * 2010-08-31 2012-03-21 索尼公司 Image adjustment

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4749139B2 (en) * 2005-12-05 2011-08-17 株式会社日立製作所 Dangerous video detection method, video difference detection method and apparatus
US20100073362A1 (en) * 2008-09-23 2010-03-25 Ike Ikizyan Method And System For Scene Adaptive Dynamic 3-D Color Management
CN102360490B (en) * 2011-09-30 2012-11-28 北京航空航天大学 Color conversion and editing propagation-based method for enhancing seasonal feature of image
WO2013080739A1 (en) * 2011-12-02 2013-06-06 ソニー株式会社 Image processing device, image recognition device, image recognition method, and program
JP2014110020A (en) * 2012-12-04 2014-06-12 Samsung R&D Institute Japan Co Ltd Image processor, image processing method and image processing program
CN104299252B (en) * 2014-10-17 2018-09-07 惠州Tcl移动通信有限公司 A kind of transition method and its system of picture display switching

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040189878A1 (en) * 2003-03-25 2004-09-30 Yamaha Corporation Apparatus and program for setting video processing parameters
US20080019610A1 (en) * 2004-03-17 2008-01-24 Kenji Matsuzaka Image processing device and image processing method
CN101594471A (en) * 2008-05-30 2009-12-02 新奥特(北京)视频技术有限公司 A kind of method for changing scenes
CN102067616A (en) * 2009-04-24 2011-05-18 索尼公司 Image information processing device, image pick-up device, image information processing method, and program
CN102387371A (en) * 2010-08-31 2012-03-21 索尼公司 Image adjustment

Also Published As

Publication number Publication date
CN108335292A (en) 2018-07-27
CN108335292B (en) 2021-07-30
CN105184773B (en) 2018-05-01

Similar Documents

Publication Publication Date Title
JP5896497B2 (en) Method and wireless handheld device for determining the hue of an image
JP2008109669A (en) Display device and method of improving flicker of image
EP2871560A1 (en) Image processing method and apparatus
CN105574813B (en) A kind of image processing method and device
US20050231741A1 (en) Apparatus, medium, and method for correcting color of an image
CN114071223A (en) Optical flow-based video interpolation frame generation method, storage medium and terminal equipment
US20140125836A1 (en) Robust selection and weighting for gray patch automatic white balancing
CN105827988A (en) Light control method and device in mobile terminal shooting
CN105955754A (en) Method and device for displaying characters of user interface
EP2595117A1 (en) Method for enabling animation during screen switching and mobile terminal
CN106385550A (en) Method and terminal for capturing screen image
CN112037160B (en) Image processing method, device and equipment
CN115022679B (en) Video processing method, device, electronic equipment and medium
CN110490838B (en) Method and device for processing boundaries of areas with different resolutions of display panel
CN114339313B (en) Frame inserting method and device and electronic equipment
US20130322748A1 (en) Method for creating thumbnail images of videos and an electronic device for display thereof
CN112419218B (en) Image processing method and device and electronic equipment
CN105184773A (en) Method and device for inserting pictures in scene switching
CN108876866B (en) Media data processing method, device and storage medium
US20170310872A1 (en) Image dynamic range adjustment method, terminal, and storage medium
CN104954892A (en) Method and device for showing video subject content
WO2022142732A1 (en) Frame rate switching control method and apparatus, terminal device, and readable storage medium
CN116546176A (en) Color correction method, related device, equipment and storage medium
CN104021579A (en) Method and device for changing colors of image
KR102215607B1 (en) Electronic device capable of correction to improve the brightness of dark images and operating method thereof

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20180501