JP7018752B2 - Video processing equipment and programs - Google Patents

Description

The present invention relates to a video processing apparatus and a program, and more particularly to a technique for reducing a visual burden when viewing a video by processing the video.

Conventionally, an image including blinking or shaking may be observed in a television program, or an image including a large parallax, a change in parallax, a change in depth, or a cut may be observed on a 3D television. It has been pointed out that the viewer feels tired and uncomfortable due to such images.

Among these problems, a technique for reducing a sense of discomfort caused by blinking of an image is disclosed (see, for example, Patent Document 1). In this technique, when an image contains flicker in a predetermined frequency range, image processing is performed on the image to suppress the degree of flicker.

Further, a technique for inserting a black fade (hereinafter referred to as black fade insertion) is known in order to reduce a sense of discomfort when viewing a 3D television. This technology, for example, in a video game machine that provides 3D video content, fades out to a black screen and then fades in to 3D video from a different viewpoint when a video with a different viewpoint changes in time. be.

On the other hand, a technique for converting the expression form of the content according to the viewer or the viewing environment is disclosed (see, for example, Patent Document 2). In the receiving device, the receiving device receives the content, the alternative presentation content and the metadata transmitted from the transmitting device, and compares the metadata with the viewing profile and the viewing environment profile to derive the alternative presentation content and provide the content. It transforms the expression form of.

Japanese Patent No. 4580347 Japanese Patent No. 4580345

The degree of discomfort caused by the image such as blinking or switching of the viewpoint is not absolute, and varies depending on the age or constitution of the viewer, the viewing environment, the content of the image, and the like. This also applies to the degree of visual burden caused by the image.

The viewing environment is, for example, viewing distance, ambient brightness, screen size, screen brightness, and whether or not it corresponds to 3D video content. The content of the video is the magnitude of movement, the frequency of video transitions, the presence or absence of subtitles, and the like.

For example, blinking images is more visually burdensome for children, but not so much for adults. By using the above-mentioned technique of Patent Document 1, since image processing is uniformly performed on an image that deviates from a predetermined standard, the degree of visual burden can be reduced. However, depending on the age of the viewer and the like, the image processed image may become a flicker image more than necessary, and the dynamic feeling of the image may be lost.

In addition, some video game machines that support 3D video content have a function of adjusting parallax. This video game machine adjusts the parallax and generates an image (for example, a two-dimensional image without parallax) in which the parallax is suppressed when the images having different viewpoints change in time.

By using such a parallax adjustment function, it is possible to solve the problem when images with different viewpoints change in time without using the black fade insertion function. That is, it is assumed that by suppressing the parallax of the image by the parallax adjustment function, it is possible to reduce the discomfort that the viewer receives when the cut-edited image changes in time.

Further, in order to reduce the degree of visual burden caused by the image, it is assumed that the above-mentioned technique of Patent Document 2 is used. This technology aims to provide content that reflects the intention of the creator while responding to the viewer or the viewing environment.

In this technique, in order to reflect the intention of the creator, it is necessary to transmit the alternative presentation content and the metadata from the transmitting device to the receiving device in advance. The receiving device derives the alternative presentation content and converts the expression form of the content by comparing the received metadata with the preset viewing profile and viewing environment profile. As a result, the original image can be processed into an image according to the viewer and the viewing environment, and the visual burden on the viewer can be reduced.

However, in the technique of Patent Document 2, in order to determine the presence / absence of video processing, the processing content, and the like on the receiving device, the transmitting device needs to generate and transmit alternative presentation content and metadata. Therefore, the transmitting device has a problem that the processing load is high. Further, the receiving device has a problem that it is indispensable to use alternative presentation contents and metadata in order to determine the presence / absence of video processing, the processing contents, and the like, and the processing load is high.

Further, in the technique of Patent Document 2, the processing of the video is performed within the range of the alternative presentation content that reflects the intention of the creator, and depending on the alternative presentation content, the viewer and the viewing environment are sufficiently satisfied. It may not be possible to reflect on. As described above, the technique of Patent Document 2 has a problem that the processing of the video is limited by the alternative presentation content, and the visual burden caused by the video may not be sufficiently reduced.

Therefore, the present invention has been made to solve the above-mentioned problems, and an object thereof is an image processing device capable of reliably reducing the visual burden on a viewer who observes an image with a simple configuration. To provide a program.

In order to solve the above-mentioned problems, the video processing apparatus according to claim 1 is a video processing apparatus that inputs and processes a video signal and generates a processed video signal , and displays the processed video signal. A profile in which information about a viewer viewing a video of a device, information about the display device on which the processed video signal is displayed, or information about the viewer and information about the display device are accumulated as a profile. The storage unit, the video analysis unit that analyzes the video signal and generates the analysis result showing the characteristics of the change in the video, and the control signal including the auxiliary data that assists in processing the video signal are input and the control signal is input. The control signal input unit that extracts the auxiliary data from the control signal, the analysis result generated by the video analysis unit , the profile stored in the profile storage unit , and the extraction extracted by the control signal input unit. An image that processes the video signal according to the control unit that determines the processing content of the video signal based on the auxiliary data and the processing content determined by the control unit, and generates the processed video signal. It is characterized by having a processed part.

Further, the video processing device according to claim 2 is a video processing device that inputs and processes a video signal and generates a processed video signal, and observes a video of a display device on which the processed video signal is displayed. Information about the viewer, information about the display device on which the processed video signal is displayed, or a profile storage unit in which information about the viewer and information about the display device are stored as a profile, and the video signal. Based on the image analysis unit that analyzes the image and generates the analysis result showing the characteristics of the change in the image, the analysis result generated by the image analysis unit, and the profile accumulated in the profile storage unit. The control unit for determining the processing content of the signal and the image processing unit for processing the video signal and generating the processed video signal according to the processing content determined by the control unit are provided. The video signal is a signal of 3D video content, and the profile storage unit stores a profile of a 3D compatible flag indicating whether or not the display device supports the 3D video content, and the video analysis unit stores the profile. , A cut point is detected from the pixel value string of the video signal, a cut flag indicating the presence or absence of the cut point is generated, the control unit generates a profile of the 3D compatible flag stored in the profile storage unit, and the profile of the 3D compatible flag is the 3D. When it indicates that it corresponds to the video content and the cut flag generated by the video analysis unit indicates that there is a cut point, it fades out based on the cut point of the video signal and a black screen is displayed. After shifting to, the black fade insertion that fades in from the black screen is determined as the processing content, and the video processing unit determines the processing content determined by the control unit with respect to the video signal. It is characterized in that a black fade insertion process is performed to generate a video signal after the process.

Further, in the video processing device according to claim 3 , the video processing device according to claim 3 uses the video signal as a signal of 3D video content, and the display device corresponds to the 3D video content in the profile storage unit. The profile of the 3D correspondence flag indicating whether or not the video signal is set is accumulated, and the control signal input unit has an edit point information indicating whether or not the video signal has an edit point and the edit point is a cut point. The edit point information is extracted from the control signal by inputting the control signal including When the edit point information extracted by the control signal input unit indicates that there is a cut point, the video signal is faded out based on the cut point to shift to a black screen, and then the screen is displayed. The black fade insertion that fades in from the black screen is determined as the processing content, and the video processing unit processes the black fade insertion with respect to the video signal according to the processing content determined by the control unit. It is characterized in that the video signal after the processing is generated.

Further, the program of claim 4 is characterized in that the computer functions as the video processing apparatus according to any one of claims 1 to 3 .

As described above, according to the present invention, it is possible to surely reduce the visual burden on the viewer who views the image with a simple configuration.

It is a block diagram which shows the structural example of the image processing apparatus by embodiment of this invention. It is a flowchart which shows the processing example of a control part. It is a figure explaining the example which performs the image processing of time direction smoothing on the image including blinking in the case of the image processing mode M = 1. It is a figure explaining an example which performs the image processing of black fade insertion for the image including the scene change by a cut in the case of the image processing mode M = 2. It is a block diagram which shows the other 1st structural example in embodiment of this invention. It is a block diagram which shows the other 2nd structural example in embodiment of this invention.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. The image processing apparatus of the present invention generates an analysis result showing the characteristics of changes in the image from the image signal, and determines the processing content of the image based on the analysis result, the information of the viewer, and the information of the viewing environment. It is characterized by.

As a result, the video processing device does not need to use content data (alternatively presented content in the above-mentioned Patent Document 2) and metadata that reflect the intention of the creator of the video signal when determining the processing content of the video. .. Therefore, since it is not necessary to generate or send / receive such content data and metadata, the processing load can be reduced.

In addition, the processing content of the video is not determined within the range of content data and metadata that reflects the intention of the creator of the video signal, and the analysis result of the video signal, the information of the viewer, and the viewing environment. Determined based on information. As a result, the processed content of the video reflects the information of the viewer and the information of the viewing environment rather than the intention of the creator of the video signal. Therefore, it is possible to reliably reduce the visual burden on the viewer who views the image with a simple configuration.

[Video processing equipment]
Next, the image processing apparatus according to the embodiment of the present invention will be described. FIG. 1 is a block diagram showing a configuration example of a video processing apparatus according to an embodiment of the present invention. The video processing device 1 includes a video receiving unit (video signal input unit) 11, a data receiving unit (control signal input unit) 12, a video analysis unit 13, a profile storage unit 14, a control unit 15, and a video processing unit 16. There is.

The video receiving unit 11 receives a video signal transmitted by a transmission device (not shown) and converts the video signal into a spatiotemporal pixel value sequence I. Then, the video receiving unit 11 outputs the pixel value sequence I of the video signal to the video analysis unit 13 and the video processing unit 16.

The functions of the video receiving unit 11 include demodulation of the video signal and decoding of the compressed bit stream. Further, the video receiving unit 11 may receive a time-series two-dimensional image as a video signal of a moving image, or may receive video signals of moving images (multicular stereoscopic video) having different viewpoints from a plurality of viewpoints. You may do it. Further, the video receiving unit 11 may read the video signal from a storage medium (not shown) in which the video signal is stored. Hereinafter, the pixel value sequence I output by the video receiving unit 11 is defined as I (t, x, y) as the time t and the image coordinates (x, y).

The data receiving unit 12 receives data transmitted or stored in addition to the video signal and the audio signal as a control signal, extracts data (auxiliary data) from the control signal, and outputs the extracted data to the control unit 15.

The control signal includes auxiliary data that assists in processing the video signal. For example, auxiliary data includes video editing points (types of video effects such as scene change times, cuts, and fades), numerical data that quantifies the risk of video sickness or light-sensitive attacks, and content genres (news programs, sports programs). , Types of entertainment programs, educational programs, etc.), rights information (copyright holder, availability of video processing, type of processing when processing is permitted), viewing target age, etc.

Here, the data receiving unit 12 extracts the right information indicating whether or not the video processing is possible from the control signal, and uses this as the processing possibility information R. Further, the data receiving unit 12 indicates from the control signal information indicating whether or not the current video signal has an edit point, and if the edit point exists, whether or not the edit point is a cut point. Information is extracted, and this is referred to as edit point information E. The data receiving unit 12 outputs the data of the processability information R and the edit point information E to the control unit 15.

The data receiving unit 12 transmits the control signal to an independent PES (Packetized Elementary Stream), DSM-CC (Digital Storage Media Command and Control) section format in the data broadcasting system, an event message transmission system, or an EPG (Electronic Programming Guide: Electronic). The program guide) may be received as a route. Further, the data receiving unit 12 may receive the control signal via a communication line such as the Internet. Further, the data receiving unit 12 may read the control signal from a storage medium (not shown) in which the control signal is stored.

The video analysis unit 13 inputs the pixel value sequence I of the video signal from the video reception unit 11, analyzes the pixel value sequence I of the video signal, and features and spatially changes with time when the video is viewed as a whole. Generates analysis results showing the features of temporal and spatial changes, or the features of temporal and spatial changes. Then, the video analysis unit 13 outputs the analysis result to the control unit 15.

For example, the video analysis unit 13 detects a cut point from the time-series pixel values in the pixel value sequence I, and generates a flag indicating the presence or absence of the cut point as an analysis result.

Specifically, as shown in the following equation, the image analysis unit 13 uses the pixel value at a certain time and the pixel value at another time (for example, one frame before) to obtain the image coordinates (x) which are the pixel positions. , Y) Calculate the absolute value difference for each, and obtain the total S of the absolute value difference in the screen.

Then, when the total sum S of the absolute value differences exceeds a predetermined threshold value θ (θ is a real number of 0 or more), the video analysis unit 13 sets a cut detection value (cut flag) C = 1 indicating that there is a cut point. do. Further, when the total sum S of the absolute value differences does not exceed the predetermined threshold value θ, the video analysis unit 13 sets the cut detection value C = 0 indicating that there is no cut point.

Further, for example, the video analysis unit 13 detects video flicker (repetitive brightness change) from the time-series pixel values in the pixel value sequence I, and generates a flag indicating the presence or absence of flicker as the analysis result. .. In this case, the video analysis unit 13 may generate a detection value indicating the degree of flicker as an analysis result.

Specifically, the image analysis unit 13 calculates a luminance change value from the pixel value sequence I for each image coordinate (x, y) which is a pixel position, and when the luminance change value exceeds a predetermined value (for example, luminance). The frequency of change (when the change value exceeds 10IRE (Institute of Radio Engineers)) is calculated. Then, the image analysis unit 13 obtains the area of pixels (blinking pixels) whose frequency exceeds a predetermined value (for example, the frequency exceeds 3 times per second), and the area ratio in the image (relative to the total number of pixels in the frame). (Percentage of blinking pixels) is calculated.

The image analysis unit 13 sets the flicker detection value F = 0 when the area ratio is 25% or less, and sets the area ratio to the flicker detection value F when the area ratio exceeds 25%. Further, the video analysis unit 13 sets the flicker flag = 0 when the area ratio is 25% or less, and sets the flicker flag = 1 when the area ratio exceeds 25%.

Further, for example, when the video receiving unit 11 receives a video signal of a multi-lens stereoscopic video (signal of 3D video content), the video analysis unit 13 detects a time change of the parallax from the pixel value sequence I for each viewpoint. This is generated as an analysis result.

Specifically, the image analysis unit 13 obtains parallax based on the difference between the pixel value of the entire screen viewed from the left eye and the pixel value of the entire screen viewed from the right eye at a certain time, and another parallax is obtained. At the time, the parallax is calculated in the same way. Then, the image analysis unit 13 obtains the amount of change in the parallax from the parallax at a certain time and the parallax at another time. When the amount of change in parallax exceeds a predetermined threshold value, the image analysis unit 13 sets a cut detection value C = 1 indicating that there is a cut point. Further, the image analysis unit 13 sets a cut detection value C = 0 indicating that there is no cut point when the amount of change in parallax does not exceed a predetermined threshold value.

Further, for example, the video analysis unit 13 detects a color change point from the time-series pixel values in the pixel value sequence I, and generates a flag indicating the presence or absence of the color change point as an analysis result.

Here, the video analysis unit 13 detects a cut point from the pixel value sequence I, generates a cut detection value C indicating the presence or absence of the cut point as an analysis result, and detects a flicker from the pixel value sequence I to detect the flicker. A flicker detection value F indicating the degree is generated as an analysis result. Then, the video analysis unit 13 outputs the analysis results of the cut detection value C and the flicker detection value F to the control unit 15.

The profile storage unit 14 stores viewer information individually set for each viewer in advance and terminal information set for each unit (or model) of the video processing device 1 as a profile. The profile storage unit 14 outputs the profile of the viewer information and the terminal information to the control unit 15.

The viewer information includes the viewer's age, viewing distance, preference (favorite program genre, etc.), presence / absence of risk of photosensitive epilepsy, presence / absence of risk of video sickness, and the like.

The terminal information includes whether or not the video processing device 1 supports 3D video content (whether or not it is a 3D television), the type of the video processing device 1 (LCD (Liquid Crystal Display), and OLED (Organic). Light Emitting Diode, PDP (Plasma Display Panel), CRT (Cathode Ray Tube), etc.), maximum brightness of display device, screen size, etc.

The profile storage unit 14 may store only the profile of the viewer information, or may store only the profile of the terminal information.

Here, the profile storage unit 14 indicates the viewer information of the photosensitive epilepsy risk flag P indicating whether or not the viewer is at risk of photosensitive epilepsy, and whether the video processing device 1 corresponds to 3D video content. The terminal information of the 3D correspondence flag D indicating whether or not it is output is output to the control unit 15 as a profile.

The control unit 15 inputs data from the data receiving unit 12, inputs the analysis result from the video analysis unit 13, and further inputs the profile from the profile storage unit 14. Then, the control unit 15 determines the processing content of the video to be applied to the video signal received by the video receiving unit 11 based on the data, the analysis result, and the profile, and the video processing unit 16 determines the control value indicating the processing content of the video. Output to. For example, the control unit 15 determines the processing content of the image according to the if-then rule.

Here, the control unit 15 inputs the data of the processability information R and the edit point information E from the data receiving unit 12, and inputs the analysis results of the cut detection value C and the flicker detection value F from the video analysis unit 13. Further, the control unit 15 inputs the profiles of the photosensitive epilepsy risk flag P and the 3D-compatible flag D from the profile storage unit 14.

The control unit 15 determines the processing content of the image based on the processing possibility information R, the editing point information E, the cut detection value C, the flicker detection value F, the photosensitive epilepsy risk flag P, and the 3D correspondence flag D. Then, the control unit 15 outputs the control value of the image processing mode M indicating the processing content of the image to the image processing unit 16.

(Data of processability information R and edit point information E input from the data receiving unit 12)
The processability information R is information included in the control signal received by the data receiving unit 12, and is information sent from the outside. The edit point information E indicates whether or not the information is sent from the outside as the control signal received by the data receiving unit 12, and if the information is sent from the outside, the edited content is included. ..

The processability information R = 0 indicates that image processing other than for the purpose of preventing photosensitive epilepsy is prohibited, and the processability information R = 1 indicates that image processing is permitted. Further, the edit point information E = 0 indicates that the edit point information has not been sent. Further, the edit point information E = 1 indicates that the edit point information has been sent, indicating that the current image is not a cut point, and the edit point information E = 2 indicates that the edit point information has been sent, and the present time. Indicates that the image of is a cut point.

(Analysis result of cut detection value C and flicker detection value F input from video analysis unit 13)
The cut detection value C and the flicker detection value F are analysis results generated from the pixel value sequence I of the video signal by the video analysis unit 13. A cut detection value C = 0 indicates that the current image is not a cut point, and a cut detection value C = 1 indicates that the current image is a cut point. Further, when the flicker detection value F = 0, flicker does not occur in the entire screen area, or flicker occurs in the area of the screen area of 25% or less, and flicker occurs as a whole in the current image. Indicates that there is no such thing. A flicker detection value F> 0 indicates that flicker is generated in the area of the screen area of F% exceeding 25%, and flicker is generated as a whole in the current image.

(Profile of Photosensitive Epilepsy Risk Flag P and 3D Corresponding Flag D Input from Profile Storage Unit 14)
The photosensitive epilepsy risk flag P is viewer information input from the profile storage unit 14, and the 3D-compatible flag D is terminal information input from the profile storage unit 14. Photosensitive epilepsy risk flag P = 0 indicates that the viewer is not at risk for photosensitive epilepsy, and photosensitive epilepsy risk flag P = 1 indicates that the viewer is at risk for photosensitive epilepsy. Further, the 3D compatible flag D = 0 indicates that the video processing device 1 does not support 3D video content, and the 3D compatible flag D = 1 indicates that the video processing device 1 supports 3D video content. show.

(Control value of the image processing mode M output by the control unit 15)
The image processing mode M = 0 indicates that no image processing is performed, and the image processing mode M = 1 indicates that time-direction smoothing is performed as image processing. The image processing mode M = 2 indicates that black fade insertion is performed as image processing, and the image processing mode M = 3 indicates that time direction smoothing and black fade insertion are performed as image processing.

FIG. 2 is a flowchart showing a processing example of the control unit 15. The control unit 15 receives the data of the processability information R and the edit point information E from the data receiving unit 12, the analysis results of the cut detection value C and the flicker detection value F from the video analysis unit 13, and the photosensitivity attack from the profile storage unit 14. The profiles of the risk flag P and the 3D correspondence flag D are input respectively (step S201).

The control unit 15 initially sets the image processing mode M = 0 (no image processing is performed) (step S202), and determines whether or not the photosensitive epilepsy risk flag P = 0 (no risk of photosensitive epilepsy). Determination (step S203).

When the control unit 15 determines in step S203 that the photosensitive epilepsy risk flag P = 0 is not (step S203: N), the control unit 15 determines that the viewer is at risk of photosensitive epilepsy, and proceeds to step S204.

On the other hand, when the control unit 15 determines in step S203 that the photosensitive epilepsy risk flag P = 0 (step S203: Y), the control unit 15 determines that the viewer has no risk of photosensitive epilepsy, and proceeds to step S206. Transition.

The control unit 15 shifts from step S203 (N) and determines whether or not the flicker detection value F ≦ 25 (flicker is generated in a region of 25% or less of the screen area) (step S204).

When the control unit 15 determines in step S204 that the flicker detection value F ≦ 25 is not satisfied (step S204: N), the control unit 15 determines that flicker is generated in a region exceeding 25% of the screen area in the current image. Then, the control unit 15 adds 1 to the image processing mode M = 0, sets the image processing mode M = 1 (performing time-direction smoothing as image processing) (step S205), and proceeds to step S206.

On the other hand, when the control unit 15 determines in step S204 that the flicker detection value F ≦ 25 (step S204: Y), it is determined that flicker is generated in an area of 25% or less of the screen area in the current image. The determination is made, and the process proceeds to step S206.

The control unit 15 shifts from step S203 (Y), step S204 (Y) or step S205, and determines whether or not the 3D correspondence flag D = 0 (the video processing device 1 does not support 3D video content). (Step S206).

When the control unit 15 determines in step S206 that the 3D correspondence flag D = 0 is not (step S206: N), the control unit 15 determines that the video processing device 1 corresponds to the 3D video content, and proceeds to step S207.

On the other hand, when the control unit 15 determines in step S206 that the 3D correspondence flag D = 0 (step S206: Y), the control unit 15 determines that the video processing device 1 does not correspond to the 3D video content, and proceeds to step S212. Transition.

The control unit 15 shifts from step S206 (N) and determines whether or not the processing possibility information R = 0 (video processing other than the purpose of preventing photosensitive epilepsy is prohibited) (step S207).

When the control unit 15 determines in step S207 that the processing possibility information R = 0 (step S207: N), it determines that video processing is permitted, and proceeds to step S208.

On the other hand, when the control unit 15 determines in step S207 that the processing enablement information R = 0 (step S207: Y), it determines that video processing other than for the purpose of preventing photosensitive epilepsy is prohibited, and in step S212. Move to.

The control unit 15 shifts from step S207 (N) and determines whether or not the edit point information E = 0 (edit point information has not been sent) (step S208).

When the control unit 15 determines in step S208 that the edit point information E = 0 (step S208: N), it determines that the edit point information has been sent, and proceeds to step S209.

On the other hand, when the control unit 15 determines in step S208 that the edit point information E = 0 (step S208: Y), it determines that the edit point information has not been sent, and proceeds to step S210.

The control unit 15 shifts from step S208 (N) and determines whether or not the edit point information E = 1 (the current image is not a cut point) (step S209).

When the control unit 15 determines in step S209 that the edit point information E = 1 (step S209: N), the control unit 15 determines that the current image is a cut point, and proceeds to step S211.

On the other hand, when the control unit 15 determines in step S209 that the edit point information E = 1 (step S209: Y), it determines that the current image is not a cut point, and proceeds to step S212.

The control unit 15 shifts from step S208 (Y) and determines whether or not the cut detection value C = 0 (the current image is not the cut point) (step S210).

When the control unit 15 determines in step S210 that the cut detection value C = 0 is not (step S210: N), the control unit 15 determines that the current image is the cut point, and proceeds to step S211.

On the other hand, when the control unit 15 determines in step S210 that the cut detection value C = 0 (step S210: Y), it determines that the current image is not the cut point, and proceeds to step S212.

The control unit 15 shifts from step S209 (N) or step S210 (N) and adds 2 to the video processing mode M = 0,1. Then, the control unit 15 sets the image processing mode M = 2 (performs black fade insertion as image processing) and 3 (performs time direction smoothing and black fade insertion as image processing) (step S211), and steps S212. Move to.

The control unit 15 shifts from step S206 (Y), step S207 (Y), step S209 (Y), step S210 (Y) or step S211 and outputs the image processing mode M to the image processing unit 16 (step). S212).

In this way, when a viewer who may cause photosensitive epilepsy is watching the image and the flicker detection value F generated by the image analysis unit 13 exceeds a predetermined threshold value (25). Is set to 1 in the least significant bit of the image processing mode M in step S205 so as to perform time-direction smoothing as image processing.

Further, when the video processing device 1 supports 3D video content and video processing is permitted, black is used to suppress a sense of discomfort (warp feeling) caused by moving the viewpoint with respect to the cut point of the video. Insert a fade.

In order to realize this, when the edit point information by the data receiving unit 12 is obtained, the presence or absence of the cut point is determined based on the edit point information. On the other hand, when the edit point information cannot be obtained by the data receiving unit 12, the presence or absence of the cut point is determined based on the analysis result generated by the video analysis unit 13. Then, when it is determined to be a cut point, 1 is set in the second lowest bit of the image processing mode M in step S211 so as to insert a black fade as image processing.

As a result, in the lower 2 bits of the image processing mode M output to the image processing unit 16, information on whether or not to perform time-direction smoothing as image processing and whether or not to insert a black fade as image processing are performed. Information is set. That is, in the image processing mode M, no image processing is performed, time-direction smoothing is performed as image processing, black fade insertion is performed as image processing, and time-direction smoothing and black fade insertion are performed as image processing. Contains any of the information in.

In the above embodiment, the control unit 15 determines the processing content of the image according to the if-then rule shown in FIG. 2, but this is an example, and other rules may be used. May be. For example, the control unit 15 may determine the processing content of the video by using the machine-learned learning model. As an example of the training model, the data of the processability information R and the edit point information E, the analysis result of the cut detection value C and the flicker detection value F, and the profile of the photosensitivity attack risk flag P and the 3D correspondence flag D are used as input data. A neural network, a support vector machine, a decision tree, a random forest, or the like whose output data is the control value of the image processing mode M is used.

Returning to FIG. 1, the video processing unit 16 inputs the pixel value sequence I of the video signal from the video receiving unit 11, and inputs the control value indicating the processing content of the video from the control unit 15. Then, the image processing unit 16 performs image processing (or no image processing) on the pixel value sequence I of the video signal according to the control value, and outputs the processing result (or the pixel value sequence I) to the video signal J. And outputs the video signal J.

The image processing may be processing for the pixel value sequence I (current frame) of only the current t only, or may be processing for the pixel value sequence I (future frame) at the future time point. Further, the image processing may be processing for the pixel value sequence I (past frame) at the past time point. In this case, the image processing unit 16 performs buffering to store the pixel value sequence I in the buffer, processes the pixel value sequence I stored in the buffer, then performs time delay processing on the processing result, and obtains the video signal J. Output.

The video processing unit 16 outputs the video signal J to a display device (including a display device such as a head-mounted display when the video processing device 1 supports 3D video content in addition to the TV screen). You may. As a result, the video signal J is displayed on the display device. Further, the video processing unit 16 may output the video signal J to a device such as a recording device or a video switching machine.

Here, the video processing unit 16 inputs the pixel value sequence I of the video signal from the video receiving unit 11, and also inputs the control value of the video processing mode M from the control unit 15. Then, the image processing unit 16 performs image processing (or no image processing) on the pixel value sequence I of the video signal according to the image processing mode M, and displays the processing result (or pixel value sequence I) as an image. It is output as a signal J.

Specifically, when the video processing mode M = 0, the video processing unit 16 does not perform video processing on the pixel value sequence I of the video signal, and the pixel value of the input video signal is expressed by the following formula. The column I is output as a video signal J.

Ｎは、平滑化に用いる時点数（フレーム数）であり、２以上の整数とする。例えば、毎秒６０フレームの映像信号に対してＮ＝２０とすると、映像加工部１６は、過去１／３秒分の映像信号の画素値列Ｉに対する相加平均を算出し、算出結果を映像信号Ｊとして出力する。 Further, when the video processing mode M = 1, the video processing unit 16 performs video processing for time-direction smoothing on the pixel value sequence I of the video signal by the following formula to generate the video signal J. And output.

N is the number of time points (number of frames) used for smoothing, and is an integer of 2 or more. For example, assuming that N = 20 for a video signal of 60 frames per second, the video processing unit 16 calculates the arithmetic mean of the video signal for the past 1/3 second with respect to the pixel value sequence I, and the calculation result is the video signal. Output as J.

Further, when the image processing mode M = 2, the image processing unit 16 performs image processing for inserting a black fade into the pixel value sequence I of the image signal, and generates and outputs the image signal J. That is, the image processing unit 16 performs image processing so as to fade out and shift to the black screen based on the cut point of the image signal, and then fade in from the black screen.

例えばＵ＝－３０、Ｖ＝＋３０とする。 For example, the video processing unit 16 uses the following equation for the integer U of U <-1 and the integer V of V> + 1 in the time range of time t + U + 1 to t + V-1 extended before and after including the current time t. Then, the black fade is inserted, and the video signal J is generated and output.

For example, U = -30 and V = +30.

Further, when the image processing mode M = 3, the image processing unit 16 performs image processing for time-direction smoothing and black fade insertion on the pixel value sequence I of the image signal by the following equation, and the image signal. Generate and output J.

FIG. 3 is a diagram illustrating an example of performing time-direction smoothing video processing on a video including blinking when the video processing mode M = 1. It is assumed that the image processing mode M = 1 is set by the control unit 15 when the frames a1 to a5 in which the image signal before processing violently blinks.

The image processing unit 16 determines the image processing mode M = 1, and performs image processing for time-direction smoothing on the pixel value strings I of the frames a1 to a5, which are the image signals before processing, according to the above equation (3). conduct. Then, the image processing unit 16 generates the image signals J of the frames b1 to b5 smoothed in the time direction as the processed image signals.

FIG. 4 is a diagram illustrating an example in which black fade insertion is performed on an image including a scene change due to a cut when the image processing mode M = 2. It is assumed that the video processing mode M = 2 is set by the control unit 15 when the video signal before processing is the frames a1 to a5 having the frame a3 as the cut point α.

The image processing unit 16 determines the image processing mode M = 2, and performs image processing by inserting a black fade into the pixel value strings I of the frames a1 to a5, which are the image signals before processing, according to the above equation (4). .. Then, the image processing unit 16 generates the image signals J of the frames b1 to b5 in which the frame b3 near the cut point α is made black and the black fades are inserted before and after the frame b3 as the processed image signal.

As described above, according to the video processing apparatus 1 of the embodiment of the present invention, the data receiving unit 12 receives the control signal, extracts auxiliary data from the control signal when processing the video signal, and obtains data. The video analysis unit 13 analyzes the pixel value sequence I of the video signal and generates an analysis result showing the characteristics of the change in the video.

The control unit 15 inputs the profile of the viewer information and the terminal information from the profile storage unit 14, the data extracted by the data reception unit 12, the analysis result analyzed by the video analysis unit 13, and the profile storage unit 14. Based on the profile input from, the processing content of the video is determined, and the control value indicating the processing content of the video is set. Then, the video processing unit 16 processes the video signal according to the control value set by the control unit 15.

Thereby, for example, the video signal can be processed without using the content data (alternatively presented content in the above-mentioned Patent Document 2) and the metadata transmitted from the transmission device and reflecting the intention of the creator of the video signal. , The processing load can be reduced. Further, since it is not necessary to generate the content data in consideration of the conditions depending on the viewer and the viewing environment in the transmitting device and it is not necessary to generate the metadata, the processing load of the transmitting device can be reduced.

Further, the processing content of the video is not determined within the range of the content data and the metadata that reflect the intention of the creator of the video signal, and the analysis result of the video signal, the control signal, the information of the viewer, and the view are not determined. Since the content is determined based on the information on the visual environment, the content reflects the viewer and the visual environment rather than the intention of the creator of the video signal. Therefore, the visual burden on the viewer who views the image can be reliably reduced by a simple configuration, and the fatigue of the viewer can also be reduced.

Furthermore, since the video signal can be appropriately processed while considering the creator's intention, the viewer, and the viewing environment reflected in the analysis result of the video signal and the control signal, it is possible to produce an effect on the video production. It is possible to view the image in the optimum state while maintaining the balance of the safety of the viewer.

[Other Configuration Example 1]
Next, another configuration example 1 (first configuration example) will be described with respect to the embodiment of the present invention. FIG. 5 is a block diagram showing another first configuration example according to the embodiment of the present invention. The video processing device 2 includes a video receiving unit 11, a video analysis unit 13, a profile storage unit 14, a control unit 17, and a video processing unit 16.

Comparing the video processing device 1 shown in FIG. 1 with the video processing device 2, both video processing devices 1 and 2 include a video receiving unit 11, a video analysis unit 13, a profile storage unit 14, and a video processing unit 16. It is common in that. On the other hand, the video processing device 2 is different from the video processing device 1 in that it does not include the data receiving unit 12 and includes a control unit 17 different from the control unit 15 of the video processing device 1.

Since the video receiving unit 11, the video analysis unit 13, the profile storage unit 14, and the video processing unit 16 have already been described with reference to FIG. 1, the description thereof will be omitted here.

The control unit 17 inputs the analysis result from the video analysis unit 13, and also inputs the profile from the profile storage unit 14. Then, the control unit 17 determines the processing content of the video based on the analysis result and the profile, and outputs the control value indicating the processing content of the video to the video processing unit 16.

For example, the control unit 17 inputs the cut detection value C and the flicker detection value F as analysis results from the video analysis unit 13, and inputs the profiles of the photosensitive epilepsy risk flag P and the 3D compatible flag D from the profile storage unit 14. .. Then, the control unit 17 determines the processing content of the image based on the cut detection value C, the flicker detection value F, the photosensitive epilepsy risk flag P, and the 3D correspondence flag D, and processes the control value of the image processing mode M into the image. Output to unit 16.

Specifically, the control unit 17 performs the processes of steps S201 to S206 and steps S210 to S212 excluding steps S207 to S209 among all the steps shown in the processing example of FIG.

With reference to FIG. 2, in step S201, the control unit 17 inputs the analysis results of the cut detection value C and the flicker detection value F from the video analysis unit 13, and also inputs the photosensitive epilepsy risk flag P and the photosensitive epilepsy risk flag P from the profile storage unit 14. Enter the profile of the 3D compatible flag D.

When the control unit 17 determines in step S206 that the 3D correspondence flag D is not 0 after the processing of steps S201 to S205, the control unit 17 determines that the video processing device 2 corresponds to the 3D video content, and steps S210. Move to. Then, in step S210, the control unit 17 determines whether or not the cut detection value C = 0 (the current image is not the cut point).

If the control unit 17 determines in step S210 that the cut detection value C = 0, it determines that the current image is the cut point, and proceeds to step S211. On the other hand, when the control unit 17 determines in step S210 that the cut detection value C = 0, it determines that the current image is not the cut point, and proceeds to step S212.

As described above, according to the video processing apparatus 2 of the embodiment of the present invention, the video analysis unit 13 analyzes the pixel value sequence I of the video signal and generates an analysis result showing the characteristics of the change in the video. The control unit 17 inputs profiles of viewer information and terminal information from the profile storage unit 14, and based on the analysis results analyzed by the video analysis unit 13 and the profile input from the profile storage unit 14, the video The processing content is determined, and the control value indicating the processing content of the image is set. Then, the video processing unit 16 processes the video signal according to the control value set by the control unit 17.

As a result, similarly to the image processing device 1 shown in FIG. 1, the visual burden on the viewer who observes the image can be surely reduced by a simple configuration, and the fatigue of the viewer is also reduced. It becomes possible to do. In addition, it is possible to view the video in an optimum state in which the production effect on the video production and the safety of the viewer are balanced.

[Other Configuration Example 2]
Next, another configuration example 2 (second configuration example) will be described with respect to the embodiment of the present invention. FIG. 6 is a block diagram showing another second configuration example according to the embodiment of the present invention. The video processing device 3 includes a video receiving unit 11, a data receiving unit 12, a profile storage unit 14, a control unit 18, and a video processing unit 16.

Comparing the video processing device 1 shown in FIG. 1 with the video processing device 3, both video processing devices 1 and 3 include a video receiving unit 11, a data receiving unit 12, a profile storage unit 14, and a video processing unit 16. It is common in that. On the other hand, the video processing device 3 is different from the video processing device 1 in that the video processing device 3 does not include the video analysis unit 13 and includes a control unit 18 different from the control unit 15 of the video processing device 1.

Since the video receiving unit 11, the data receiving unit 12, the profile storage unit 14, and the video processing unit 16 have already been described with reference to FIG. 1, the description thereof will be omitted here.

The control unit 18 inputs data from the data receiving unit 12, and also inputs a profile from the profile storage unit 14. Then, the control unit 18 determines the processing content of the video based on the data and the profile, and outputs the control value indicating the processing content of the video to the video processing unit 16.

For example, the control unit 18 inputs the data of the processability information R and the edit point information E from the data reception unit 12, and inputs the profiles of the photosensitive epilepsy risk flag P and the 3D correspondence flag D from the profile storage unit 14. Then, the control unit 18 determines the processing content of the image based on the processing possibility information R, the editing point information E, the photosensitive epilepsy risk flag P, and the 3D correspondence flag D, and processes the control value of the image processing mode M into the image. Output to unit 16.

Specifically, the control unit 18 of all the steps shown in the processing example of FIG. 2 includes steps S201 to S203, steps S205 to S209, steps S211 and step S212 excluding step S204 and step S210. Perform processing.

With reference to FIG. 2, in step S201, the control unit 18 inputs the data of the processability information R and the edit point information E from the data reception unit 12, and the photosensitivity attack risk flags P and 3D are input from the profile storage unit 14. Enter the profile of the corresponding flag D.

If the control unit 18 determines in step S203 that the photosensitive epilepsy risk flag P = 0 is not obtained after the processing of steps S201 and S202, the control unit 18 determines that the viewer is at risk of photosensitive epilepsy, and steps S205. Move to. Then, in step S205, the control unit 18 adds 1 to the image processing mode M = 0 and sets the image processing mode M = 1 (time-direction smoothing is performed as image processing).

Further, when the control unit 18 determines in step S208 that the edit point information E = 0, it determines that the edit point information has not been sent, and proceeds to step S212.

The data receiving unit 12 may extract flicker information indicating the presence or absence of flicker and the degree of flicker from the control signal, and the control unit 18 may input data including flicker information from the data receiving unit 12. .. In this case, if the control unit 18 determines in step S203 that the photosensitive epilepsy risk flag P = 0 is not present, the control unit 18 determines that the viewer is at risk of photosensitive epilepsy, and based on the input flicker information data. If it is determined that the degree of flicker exceeds a predetermined value (for example, the flicker area is 25% of the total area), the process proceeds to step S205 and the image processing mode M = 1 is set.

As described above, according to the video processing apparatus 3 of the embodiment of the present invention, the data receiving unit 12 receives the control signal and extracts the auxiliary data when processing the video signal from the control signal. The control unit 18 inputs a profile of viewer information and terminal information from the profile storage unit 14, and processes an image based on the data extracted by the data reception unit 12 and the profile input from the profile storage unit 14. Determine the content and set the control value indicating the processing content of the video. Then, the video processing unit 16 processes the video signal according to the control value set by the control unit 18.

As a result, similarly to the image processing device 1 shown in FIG. 1, the visual burden on the viewer who observes the image can be surely reduced by a simple configuration, and the fatigue of the viewer is also reduced. It becomes possible to do. In addition, it is possible to view the video in an optimum state in which the production effect on the video production and the safety of the viewer are balanced.

Although the present invention has been described above with reference to embodiments, the present invention is not limited to the above-described embodiment and can be variously modified without departing from the technical idea. For example, the video processing device 1 shown in FIG. 1, the video processing device 2 shown in FIG. 5, and the video processing device 3 shown in FIG. 6 may be television receivers that receive broadcast waves of video signals. .. Further, the video processing devices 1, 2, and 3 may be video playback devices that read video signals from, for example, a storage medium.

Further, in the above embodiment, the data extracted by the data receiving unit 12 is used as the processability information R and the edit point information E, and the analysis results generated by the video analysis unit 13 are used as the cut detection value C and the flicker detection value F. The profile input from the profile storage unit 14 is set as the light-sensitive attack risk flag P and the 3D-compatible flag D, and the control values generated by the control units 15, 17 and 18 are set to the image processing mode M (no processing, time-direction smoothing). (Value indicating any of conversion, black fade insertion, temporal smoothing, and black fade insertion).

The present invention uses data extracted by the data receiving unit 12, analysis results analyzed by the video analysis unit 13, a profile input from the profile storage unit 14, and control values generated by the control units 15, 17, and 18. , The information is not limited to the above. The above information is an example.

As the hardware configuration of the video processing devices 1, 2, and 3 according to the embodiment of the present invention, a normal computer can be used. The video processing devices 1, 2, and 3 are composed of a volatile storage medium such as a CPU and RAM, a non-volatile storage medium such as a ROM, and a computer provided with an interface and the like.

Each function of the video receiving unit 11, the data receiving unit 12, the video analysis unit 13, the profile storage unit 14, the control unit 15, and the video processing unit 16 provided in the video processing device 1 uses a program describing these functions in the CPU. It is realized by executing each. Further, each function of the video receiving unit 11, the video analysis unit 13, the profile storage unit 14, the control unit 17, and the video processing unit 16 provided in the video processing device 2 also causes the CPU to execute a program describing these functions. It is realized by each. Further, each function of the video receiving unit 11, the data receiving unit 12, the profile storage unit 14, the control unit 18, and the video processing unit 16 provided in the video processing device 3 also causes the CPU to execute a program describing these functions. It is realized by each.

These programs are stored in the storage medium, read by the CPU, and executed. In addition, these programs can be stored and distributed in storage media such as magnetic disks (floppy (registered trademark) disks, hard disks, etc.), optical disks (CD-ROM, DVD, etc.), semiconductor memories, etc., and can be distributed via a network. You can also send and receive.

1, 2, 3 Video processing device 11 Video receiver (video signal input)
12 Data receiving unit (control signal input unit)
13 Video analysis unit 14 Profile storage unit 15, 17, 18 Control unit 16 Video processing unit I Pixel value sequence R Processing availability information E Editing point information C Cut detection value F Flicker detection value P Light-sensitive attack risk flag D 3D compatible flag M Video processing mode J Video signal S Total of absolute value differences a1 to a5, b1 to b5 Frame α Cut point θ Threshold

Claims (4)

In an image processing device that inputs and processes a video signal and generates a processed video signal.
Information about the viewer viewing the image of the display device on which the processed video signal is displayed, information about the display device on which the processed video signal is displayed, or information about the viewer and the above. A profile storage unit that stores information about display devices as a profile,
An image analysis unit that analyzes the image signal and generates an analysis result showing the characteristics of the change in the image.
A control signal input unit that inputs a control signal including auxiliary data that is auxiliary when processing the video signal and extracts the auxiliary data from the control signal.
The processing content of the video signal is determined based on the analysis result generated by the video analysis unit , the profile stored in the profile storage unit , and the auxiliary data extracted by the control signal input unit . Control unit and
An image processing unit that processes the video signal and generates the processed video signal according to the processing content determined by the control unit.
A video processing device characterized by being equipped with. In an image processing device that inputs and processes a video signal and generates a processed video signal.
Information about the viewer viewing the image of the display device on which the processed video signal is displayed, information about the display device on which the processed video signal is displayed, or information about the viewer and the above. A profile storage unit that stores information about display devices as a profile,
An image analysis unit that analyzes the image signal and generates an analysis result showing the characteristics of the change in the image.
A control unit that determines the processing content of the video signal based on the analysis result generated by the video analysis unit and the profile stored in the profile storage unit.
A video processing unit that processes the video signal and generates the processed video signal according to the processing content determined by the control unit is provided .
The video signal is used as a signal for 3D video content.
In the profile storage part,
A profile of a 3D compatible flag indicating whether or not the display device supports the 3D video content is accumulated.
The video analysis unit
A cut point is detected from the pixel value sequence of the video signal, and a cut flag indicating the presence or absence of the cut point is generated.
The control unit
The profile of the 3D-compatible flag stored in the profile storage unit indicates that the profile corresponds to the 3D video content, and the cut flag generated by the video analysis unit indicates that there is a cut point. In this case, the black fade insertion that fades in from the black screen after fading out and shifting to the black screen based on the cut point of the video signal is determined as the processing content.
The video processing unit
An image processing apparatus characterized in that the black fade insertion process is performed on the video signal according to the processing content determined by the control unit, and the processed video signal is generated. In the video processing apparatus according to claim 1 ,
The video signal is used as a signal for 3D video content.
In the profile storage part,
A profile of a 3D compatible flag indicating whether or not the display device supports the 3D video content is accumulated.
The control signal input unit is
The video signal has an edit point, a control signal including edit point information indicating whether or not the edit point is a cut point is input, and the edit point information is extracted from the control signal.
The control unit
The profile of the 3D-compatible flag stored in the profile storage unit indicates that the profile corresponds to the 3D video content, and the edit point information extracted by the control signal input unit indicates that there is a cut point. If so, the black fade insertion that fades in from the black screen after fading out based on the cut point of the video signal and shifting to the black screen is determined as the processing content.
The video processing unit
An image processing apparatus characterized in that the black fade insertion process is performed on the video signal according to the processing content determined by the control unit, and the processed video signal is generated. A program for causing a computer to function as the video processing device according to any one of claims 1 to 3 .

Images