JP2005222098A - Image classification device and method, program for image classification method, and recording medium recording program for image classification method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image classification device, an image classification method, a program for image classification method, and a recording medium for recording the program for image classification method, in which, for example, television programs such as dialogues are sorted and displayed, whereby classification of images provided for the sorting can be properly executed. <P>SOLUTION: Sorting for image classification is set in advance according to a priority order set to each system 12A-12D, whereby programs of television broadcasting are sorted and displayed, so that classification of images for the sorting can be properly executed. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a video classification device, a video classification method, a video classification method program, and a recording medium on which the video classification method program is recorded, and can be applied to viewing of television programs such as conversations. The present invention sets the video classification distribution according to the priorities set in advance for each system, so that, for example, television broadcast programs are distributed and displayed, and the video classification used for this distribution is appropriately set. To be able to run.

  2. Description of the Related Art Conventionally, various methods for detecting scene changes and using them for cueing have been proposed for television broadcast program processing.

  That is, for example, Japanese Patent Laid-Open No. 2000-69420 proposes a method for detecting a scene change by detecting a cut based on a change in a feature amount between frames and determining a similarity between shots divided by the cut. It is made so that. Japanese Patent Laid-Open No. 7-193748 proposes a method of detecting the appearance scene of a main caster in a news program, for example, by comparing pixel values and classifying each scene in a series of programs based on the appearance scene. It is made like that. Japanese Patent Application Laid-Open No. 7-288840 proposes a method for detecting a scene change from a color histogram.

  On the other hand, as a method for improving the sense of presence when viewing a program, and as a method for viewing a plurality of programs simultaneously, a technique such as multi-screen, picture-in-picture, etc. has been provided. Japanese Patent No. 169268 proposes a method for simplifying the control of a memory provided for a plurality of small-screen displays related to this picture-in-picture.

  By the way, for example, if monitor devices are arranged so as to surround the viewers, and the participants of the talk program are distributed and displayed on these monitor devices, the atmosphere is as if the viewers themselves are participating in the debate. It is thought that the sense of reality when watching a program can be further improved. In other words, in this case, the moving images based on the video signals sequentially input to the plurality of monitor devices are sorted and displayed, and in the monitor device that does not distribute the moving images at that time, the moving image that has been distributed immediately before is stopped. Display by drawing.

  For this purpose, it is necessary not only to detect a scene change for a discussion program or the like broadcasted on television, but to appropriately distribute a person's video imaged by the scene change to the plurality of monitor devices.

However, in the processing of a program related to television broadcasting by a conventional method, there is a problem that is still insufficient in practice when applied to this type of classification. That is, immediately after starting the trial listening of the program, no video is assigned to any monitor device. In this case, if the distribution to the monitor device is set according to an instruction from the user, the burden on the user is remarkably increased. Further, when a video is distributed to all the monitor devices and a new scene appears, in this case, it becomes difficult to distribute the new scene, and eventually it is impossible to continue listening to the program. On the other hand, when the program is switched, the still image related to the old program is continuously displayed, which makes the new program feel uncomfortable.
JP 2000-69420 A JP-A-7-193748 JP-A-7-288840 JP 2003-169268 A

  The present invention has been made in consideration of the above points. A video classification apparatus and a video classification method capable of appropriately executing a classification used for the distribution by distributing and displaying television broadcast programs and the like. A video classification method program and a recording medium on which the video classification method program is recorded are proposed.

  In order to solve such a problem, the invention according to claim 1 is applied to a video classifying apparatus that classifies videos by sequentially inputted image sequences into a plurality of systems, and classifies videos into systems that have not yet been classified. The determination criterion setting means for setting the determination criterion for classification of the subsequent video to the system and the determination criterion already set determine whether the subsequent video is similar to the video classified into which system And classifying means for classifying the subsequent video into a system for which a criterion is already set, and the classifier is difficult to classify the video into a system for which a criterion is set depending on the criterion that has already been set. In this case, the video is classified into a system in which the video is not yet classified according to the priority order set in advance for a plurality of systems.

  According to the twelfth aspect of the present invention, the present invention is applied to a video classification method for classifying videos based on sequentially input image sequences into a plurality of systems, and the video is classified into a system to which the video has not yet been classified. In accordance with the determination criteria setting step for setting the determination criteria used for the subsequent video classification, and the determination criteria already set in the system, it is determined whether the subsequent video is similar to the video classified into which system, And a classification step of classifying the subsequent video into a system in which the criterion is already set. The classification step classifies the video into a system in which the criterion is set depending on the criterion that has already been set. When it is difficult, according to the priority order set in advance for a plurality of systems, the videos are classified into systems that have not yet been classified.

  According to a thirteenth aspect of the present invention, the video is still classified by applying it to a program of a video classification method for classifying videos based on sequentially inputted image sequences into a plurality of systems by causing a computer to execute a predetermined processing procedure. Depending on the classification of the video to the non-system, the determination standard setting step for setting the determination standard used for the classification of the subsequent video to the system and the subsequent video is classified into any system by the already set determination standard. A classification step of classifying the subsequent video into a system in which the determination criteria are already set, and the classification step is determined depending on the determination criteria already set. If it is difficult to classify videos into systems for which standards are set, videos are classified into systems that have not yet been classified according to the priorities set in advance for multiple systems. .

  Further, in the invention of claim 14, the invention is applied to a recording medium on which a program of a video classification method for classifying videos by an image sequence sequentially input by causing a computer to execute a predetermined processing procedure is classified into a plurality of systems, The processing procedure is based on the determination criteria setting step for setting the determination criteria used for the classification of the video to the system by the classification of the video into the system where the video is not yet classified, and the determination criteria already set, A classification step of determining whether a subsequent video is similar to a video classified into which system, and classifying the subsequent video into a system in which a criterion is already set, and the classification step is already set If it is difficult to classify videos into systems for which judgment criteria are set, depending on the judgment criteria used, videos are still classified according to the priorities set in advance for multiple systems. To classify the video to have not been systematic.

  According to the configuration of the first aspect, the present invention is applied to a video classification device that classifies videos based on sequentially input image sequences into a plurality of systems, and the video is classified into a system to which the video has not yet been classified. Judgment criteria setting means for setting judgment criteria for video classification and judgment criteria that have already been set determine whether the following video is similar to the video classified into which system, and the subsequent video is judged And classifying means for classifying into systems that have already been set. If the video is classified into a system in which the video is not yet classified according to the priority order set in advance, the process of allocating the video to which system can be automatically executed according to the prior priority. It is displayed and distributes the programs and the like of the television broadcast, the classification to be subjected to this distribution can be properly executed.

  Thereby, according to the structure of Claim 12, Claim 13, and Claim 14, the television broadcast program etc. can be sorted and displayed, and the video classified for this sorting can be appropriately executed. A classification method, a video classification method program, and a recording medium recording the video classification method program can be provided.

  According to the present invention, for example, television broadcast programs and the like can be sorted and displayed, and the classification of videos used for the sorting can be appropriately executed.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.

(1) Configuration of Embodiment FIG. 2 is a block diagram showing a broadcasting system according to an embodiment of the present invention. In this broadcasting system 1, a broadcasting station 2 shoots a subject using a plurality of television cameras A, B, and C provided in a studio 3, for example, and images 5A, 5B, and 5C of the television cameras A, B, and C are taken. Is edited by the editing unit 6 to be edited into one video. The broadcasting station 2 sends the edited video D1 by wire or wireless by the broadcasting equipment 7, and thereby, for example, photographers 4A, 4B, and 4C of the conversation program are captured by the television cameras A, B, and C, respectively. In some cases, the videos of the characters 4A, 4B, and 4C are sequentially switched and provided corresponding to the switching of the videos by the televisions A, B, and C. The receiving side 9 receives the video D1 by the receiving unit 10, and classifies the video D1 into videos of the respective characters 4A, 4B, and 4C by the classification device 11. Further, by this classification, the images of the characters 4A, 4B, and 4C are distributed and displayed on the display units 12A, 12B, and 12C.

  FIG. 3 is a block diagram showing the receiving side 9. The receiving side 9 is formed by connecting to a plurality of display devices 14A to 14N, and an image received by the receiving unit 10 by control by data communication between the determination devices 15A to 15N provided in the display devices 14A to 14N. D1 is distributed to the systems related to the display units 12A to 12N of the display devices 14A to 14N.

  That is, the display devices 14A to 14N are formed by display units 12A to 12N by liquid crystal display devices or the like, and determination devices 15A to 15N that display the images DA to DN on the display units 12A to 12N, respectively. The receiving unit 10 is configured so as to be shared by the plurality of display devices 14A to 14N or to each of the plurality of display devices 14A to 14N. Accordingly, in this embodiment, the classification device 11 described above with reference to FIG. 2 is configured by the determination devices 15A to 15N of the display devices 14A to 14N.

  These display devices 14A to 14N can be easily connected by, for example, cascade connection, and conversely, can be easily removed from other display devices. In addition, an ID, which is a unique identification code, is assigned to each of the display units 12A to 12N of each of the display devices 14A to 14N by an ID assignment process described later, and a receiving unit is provided by data communication between the determination devices 15A to 15N based on this ID The video D1 received at 10 is classified into the systems of the display units 12A to 12N and distributed. Accordingly, in this embodiment, as shown in FIG. 4 in comparison with FIG. 3, the display device 14 is simply added as necessary to increase the number of display units 12 used for displaying one program. On the contrary, the number of display units 12 used to display one program can be reduced. In this embodiment, these display devices 14A to 14N are arranged side by side in front of the viewer, for example.

  FIG. 5 is a block diagram showing the determination device 15A. Since the determination devices 15A to 15N are configured substantially the same, the determination device 15A will be described in detail below, and detailed description of the other determination devices 15B to 15N will be omitted.

  The determination device 15A determines whether or not the video D1 sequentially input from the receiving unit 10 is classified into the system of the responsible display unit 12A under the control of the arithmetic processing unit 20, and the determination result of the responsible display unit 12A During the period when the video D1 classified into the system is provided, the sequentially input video D1 is displayed on the display unit 12A in charge, and the video D1 classified into the system of the other display units 12B to 12N is provided. During the period, the last scene of the video D1 classified into the system of the display unit 12A is displayed on the display unit 12A by the still image DS.

  Therefore, in this determination device 15A, the video switching unit 22 switches the operation under the control of the arithmetic processing unit 20, and selectively outputs the video data related to the video D1 and the video data related to the still image DS by the broadcast to the display unit 12A. To do. Under the control of the arithmetic processing unit 20, the communication unit 23 performs various data communication between the arithmetic processing unit 20 and the other determination devices 15B to 15N with respect to data communication related to the ID setting process, classification of video D1, and data communication related to distribution. The data D2 is input / output.

  The recording device 21 is composed of a large-capacity hard disk device or the like. Under the control of the arithmetic processing unit 20, the recording device 21 temporarily records and holds video data related to the video D1 sequentially input from the receiving unit 10, and holds the stored video. Video data related to the still image DS (which is a frame image before a scene change described later) is output as data. The recording device 21 records and holds various data and processing programs necessary for the processing of the arithmetic processing unit 20. In this embodiment, the data necessary for the processing of the arithmetic processing unit 20 is constituted by the data of the reference image, which is the judgment criterion for the scene change detection and the classification of the video D1. Here, the reference image is an image that is appropriately updated and held by the video DA assigned to the display unit 12A in charge of the determination device 15A.

  In this embodiment, the processing program recorded in the recording device 21 is provided by being preinstalled in the display device 14A. However, instead of provision by such preinstallation, it is provided via a network such as the Internet. It may be provided by downloading or may be provided via various recording media. As such a recording medium, various recording media such as an optical disk such as a CD-ROM and a DVD, a magnetic disk such as a floppy disk, a hard disk device integrated with a removable drive mechanism, a memory card, and the like are widely applied. be able to.

  The arithmetic processing unit 20 is a computer that controls the overall operation of the determination device 15A. By executing the processing program recorded in the recording device 21, an ID is set in the display unit 12A and is sequentially input from the receiving unit 10. A scene change is detected from the video D1. Further, it is determined whether or not the video D1 sequentially input by the detection of the scene change is classified into the display unit 12A in charge, and the display on the display unit 12A is switched based on the determination result. In addition, the reference image which is a premise of these processes is updated as appropriate.

  As a result, the arithmetic processing unit 20 assigns the ID setting unit 21A for setting the ID, the scene change detection unit 21B for detecting the scene change, and the video D1 sequentially input by the detection of the scene change to the display unit 12A in charge. The same camera scene determination unit 21C that switches the display of the display unit 12A by determining whether or not the received image is displayed, and a reference information update unit 21D that is used for update processing of the reference image and reference information are configured.

  FIG. 6 is a flowchart showing an ID setting processing procedure related to the ID setting unit 21A of the arithmetic processing unit 20 formed as described above. The arithmetic processing unit 20 sets the ID in each of the display units 12A to 12N by executing the processing procedure of FIG. 6 at regular time intervals. This process may be executed in the display devices 14A to 14N where the power is turned on when the power is turned on, and the display devices 14A to 14N are newly connected by a plug-and-play method. Then, it may be executed in the newly connected display devices 14A to 14N.

  That is, when this processing procedure is started, the arithmetic processing unit 20 moves from step SP1 to step SP2, where the variable n used for setting the ID is set to a value 1, and then moves to step SP3. 23, a response is obtained from the other determination devices 15B to 15N. When the arithmetic processing unit 20 obtains a response in this way, the arithmetic processing unit 20 waits for a response for a predetermined time, and determines whether or not a response has been obtained in the subsequent step SP4. If a response is obtained here, the ID by the variable n has already been assigned to the display units 12B to 12N of the other determination devices 15B to 15N, so that the process proceeds from step SP4 to step SP5. After shifting the variable n by the value 1, the process returns to step SP3.

  As a result, the arithmetic processing unit 20 sequentially increments the variable n and determines whether or not the ID by the variable n is assigned to the display units 12B to 12N of the other devices 14B to 14N to the other devices 14B to 14N. If the ID by the variable n is not assigned to the other devices 14B to 14N, a negative result is obtained in step SP4, and the process moves from step SP4 to step SP6. After setting the ID of the display unit 12A that he is in charge of, the process proceeds to step SP7 and the processing procedure is terminated. Accordingly, the receiving side 9 appropriately sets a unique identification code for each of the display units 12A to 12N, and transmits and receives various data to and from the other display devices 14B to 14N using this identification code. ing.

  Accordingly, the arithmetic processing unit 20 responds to the ID setting process in the other determination devices 15B to 15N, and when the response related to step SP3 is obtained from these other determination devices 15B to 15N, the arithmetic processing unit 20 responds to its own ID and inquiry. When the ID matches, a response is returned.

  On the other hand, FIG. 7 is a flowchart showing a processing procedure related to the classification and distribution of the video D1 by the arithmetic processing unit 20 based on the ID set in this way. The arithmetic processing unit 20 executes this processing procedure when the video D1 currently input from the receiving unit 10 is not classified into the other display units 12B to 12N. Thereby, for example, in the display units 12A to 12N, if the arithmetic processing unit 20 has not yet decided to which display unit 12A to 12N the video D1 is to be allocated immediately after starting the viewing of the program, the arithmetic processing unit 20 is in charge. 7 is executed, the processing procedure shown in FIG. 7 is executed.

  Thus, when starting this processing procedure, the arithmetic processing unit 20 moves from step SP11 to step SP12, and detects the scene change detection processing of the video D1 sequentially input by the receiving unit 10. Here, the arithmetic processing unit 20 waits for the occurrence of a scene change by sequentially determining the video D1 in units of frames with reference to the reference image held in the recording device 21, and when the scene change is detected, the scene change is detected. Is notified to the other determination devices 15B to 15N, and then the process proceeds from step SP12 to step SP13. In step SP13, the arithmetic processing unit 20 determines whether the video D1 after the scene change is a video classified into the system of the display unit 12A in charge by the same camera determination process, and the display unit 12A of the charge display unit 12A In the case of the video classified into the system, the video D1 after this scene change is displayed on the display unit 12A under the control of the video switching unit 22. When the video D1 after the scene change is a video classified into the other display units 12B to 12N, the still image DS related to the last scene of the video classified into the display unit 12A is displayed on the display unit 12A. The operation of the recording device 21 and the video switching unit 22 is controlled.

  After executing the same camera determination process in this way, the arithmetic processing unit 20 proceeds to step SP14, and the video after the scene change is displayed on the display by distributing the one program to the plurality of display units 12A to 12N. It is determined whether or not the display units are classified into the other display units 12B to 12N. If a negative result is obtained here, the process returns from step SP14 to step SP12 and waits for detection of a subsequent scene change, whereas an affirmative result is obtained. Then, the process proceeds from step SP14 to step SP15, and this processing procedure is terminated.

  On the other hand, when the processing procedure of FIG. 7 is not executed, that is, when the still image DS is displayed on the display unit 12A in charge, the arithmetic processing unit 20 receives scenes from the other determination devices 15B to 15B. When the same camera determination process shown in FIG. 7 is executed based on the notification of the change, and the video after the scene change is displayed on the display unit in charge based on the determination result, the processing procedure shown in FIG. 7 is executed.

  FIG. 8 is a flowchart showing a scene change detection processing procedure in the arithmetic processing unit 20. When this processing procedure is started, the arithmetic processing unit 20 moves from step SP21 to step SP22, and acquires an image based on the video D1 for one frame from the receiving unit 10. In the following, the video D1 for one frame acquired in step SP22 is referred to as a current image. Subsequently, the arithmetic processing unit 20 moves to step SP23, and acquires the evaluation function value of the acquired video D1 by the calculation process of the evaluation function value S executed in parallel with these series of processing procedures. Here, the evaluation function value is a value indicating the degree of similarity of the video used for classification with respect to the classified video, and the system of each of the display units 12A to 12N based on the reference image by calculation processing of the evaluation function described later. Each time, the calculation processing unit 20 of each of the corresponding determination devices 15A to 15N is obtained for each image of the video D1.

  As a result, upon obtaining the evaluation function value, the arithmetic processing unit 20 moves to step SP24 and determines whether or not the evaluation function value is equal to or greater than a predetermined value. If the evaluation function value is less than the predetermined value, a scene change occurs. It judges that it is not, and returns from step SP24 to step SP22. On the other hand, if the evaluation function is greater than or equal to a predetermined value, it is determined that a scene change has occurred, the process proceeds from step SP24 to step SP25, and this processing procedure is terminated.

  Accordingly, the arithmetic processing unit 20 uses a scene change as a trigger to determine which display unit 12A to 12N the video D1 after the scene change should be classified into. The classification process is not executed for each frame, and the process is simplified accordingly. If the video D1 input from the receiving unit 10 has been displayed on the display unit 12A so far and the scene change is detected by itself, the television cameras A to C are switched on the broadcasting station 2 side, and the scene is changed. After the change, it can be determined that the video D1 to be allocated to the other display units 12B to 12N is provided. As a result, it is possible to reduce the options related to the classification destination by omitting the options distributed to the display unit 12A in charge in the subsequent specific classification process, and in this case, the classification can be made more robust. It becomes possible to do.

  In addition, the detection of such a scene change is executed by the determination device in charge of the video D1 being input, thereby reducing the burden on the arithmetic processing unit 20 in the other determination devices.

  FIG. 9 is a flowchart showing a procedure for calculating the evaluation function value. The arithmetic processing unit 20 executes this processing procedure for each frame of the video D1 input from the receiving unit 10. That is, when this processing procedure is started, the arithmetic processing unit 20 moves from step SP31 to step SP32, and, as shown in FIGS. 10A and 10B, the reference image DS and the current image DN are shared by a predetermined number. Set the area. In this embodiment, the reference image DS and the current image DN are equally divided in the horizontal direction and the vertical direction, and 5 × 5 areas are set respectively.

  The arithmetic processing unit 20 then proceeds to step SP33, where the variables i and j used for the calculation of the evaluation function value are initialized to the value 1, and the evaluation function value S is initialized to the value 0. Here, the variables i and j are variables that specify the respective regions set in the reference image DS and the current image DN in the horizontal direction and the vertical direction in the order of raster scanning.

  Subsequently, the arithmetic processing unit 20 proceeds to step SP34, and calculates the average value of the pixel values of the pixels belonging to this area for the area of the current image DN specified by the variables i and j. In this embodiment, the arithmetic processing unit 20 calculates the average value of the pixel values in the red, green, and blue color planes, thereby calculating the average value of the pixel values in this region for the red, green, and blue pixel values, respectively. Detection is based on average values Ar, Ag, Ab (so-called chromaticity). In the calculation of the average value, the average value may be detected by luminance, or may be detected by luminance and color difference.

  Subsequently, the arithmetic processing unit 20 proceeds to step SP35, and similarly calculates the average value of the pixel values of the pixels belonging to this area for the area of the reference image DS specified by the variables i and j. This average value calculation is executed in correspondence with the calculation of the average value in step SP35. When calculating the average values Br, Bg, Bb based on the chromaticity for the current image DN, an arithmetic process is performed. Similarly, the unit 20 calculates an average value based on chromaticity.

When the average value of the pixel values in the region specified by the variables i and j is calculated for the reference image DS and the current image DN in this way, the arithmetic processing unit 20 moves to step SP36 and (Ar−Br) ² + The sum of squared differences by the average value of these pixel values is calculated by the calculation process of (Ag−Bg) ² + (Ab−Bb) ² , and the calculation result is added to the evaluation function value S. Instead of such a sum of squared differences, a sum of absolute differences may be applied.

  Subsequently, the arithmetic processing unit 20 moves to step SP37, increments the variable i in the horizontal direction by a value of 1, and then moves to step SP38, where it is determined whether or not the variable i has exceeded the number of horizontal regions 5. To do. If a negative result is obtained here, the arithmetic processing unit 20 returns from step SP38 to step SP34, calculates the average value of the pixel values for the region that continues in the horizontal direction by the updated variable i, and further calculates the average value. The sum of squared differences is added to the evaluation function value S.

  As a result, the arithmetic processing unit 20 sequentially switches the areas in the raster scanning direction to calculate the average value of the pixel values of each area, adds the sum of squares of the difference between the average values to the evaluation function value S, and outputs the rightmost area. When the process is completed, an affirmative result is obtained in step SP38, and the process proceeds from step SP38 to step SP39.

  Here, the arithmetic processing unit 20 initializes the variable i related to the horizontal direction to the value 1, and increments the variable j related to the vertical direction by the value 1. Then, in the subsequent step SP40, the variable j related to the vertical direction becomes the vertical direction. It is determined whether or not the number of areas has exceeded 5, and if a negative result is obtained here, the process returns from step SP40 to step SP34. As a result, the arithmetic processing unit 20 sequentially calculates the average value of the pixel values in each region in the order of raster scanning, as shown in FIGS. 10A1 and 10B1, and calculates the difference sum of squares based on the average value as the evaluation function value. When the processing is completed for all the regions by adding to S, an affirmative result is obtained in step SP40, so that the process moves from step SP40 to step SP41, where the evaluation function values are sequentially accumulated in each region in this way. After setting S to the final evaluation function value S, the process proceeds to step SP42 and this processing procedure is terminated.

  Thus, the evaluation function value S calculated in this way has a value of 0 in a still image, and when the same person is photographed without zooming or panning, Extremely small value. Thereby, in the arithmetic processing unit 20, the evaluation function value S indicating the degree of similarity with the sequentially input video D1 on the basis of the reference image by the video DA that has been displayed on the display unit 12A in charge. , And the evaluation function value S is determined by a predetermined value in step SP24 described above with reference to FIG. 8 to detect a case where the video D1 has changed greatly, thereby detecting a scene change. ing.

  FIG. 11 is a flowchart showing a reference image update processing procedure that is a reference for calculating such an evaluation function value. The arithmetic processing unit 20 sequentially executes the processing procedure in the recording device 21 during the period in which the video D1 input from the receiving unit 10 is displayed on the display unit 12A in charge according to the processing procedure in step SP13. Update the stored reference image.

  That is, when this processing procedure is started, the arithmetic processing unit 20 moves from step SP51 to step SP52, and acquires the evaluation function value S calculated by the evaluation function calculation process. In the subsequent step SP53, the evaluation function value related to the corresponding current image is acquired from the other determination devices 15B to 15N. In the subsequent step SP54, the evaluation having the smallest value is obtained in the acquired other determination devices 15B to 15N. The function value C is detected. As a result, the arithmetic processing unit 20 is currently detected by the other determination devices 15B to 15N that are in charge of the image that is most similar to the image DA that is currently input and output to the display unit 12A in charge. The evaluation function value of the image is detected.

  Subsequently, the arithmetic processing unit 20 moves to step SP54, and with respect to the evaluation function value C in the other determination devices 15B to 15N in charge of the most similar video in this way, the evaluation function value related to its own device It is determined whether S has a sufficient margin. Here, whether or not there is a sufficient margin is determination of the possibility of misclassification of the video D1, and in this embodiment, the evaluation function value C in the other determination devices 15B to 15N having the smallest value is used. By determining whether the value divided by the evaluation function value S related to the own device is larger than the predetermined determination reference value p, the evaluation function value related to the own device is determined to the extent that there is a possibility of misclassification. It is determined whether or not S has a margin with respect to the evaluation function value C related to another device.

  If there is a sufficient margin here, the arithmetic processing unit 20 proceeds from step SP55 to step SP56, and here, without updating any reference image, proceeds to step SP57 and ends this processing procedure.

  If there is not enough room for this, the arithmetic processing unit 20 updates the reference image by interpolating the previous reference image and the current image with a predetermined interpolation ratio, and then proceeds to step SP57. This processing procedure is completed. Accordingly, in this case, the arithmetic processing unit 20 sequentially performs the arithmetic processing of (1−q) × B + q × A for each pixel, assuming that the pixel value of the reference image so far is B and the pixel value of the input image is A. By executing, an interpolation process is executed. Here, q is a predetermined value larger than the value 0 and smaller than the value 1. It should be noted that such interpolation processing and further reference image update processing are not limited to the case where the margin is equal to or less than a predetermined value as in this embodiment, and may be always executed. Immediately after the scene change, it may be executed at certain intervals in the scene. Further, the weighting coefficient q used for the interpolation process may be changed depending on the degree of margin.

  By this update process, the arithmetic processing unit 20 updates the reference image corresponding to these processes when the cameraman executes processes such as pan, tilt, zoom, and the like, and shoots the same subject. In the evaluation function value calculation process described above, an evaluation function value having a small value can be obtained, so that misclassification of the video D1 is effectively avoided.

  12 and 13 are flowcharts showing the same camera determination processing procedure. When this processing procedure is started, the arithmetic processing unit 20 moves from step SP61 to step SP62, and whether or not the distribution video has been set in the display unit 12A in charge until that time, that is, in the example of FIG. It is determined whether any one of A to C is already assigned to the display unit 12A in charge. Here, when the display device 14A is immediately after the power is turned on, and further when the display device 14A is connected to another display device 14B or the like, a negative result is obtained, so that the arithmetic processing unit 20 Exchanges evaluation function values with the other determination devices 15B to 15D.

  Further, in the subsequent step SP64, the evaluation function values of the other determination devices 15B to 15D acquired by exchanging the evaluation function values with the other determination devices 15B to 15D and the evaluation function value by the arithmetic processing unit 20 are both. It is determined whether or not the predetermined threshold value is exceeded. However, immediately after the power sources of the determination devices 15A to 15N are collectively started up on the receiving side 9, and immediately after switching the program to be received, a positive result is obtained in this case, so that the arithmetic processing unit 20 The process moves from step SP64 to step SP65, and it is determined whether or not its own ID is the youngest among the determination devices released here. Here, the release means a state in which the video D1 is not yet classified.

  If a positive result is obtained here, the arithmetic processing unit 20 moves from step SP65 to step SP66, and sets the video D1 that is currently input via the receiving unit 10 to be assigned to the display unit 12A in charge. Further, after controlling the operation of the recording device 21 so as to sequentially record the currently input video D1 on the recording device 21 for outputting the still image DS, the process proceeds to step SP67 and the processing procedure is terminated. On the other hand, if a negative result is obtained in step SP65, the arithmetic processing unit 20 moves from step SP65 to step SP68, and in this case, the video D1 currently input is in charge in the other determination devices 15B to 15N. By assigning to the display units 12B to 12N, the process waits until the next scene change is detected on the display devices 14B to 14N to which the currently input video D1 is distributed, step SP67. Then, this processing procedure is completed.

  If a negative result is obtained in step SP64, in this case, the video D1 that is currently input to any of the other determination devices 15B to 15N in which an evaluation function value equal to or less than the threshold is detected is distributed. Also in this case, after moving from step SP64 to step SP68, the process moves to step SP67 and the processing procedure is terminated.

  As a result, as shown in FIG. 1, in this embodiment, when the receiving side 9 is constituted by, for example, four display devices 14A to 14D, the display devices 14A and 14B are respectively performed by the ID setting process of FIG. , 14C, and 14D in the order of 14D, when viewing of the program by the four television cameras A to D is started from time tA, it responds to the switching of the television cameras A to D on the broadcasting station 2 side. Thus, each time a new camera scene is generated, the scenes of the respective cameras are sequentially assigned in the order of the display devices 14A, 14B, 14C, and 14D. In FIG. 1, a period indicated by hatching is a period during which the video D1 sequentially input is displayed on the corresponding display unit.

  On the other hand, when the distribution video is already set on the display unit 12A in charge of the arithmetic processing unit 20, the arithmetic processing unit 20 obtains an affirmative result at step SP62, so that the processing proceeds from step SP62 to step SP70. Move. Here, the arithmetic processing unit 20 determines whether or not the video D1 previously input from the receiving unit 10 has been distributed to the display unit 12A in charge.

  If this affirmative result is obtained here, in this case, the scene change is detected by the arithmetic processing unit 20 based on the reference image held in the recording device 21 related to the arithmetic processing unit 20, so that the scene change Of course, in the later video D1, it can be determined that the video is distributed to the display units 12B to 12N in charge of the other determination devices 15B to 15N. Accordingly, in this case, the arithmetic processing unit 20 moves from step SP70 to step SP68, and then moves to step SP67 to end this processing procedure. In this case, the arithmetic processing unit 20 switches the operations of the recording device 21 and the video switching unit 22 so that the image immediately before the scene change held in the recording device 21 is output as the still image DS.

  Thus, in this case, the arithmetic processing unit 20 displays the corresponding image immediately before the scene change as a still image DS during the period indicated by the broken line in FIG.

  On the other hand, if a negative result is obtained in step SP70, in this case, since the scene change is detected in the other determination devices 15B to 15N, the arithmetic processing unit 20 moves from step SP70 to step SP71. Then, an evaluation function value based on a reference image executed simultaneously in parallel is acquired. Further, in the subsequent step SP72, the evaluation function value is exchanged with the other determination devices 15B to 15D, and in the subsequent step SP73, the other obtained by exchanging the evaluation function value with the other determination devices 15B to 15D. By comparing the evaluation function values of the determination devices 15B to 15D and the evaluation function value by the calculation processing unit 20, it is determined whether or not the evaluation function value by the calculation processing unit 20 is the smallest.

  If an affirmative result is obtained here, the arithmetic processing unit 20 proceeds from step SP73 to step SP74, and determines whether or not the evaluation function value by the arithmetic processing unit 20 is equal to or greater than a predetermined value. That is, when the evaluation function value by the arithmetic processing unit 20 is the smallest, it can be said that the current image after the scene change is most similar to the video distributed to the display unit 12A in charge of the arithmetic processing unit 20, but it has been determined so far. There may be cases where the video does not correspond to any of the images DA to DN distributed to the devices 15A to 15N. In this case, the evaluation function value has a large value.

  As a result, if a negative result is obtained in step SP74, the arithmetic processing unit 20 moves from step SP74 to step SP75, and distributes the video D1 after this scene change to the display unit 12A in charge as described above for step SP66. After controlling the operation of each part as described above, the process proceeds to step SP67 and this processing procedure is terminated.

  On the other hand, if a positive result is obtained in step SP74, the process proceeds from step SP74 to step SP76, and it is determined whether or not there is a released determination device through data communication with the other determination devices 15B to 15N. To do. If there is no determination device released here, the arithmetic processing unit 20 moves from step SP76 to step SP75, and the video D1 after this scene change is displayed on the display unit 12A in charge as described above for step SP66. After controlling the operation of each unit so as to distribute, the process proceeds to step SP67 and this processing procedure is terminated.

  On the other hand, when a negative result is obtained in step SP73, an affirmative result is obtained in step SP72 of the same camera determination process in the arithmetic processing unit 20 of another determination apparatus, and the video after the scene change is obtained as a positive result. Is assigned to the determination device obtained, the arithmetic processing unit 20 proceeds from step SP73 to step SP77. If an affirmative result is obtained in step SP76, an affirmative result is obtained in step SP62 of the same camera determination process in the arithmetic processing unit 20 of the released determination apparatus, and the video after the scene change is determined to be released. By being assigned to the apparatus, the arithmetic processing unit 20 proceeds from step SP76 to step SP77.

  Here, the arithmetic processing unit 20 stops outputting the video D1 to the display unit 12A in charge, and determines whether or not a predetermined time has elapsed. If an affirmative result is obtained here, in this case, the camera related to the video that has been distributed to the display unit 12A can determine that the imaging target has been switched to another subject, and so on, so that the arithmetic processing unit 20 After moving from SP77 to step SP78 and releasing the system related to its own determination device 15A, the process proceeds to step SP67 and this processing procedure is terminated. Note that the arithmetic processing unit 20 is configured to reset the reference image held in the recording device 21 when released in this way.

  On the other hand, if a negative result is obtained in step SP77, the arithmetic processing unit 20 moves from step SP77 to step SP68, and then moves to step SP67 to end this processing procedure.

  Accordingly, in this embodiment, the arithmetic processing unit 20 sets a reference image, which is a determination reference for use in classification of the subsequent video to the system, based on the classification of the video into the system where the video D1 is not yet classified. The setting means is configured. In addition, the arithmetic processing unit 20 determines whether the subsequent video is similar to the video classified into which system based on the already set determination criterion, and sets the subsequent video to the system for which the determination criterion is already set. If the classification means for classifying is configured and it is difficult to classify the video D1 into the system for which the determination standard is set depending on the determination standard that has already been set, according to the priority order based on IDs set in advance for these multiple systems The video D1 is classified into a system in which the video D1 is not yet classified.

(2) Operation of the embodiment In the above-described configuration, in the broadcasting system 1 (FIG. 2), images formed by photographing the characters 4A, 4B, and 4C with the television cameras A, B, and C of the broadcasting station 2, respectively. One video D1 that is switched and used for broadcasting is generated, and this video D1 is transmitted by the transmission facility 7 and received by the receiving side 9. On the receiving side 9, the video D1 related to the broadcast is classified into a plurality of systems related to the display units 12A to 12N (FIG. 3) by the determination devices 15A to 15N constituting the classification device 11, and the classified video DA To DN are displayed by the display units 12A to 12N, respectively. In this way, the video D1 is sorted and displayed, and during the period when the video D1 is not sorted, the last image to which the video D1 is sorted is displayed as the still image DS.

  As a result, on the receiving side 9 according to the broadcasting system 1, the display devices 14A to 14N are arranged in front of the viewer, and the images of the characters 4A, 4B, and 4C are displayed on the display devices 14A to 14N, respectively. In addition, it is possible to provide viewers with a sense of reality as if they are participating in a debate, thereby improving the sense of reality compared to the prior art.

  As a premise that the video D1 is classified and distributed to the display units 12A to 12N in this way, in the determination devices 15A to 15N, the ID which is the identification code of each device is automatically set between the devices by the ID setting process (FIG. 6). Each determination device 15A to 15N is formed so that it can be easily connected and disconnected.

  As a result, on the receiving side 9, the number of display units 12A to 12N used for displaying one program can be easily increased or decreased as necessary, and the user-friendliness can be improved accordingly (FIG. 3). And FIG. 4).

  On the receiving side 9, by the classification of the video D1 into a system in which the video D1 has not yet been classified, a reference image, which is a criterion for use in classification of the subsequent video into the system, is set by the arithmetic processing unit 20, as described above. In accordance with the determination criteria set to ## EQU2 ## it is determined to which system the subsequent video D1 is similar to the video classified into which system, and this subsequent video D1 is classified into each system (FIGS. 1, 12, and 13). ). In this process, for example, immediately after the power is turned on, in a new scene or the like, depending on the determination criteria already set, if it is difficult to classify the video into a system in which the determination criteria are set, according to the priority order by the ID, The video is classified into a system in which the video D1 is not yet classified.

  As a result, the process of allocating the video D1 to which system can be automatically executed in accordance with the priorities in order to distribute and display television broadcast programs, etc. It has been made possible to reduce the burden on the user by running it.

  Also, in this process, in this embodiment, an evaluation function value indicating the degree of similarity between the video already classified into each system by the reference image and the video used for classification is detected, and the video is obtained from this evaluation function value. By using this evaluation function value, it is possible to easily and reliably classify the video by detecting the case where it is difficult to classify the video into the system in which the criterion is set. .

  In other words, in FIG. 1, for example, when viewing of a program is started at time tA, the display portion 12A having the youngest ID is obtained because the video D1 from any television camera A to D is not assigned to any system. This video D1 is assigned to the system. Further, in the video after the scene change at the subsequent time point tB, the video D1 is subsequently classified into the display unit 12B having a lower ID. If the video D1 after the scene change at the subsequent time tC is similar to the classified video after the time tA, the video D1 is classified into the display unit 12A to which the video after the time tA is assigned. Further, in the video D1 after the scene change at the subsequent time point tD, the video D1 is subsequently classified into the display unit 12C having a lower ID.

  On the other hand, for a system in which the criterion is already set and the video D1 is not classified for a predetermined time or more, the corresponding display unit is released, so that the video is not yet classified. Is done. Thus, for example, after the video is assigned at the time tE, the system of the display unit 12B to which the video D1 is not assigned for a long time is released after the scene change at the time tF. Further, the video after the scene change at the subsequent time point tG is classified into this released system.

  As a result, on the receiving side 9, even if the user does not perform an operation such as resetting, for example, after a program is switched, an unnatural situation in which a still image related to the previous program is displayed for a long time. Thus, television broadcast programs and the like can be distributed and displayed, and classification for this distribution can be appropriately executed to reduce the burden on the user.

  Thus, in this embodiment, for a system in which the determination criterion is already set and the video is not classified for a predetermined time or more, by setting the system in which the video is not yet classified, for example, the program Even in the case of switching, the video can be appropriately classified.

  Therefore, in the determination devices 15A to 15N, once the video D1 is classified and a reference image that is a determination reference is set, each image of the video D1 that is sequentially input based on the determination reference is digitized and classified. The evaluation function value having a smaller value is calculated as the image D1 is more similar to the completed image. Further, the video D1 is classified based on the evaluation function value, and a case where it is difficult to classify the video is detected.

  In this embodiment, the numerical value processing of the image related to the video D1 based on the determination criterion performs comparison between the reference image, which is an image related to the determination criterion, and the image of the video D1, and the video D1 is classified by this comparison. Is done. Further, this comparison is executed by comparing the average value of the pixel values between a plurality of areas set in common for the reference image and the video image (FIG. 10). As a result, in this embodiment, even if the television camera moves slightly or the background changes, the same subject photographed by the same television camera can be correctly classified. Misclassification can be effectively avoided.

  That is, when comparing the reference image and the input image in this way, there is also a method of comparing by an evaluation value such as a difference sum of pixel values, for example, but in this method, the evaluation value changes greatly even if the camera moves a little. This often leads to misclassification. However, as in this embodiment, if the comparison is performed by comparing the average values of the pixel values for a plurality of areas set in common for the reference image and the video image, the evaluation value changes when the camera moves a little. It is possible to reduce this, and thus it is possible to effectively avoid misclassification and improve classification accuracy.

  Thus, in this embodiment, the image values are classified into each system by comparison with the reference image, and the pixel values between a plurality of regions set in common for the determination reference image and the image used for classification are compared. By performing the comparison, for example, the comparison of the pixel values is performed based on the average value of the pixel values, so that misclassification in the case where the television camera is moved a little is effectively avoided.

  In the determination devices 15A to 15N, the video is finally classified based on the evaluation function value thus obtained, and when a scene change is detected, the video after the scene change is classified into any one by this evaluation function. Is determined. That is, if it is determined for each image constituting the video D1, the processing becomes extremely complicated. As a result, in this embodiment, the number of classification processes by this evaluation function is reduced, and the burden on the arithmetic processing unit 20 is reduced accordingly.

  Thus, in this embodiment, by detecting the scene change, the classification of the video after the scene change is determined, thereby reducing the burden of the configuration used for the classification.

  In addition, the scene change detection is performed by the evaluation function determination in the determination device in charge of the video D1, thereby simplifying the configuration for scene change detection.

  Also, in this embodiment, an evaluation function value indicating the degree of similarity between the video already classified in each system and the video used for classification is detected, and the video classification after the scene change is determined based on this evaluation function value. In addition, by detecting a scene change, it is possible to detect a scene change by effectively using a judgment criterion used for classification, and the configuration for detecting a scene change can be simplified accordingly. Yes.

  In the determination devices 15A to 15N according to this embodiment (FIG. 11), the video D1 related to the input image is classified by the reference image in this way, and the determination device in charge of the video D1 uses the input image. The reference image is updated as appropriate. That is, for example, when the reference image is fixed to the image at the beginning of the classification, in the actual broadcast, the video gradually changes due to zooming, panning, etc., so that the accuracy of the classification gradually decreases, and finally the classification is mistaken. There is a fear. In addition, after once misclassified in this way, it is difficult to return to an operation state in which classification is correctly performed.

  However, in this embodiment, by appropriately updating the reference image with the input image in this way, if the video changes gradually due to zooming, panning, etc., the change in the video can be reflected in the reference image. As a result, misclassification can be effectively avoided, and even after misclassification, the normal operation state can be easily restored.

  As a result, in this embodiment, the classification criteria of the classification destination system is updated by the video classified into each system, so that misclassification can be effectively avoided even when the television camera pans. It is made to be able to.

  In this embodiment, such an update process is executed when the margin for misclassification is determined based on the evaluation function value of each system, and the margin is reduced according to the determination result. In this way, when a video D1 that is obtained by switching a similar video is input, the reference image is updated frequently and enough to ensure a sufficient margin, thereby further reliably preventing misclassification. be able to. In addition, with respect to the video D1 in which a significantly different video is switched, frequent updating of the reference image can be prevented, and the processing load can be reduced accordingly.

  Thus, in this embodiment, the update process determines the margin for misclassification based on the evaluation function value, which is a criterion used for classification, and updates the criterion according to the determination result, thereby correcting the misclassification. It is possible to start effectively, and depending on the video, it is possible to reduce the processing load related to the update.

(3) Effects of the embodiment According to the above configuration, by setting the video classification distribution according to the priorities set in advance for each system, for example, television broadcast programs and the like are distributed and displayed. Thus, it is possible to appropriately execute the classification of the images used for the distribution.

  FIG. 14 is a flowchart showing the processing procedure of the determination apparatus according to the second embodiment of the present invention in comparison with FIG. The receiving side according to this embodiment is configured in the same way as the receiving side described above with respect to Embodiment 1 except that the processing procedure shown in FIG. 14 is different.

  In this embodiment, in the processing unit of the evaluation function, the arithmetic processing unit of the determination apparatus uses the average pixel for the region of the current image and the reference image specified by the variables i and j by the processing of step SP34 and step SP35. When the value is calculated, in the subsequent step SP35-1, the weighting coefficient W set for each region is used to weight the average pixel value detected in step SP34 and step SP35.

  Accordingly, for example, a weighting coefficient W having a larger value than the periphery of the screen is set at the center of the screen, and the video can be classified with emphasis on the center of the screen. Thus, in this embodiment, the video can be classified with emphasis on a specific area by setting the weighting coefficient, and misclassification can be effectively avoided accordingly. In the setting of such weighting coefficients, the weighting coefficient may be set adaptively so that the margin for misclassification is increased according to the criterion for misclassification relating to the update process of the reference image described above.

  According to the above configuration, the comparison is performed between a plurality of regions set in these images by classifying the reference images based on the videos classified into each system and the images of the videos used for classification. In the case of comparison of pixel values, by assigning weights to the pixel values using the weighting coefficient set for each area and comparing them, it is possible to set an area to be emphasized in classification, thereby further preventing misclassification. be able to.

  FIG. 15 is a schematic diagram illustrating the configuration of the reception side according to the third embodiment. In this embodiment, a plurality of display devices 14A to 14D are sequentially arranged from the viewer side so that the display screen of the rear display device can be seen. In addition, the display devices 14A to 14D are arranged in this way, and each display device 14A to 14D is configured by a determination device and a display unit, respectively, similarly to the display device according to the first embodiment. The classified images are classified and distributed by the display devices 14A to 14D.

  In this embodiment, when the received video is classified and distributed to the display devices 14A to 14D, the currently received video is distributed so as to be displayed by the front display device 14A. In addition, in this way, the currently received video is distributed to the display device 14A on the foremost side, and the display of these display devices 14A to 14D is switched so as to display still images related to the old classification in the rear side.

  As a result, the determination device provided in each of the display devices 14A to 14D is configured in the same manner as the determination device described above with respect to the first embodiment, except that the processing relating to such video distribution is different.

  That is, in the arithmetic processing unit of the determination device provided in the display devices 14A to 14D, the evaluation function value is calculated by comparison with the received video using the reference image, respectively, and is provided in the display device 14A arranged in the foremost position. The arithmetic processing unit of the determination apparatus further detects a scene change. Also, by detecting a scene change, evaluation function values are transmitted and received between the determination devices provided in the display devices 14A to 14D, and it is determined to which display unit 12A to 12D each video after the scene change should be classified. The received video classification process is executed based on the determination result, and the video is sorted according to the classification. In addition, the reference image and the like are exchanged between the determination devices so as to correspond to this distribution. In addition, the reference image is updated only in the arithmetic processing unit of the determination device provided in the display device 14A arranged at the foremost position.

  FIG. 16 is a flowchart showing a processing procedure related to video distribution for the arithmetic processing unit provided in the display device 14A arranged in the foremost position. In this processing procedure, the arithmetic processing unit moves from step SP81 to step SP82, waits for the occurrence of a scene change, and moves to step SP83 from step SP82 when a scene change is detected. Here, the arithmetic processing unit executes the same camera determination processing described above with reference to FIGS. 12 and 13 together with the arithmetic processing unit provided in another determination apparatus, and thereby classifies the video after the scene change.

  Subsequently, the arithmetic processing unit moves to step SP84, and acquires various parameters, reference images, still images, etc. used for classification processing from the display device in which the video after the scene change is classified, and temporarily records them in the memory. To do. In the subsequent step SP85, an ID is acquired from the display device in which the video after the scene change is classified, and this ID is set to a variable n. In the subsequent step SP86, the arithmetic processing unit determines whether or not the variable n is a value 1 indicating its own device.

  If an affirmative result is obtained in this step SP86, the arithmetic processing unit in this case, for example, immediately after the power is turned on, the video after the scene change is the video classified into the display device 14A in charge. In step SP84, the parameters and the like acquired in step SP84 are restored, and in the subsequent step SP88, the received video is set to be displayed on the display unit 12A in charge, and then the process proceeds to step SP89. Here, the arithmetic processing unit determines whether or not the user has instructed the end of viewing, and if a negative result is obtained here, the process returns to step SP82 and waits for a subsequent scene change, whereas an affirmative result is obtained in step SP89. Is obtained, the process proceeds from step SP89 to step SP90, and this processing procedure ends.

  On the other hand, if a negative result is obtained in step SP86, the arithmetic processing unit moves from step SP86 to step SP92. Here, the arithmetic processing unit sets various parameters, a reference image, a still image, and the like used for the classification processing held in the ID determination device based on the variable n−1 in the ID determination device based on the variable n. Further, in the subsequent step SP93, after instructing to display the still image thus transferred, the process proceeds to step SP94, the variable n is decremented, and then the process returns to step SP86.

  As a result, the arithmetic processing unit sorts the video so that the video after the scene change is always displayed on the display device 14A on the foremost side, and sequentially displays the still images according to each classification corresponding to the video sorting. It is designed to move backward.

  Even if the distribution is switched according to the classification result as in this embodiment, it is possible to appropriately classify the video as in the first embodiment and improve the sense of reality. Further, if displayed in this way, each scene can be displayed in time series from the front side, and the time direction can be recognized by the user.

  In the above-described embodiment, the case has been described in which the reference image is set as the determination reference and the evaluation function value indicating the degree of similarity between the images used for classification is calculated, but the present invention is not limited thereto, The evaluation function value may be calculated based on image dispersion, the feature amount of each region, or the like, or may be calculated in combination.

  That is, in the case where the calculation is performed based on the image variance and the evaluation value is calculated based on the spatial variance of the image, as in the case of the first embodiment, a common area is set for each of the reference image and the current image. The variance of pixel values is detected in the area. Here, in the dispersion of the pixel values, for example, when the difference value between the maximum value and the minimum value of the pixel values is applied, the difference absolute value sum or the difference square of each pixel value based on the average value is used. There are cases where the sum is calculated and provided. When calculating an evaluation value based on such spatial dispersion of an image, a difference value between the maximum value and the minimum value calculated in this way, a sum of absolute differences or a sum of squared differences, and further The evaluation value can be calculated by comparing the average value of each region between the reference image and the current image. In such variance calculation, various methods such as a case where the calculation is performed by using a plurality of sampling points set for one image instead of setting such an area can be applied.

  On the other hand, when the evaluation value is calculated based on temporal dispersion of the images, in the continuous image of the video distributed to each system, it is further common to each image by the area set in the same manner as in the first embodiment. The change value of the pixel value is represented directly by the sampling point set to, or the change value of the pixel value is represented indirectly by the maximum value, the minimum value, the average value, etc., and the maximum value and the minimum value of these change values. A difference value, a sum of absolute differences, or a sum of squared differences is set as an evaluation standard, and the variance in the time axis direction between the reference image calculated in the same manner and the current image is compared with this evaluation standard. By doing so, the evaluation value can be calculated.

  On the other hand, in the case of using the feature amount, for example, the current image and the reference image are compared based on the area of the specific hue area in one image and the area of the specific hue area in each area described above. An evaluation value can be calculated.

  When these various criteria are applied, the accuracy of the classification image is improved by appropriately changing the reference image in the same manner as described above for the first and second embodiments according to these criteria. can do. In this case, for example, when the evaluation value is calculated by applying the variance in the time axis direction, the portion having a small variance is the background or the like. By updating the reference image with a smaller size, the accuracy of classification can be further improved.

  Further, in the above-described embodiment, a case has been described in which the reference image is appropriately updated by setting an area common to the reference image and the current image and calculating the evaluation value. However, the present invention is not limited to this. Instead of updating the reference image, a calculation value used for calculation of the evaluation value such as an average value of each region used for calculation of the evaluation value may be directly updated.

  In the above-described embodiments, the case where the reference image and the current image are equally divided and a plurality of regions are set in these images has been described. However, the present invention is not limited to this, and for example, only the central portion of these images has regions. For example, when the same effect as when weighted is set and the same effect is secured, the plurality of regions can be set in various ways.

  Further, in the above-described embodiment, the case where the received video is classified and distributed by sharing the processing by the determination device provided in each display device has been described. However, the present invention is not limited to this and is combined into one classification device. These processes may be executed.

  In the above-described embodiment, the case where the video is distributed to the plurality of display units according to the classification result is described. However, the present invention is not limited to this, and for example, as shown in FIG. The present invention can be widely applied to the case of distributing videos classified into a plurality of display areas.

  Further, in the above-described embodiment, the case where the video is distributed to the display based on the classification result has been described. However, the present invention is not limited to this, and the case where a specific scene is extracted based on the classification result and is dubbed to the recording apparatus for processing. Furthermore, the present invention can be widely applied to the case where such a classification result is transmitted together with video and the display can be switched by the classification result on the receiving side.

  The present invention can be applied to viewing of television programs such as conversations.

It is a schematic diagram with which it uses for description of image | video distribution which concerns on the Example of this invention. It is a block diagram which shows the broadcasting system which concerns on the Example of this invention. It is a block diagram which shows the receiving side in the broadcasting system of FIG. It is a block diagram with which it uses for description of the addition of a receiver. It is a block diagram which shows the determination apparatus in the receiver of FIG. It is a flowchart which shows ID setting process sequence. It is a flowchart which shows the process sequence which concerns on the classification | category and distribution of an image | video. It is a flowchart which shows the process sequence of a scene change. It is a flowchart which shows the calculation processing procedure of an evaluation function value. It is a schematic diagram with which it uses for description of the process sequence of FIG. It is a flowchart which shows the update process sequence of a reference | standard image. It is a flowchart which shows the same camera determination processing procedure. 13 is a flowchart showing a continuation of FIG. It is a flowchart which shows the process sequence of the determination apparatus which concerns on Example 2 of this invention. FIG. 10 is a schematic diagram illustrating a configuration on a receiving side according to a third embodiment. 12 is a flowchart illustrating a processing procedure related to video distribution according to a third embodiment. It is a top view which shows the example of a display by another Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Broadcasting system, 2 ... Broadcasting station, 10 ... Reception part, 11 ... Classification part, 12A-12N + 1 ... Display part, 14A-14N + 1 ... Display apparatus, 15A-15N + 1 ... Determination apparatus, 20 ... ... Operation processing unit

Claims (14)

A video classification device for classifying videos according to sequentially input image sequences into a plurality of systems,
A criterion setting means for setting a criterion to be used for classification of the video to the system by classification of the video into the system where the video is not yet classified;
Classification means for determining, based on the determination criterion already set, whether the subsequent video is similar to a video classified into which system, and classifying the subsequent video into a system where the determination standard is already set And
The classification means includes
Depending on the determination criteria that have already been set, if it is difficult to classify the video into a system for which the determination standard is set, the video is still classified according to the priorities set in advance for the plurality of systems. The video classification apparatus characterized in that the video is classified into no system. The classification means includes
According to the determination criteria, an evaluation function value indicating a degree of similarity between the video already classified into the systems and the video used for classification is detected, and the video is classified according to the evaluation function value,
The video classification apparatus according to claim 1, wherein a case where it is difficult to classify the video into a system in which the determination criterion is set is detected based on the evaluation function value. The classification means includes
For the system in which the determination criterion is already set and the video is not classified for a predetermined time or more, the system is set to a system in which the video is not yet classified. The video classification device according to claim 1. The classification means includes
Set the image of the video classified into the system as the determination criterion of the system,
The classification means includes
By classifying the video into the system by comparing the image according to the criterion and the image of the video,
The comparison between the image according to the determination criterion and the image of the video is a comparison of pixel values between a plurality of areas set in common for the image according to the determination criterion and the image of the video. Item 2. The video classification device according to Item 1. The video classification apparatus according to claim 4, wherein the comparison of the pixel values is a comparison based on an average value of the pixel values in the region. The video classification apparatus according to claim 4, wherein the comparison of the pixel values is a comparison by weighting the pixel values using a weighting coefficient set for each region. The classification means includes
Scene change detecting means for detecting a scene change of the video;
The video classification apparatus according to claim 1, further comprising: a determination unit that determines the classification of the video after the scene change by detecting the scene change by the scene change detection unit. The classification means includes
According to the determination criterion, an evaluation function value indicating a degree of similarity between the video already classified into each system and the video used for the classification is detected,
The determination means includes
Determine the video classification after the scene change by the evaluation function value,
The scene change detection means includes
The video classification device according to claim 7, wherein the scene change is detected based on the evaluation function value. The determination criterion setting means includes
The video classification apparatus according to claim 1, wherein the determination criterion of the classification destination system is updated with the video classified by the classification unit. The classification means includes
According to the determination criteria, an evaluation function value indicating a degree of similarity between the video already classified into the systems and the video used for the classification is detected, and the video is classified according to the evaluation function value,
The determination criterion setting means includes
The video classification apparatus according to claim 9, wherein a margin for misclassification is determined based on the evaluation function value of each system, and the determination criterion of the system of the classification destination is updated according to the determination result. . The system is
The video classification apparatus according to claim 1, wherein each of the systems is used for displaying the video. A video classification method for classifying videos by sequentially input image sequences into a plurality of systems,
A criterion setting step for setting a criterion to be used for classification of the video to the system by classification of the video into the system where the video is not yet classified;
Based on the determination criterion already set for the system, it is determined whether the subsequent video is similar to the video classified into which system, and the subsequent video is classified into the system for which the determination standard is already set. And a classification step to
The classification step includes:
Depending on the determination criteria that have already been set, if it is difficult to classify the video into a system for which the determination standard is set, the video is still classified according to the priorities set in advance for the plurality of systems. The video classification method, wherein the video is classified into the system that does not exist. A program of a video classification method for classifying videos based on image sequences sequentially input by causing a computer to execute a predetermined processing procedure, into a plurality of systems,
A criterion setting step for setting a criterion to be used for classification of the video to the system by classification of the video into the system where the video is not yet classified;
Based on the determination criterion that has already been set, it is determined whether the subsequent video is similar to the video classified into which system, and the subsequent video is classified into a system in which the determination criterion is already set. And having steps
The classification step includes:
Depending on the determination criteria that have already been set, if it is difficult to classify the video into a system for which the determination standard is set, the video is still classified according to the priorities set in advance for the plurality of systems. A video classification method program comprising classifying the video into the non-system. A recording medium on which is recorded a program of a video classification method for classifying video based on an image sequence sequentially input by causing a computer to execute a predetermined processing procedure, into a plurality of systems,
The processing procedure is as follows:
A criterion setting step for setting a criterion to be used for classification of the video to the system by classification of the video into the system where the video is not yet classified;
Based on the determination criterion that has already been set, it is determined whether the subsequent video is similar to the video classified into which system, and the subsequent video is classified into a system in which the determination criterion is already set. And having steps
The classification step includes:
Depending on the determination criteria that have already been set, if it is difficult to classify the video into a system for which the determination standard is set, the video is still classified according to the priorities set in advance for the plurality of systems. A recording medium on which a video classification method program is recorded, wherein the video is classified into a non-system.

Images