JP6569315B2 - Video surveillance system, video surveillance method, and video surveillance program - Google Patents

Description

  The present invention relates to a moving image monitoring system, a moving image monitoring method, and a moving image monitoring program capable of monitoring a distributed moving image.

  There is a moving image distribution service that provides replay even while receiving a moving image via a communication network such as the Internet. In such a moving image distribution service, for example, an improper moving image such as an improper subject being captured or infringement of copyright can be distributed.

  Therefore, in Patent Document 1, the management server and the client terminal that has received the moving image distribution service and received the moving image transmit and receive the moving image, and depending on whether the moving image is inappropriate or not, A method for controlling playback is described.

  Patent Document 2 describes a system that downloads video at a predetermined timing in a video uploaded to a video distribution server and monitors the uploaded video.

JP 2009-49949 A International Publication No. 2012/132532 International Publication No. 2013/073273

  In the method described in Patent Document 1, the client terminal transmits the distributed video to the management server, and the management server does not perform the processing based on the transmitted video and information registered in the database in advance. If it is determined to be inappropriate, the video is transmitted to the client terminal. And the said client terminal reproduces | regenerates the moving image transmitted from the management server.

  Therefore, since the video distributed to the client terminal is transmitted / received to / from the management server, the amount of data transmitted / received increases, and the processing load and the communication load in the communication network between the client terminal and the management server are increased. There is a problem of being overloaded.

  In the system described in Patent Document 2, when the video download interval is shortened to enhance monitoring, the amount of data transmitted to and received from the video distribution server increases, and the processing load and the video distribution server There is a problem that the communication load in the communication network between the two becomes excessive.

  Therefore, an object of the present invention is to provide a moving image monitoring system, a moving image monitoring method, and a moving image monitoring program capable of monitoring a distributed moving image while satisfactorily suppressing an increase in processing load and communication load. To do.

  The moving image monitoring system according to the present invention includes a data generation unit that generates feature data corresponding to the characteristics of a frame that forms a moving image based on the moving image data received from the moving image distribution device, and data that transmits the feature data generated by the data generation unit. A reception playback terminal including a transmission unit, an image processing unit that performs image processing on video data, and a video playback unit that plays back a video based on the video data, and a determination for determining whether the video is inappropriate Video data corresponding to the feature data is inappropriate when the storage means stores the judgment data that matches the feature data transmitted by the data transmission means of the receiving / playback terminal and the data transmission means A management terminal including information transmission means for transmitting determination result information indicating that the video data is to the reception / playback terminal, and indicating that the video data is inappropriate When the fixed result information is transmitted, the image processing unit performs image processing on the moving image data, and the moving image reproduction unit reproduces a moving image based on the moving image data subjected to the image processing by the image processing unit. .

  The moving image monitoring method according to the present invention includes a data generation step for generating feature data corresponding to characteristics of a frame constituting a moving image based on the moving image data received from the moving image distribution device, and transmission for transmitting the feature data generated in the data generation step. Determination data for determining whether or not the video is inappropriate is stored, including a step, an image processing step for performing image processing on the video data, and a video playback step for playing back a video based on the video data Determination result information indicating that the moving image data corresponding to the feature data is inappropriate when the storage means and the determination data matching the feature data transmitted in the transmission step are stored in the storage means When determination result information indicating that the video data is inappropriate is transmitted from a management terminal including an information transmission means for transmission, image processing is performed. In step, the image processing on the video data, a video reproducing step, characterized in that it reproduces video based on video data subjected to image processing in the image processing step.

  The moving image monitoring program according to the present invention includes a data generation process for generating feature data corresponding to the characteristics of a frame constituting a moving image based on the moving image data received from the moving image distribution apparatus, and the feature data generated by the data generation process. To determine whether or not the video is inappropriate by executing a transmission process for transmitting video, an image processing process for performing image processing on the video data, and a video playback process for playing a video based on the video data That the moving image data corresponding to the feature data is inappropriate when the storage means storing the data for use and the determination data matching the feature data transmitted in the transmission step are stored in the storage means. When determination result information indicating that the video data is inappropriate is transmitted from a management terminal including an information transmission means for transmitting the determination result information indicating In image processing, thereby subjected to image processing to moving image data, a video playback process, characterized in that to play the video based on the video data subjected to image processing in the image processing step.

  ADVANTAGE OF THE INVENTION According to this invention, the moving image delivered can be monitored, suppressing the increase in processing load and communication load favorably.

It is a block diagram which shows the structural example of the moving image monitoring system of the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the client terminal in the moving image monitoring system of the 1st Embodiment of this invention, and a management server. It is a flowchart which shows operation | movement of the client terminal in the moving image monitoring system of the 1st Embodiment of this invention. It is a flowchart which shows operation | movement of the management server in the moving image monitoring system of the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the moving image monitoring system of the 2nd Embodiment of this invention.

Embodiment 1. FIG.
A moving image monitoring system 100 according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of a moving image monitoring system 100 according to the first embodiment of this invention. As shown in FIG. 1, the moving image monitoring system 100 according to the first embodiment of the present invention includes a client terminal 200 and a management server 300. As shown in FIG. 1, the client terminal 200 and the management server 300 are connected to a communication network 500 to which a moving image distribution server 400 is connected. Note that the communication network 500 is, for example, the Internet. In the example illustrated in FIG. 1, the moving image distribution server 400 and the management server 300 are connected to the client terminal 200 via one communication network 500, but the moving image distribution server 400 and the management server 300 are different from each other. The client terminal 200 may be connected via a network.

  FIG. 2 is a block diagram illustrating a configuration example of the client terminal 200 and the management server 300 in the moving image monitoring system 100 according to the first embodiment of this invention. In FIG. 2, the communication network 500 is omitted. As shown in FIG. 2, the client terminal 200 in the moving image monitoring system 100 according to the first embodiment of the present invention includes a redirect process execution unit 210 and a reproduction process execution unit 220.

  As shown in FIG. 2, the redirect process execution unit 210 includes a moving image data acquisition unit 211, a feature data conversion unit 212, and a feature data transmission unit 213.

  The moving image data acquisition unit 211 receives moving image data from the moving image distribution server 400 via the communication network 500. Then, the moving image data acquisition unit 211 inputs the received moving image data to the feature data conversion unit 212 and the reproduction processing execution unit 220. The feature data conversion unit 212 generates feature data corresponding to the feature of the moving image data input by the moving image data acquisition unit 211 and inputs the generated feature data to the feature data transmission unit 213. The feature data transmission unit 213 transmits the feature data input by the feature data conversion unit 212 to the management server 300. Note that the moving image data may be multimedia data including other types of data such as audio data and character data.

  The feature data will be described. The feature data is data indicating the feature amount of an image that is a target for generating the feature data and the data amount of the feature amount. In this example, in order to generate feature data, the feature data conversion unit 212 first selects, for example, each frame corresponding to a timing that becomes an interval of 1 second during playback in the video data input by the video data acquisition unit 211. To do. Then, the feature data conversion unit 212 extracts the feature amount of the selected frame, and generates feature data indicating the feature amount and the data amount of the feature amount. Note that the feature data conversion unit 212 performs feature amount extraction based on, for example, an algorithm of SIFT (Scale-Invariant Feature Transform). Further, the feature data conversion unit 212 may be configured to extract a local feature amount by a method described in Patent Document 3, for example. According to such a configuration, the amount of feature data can be further reduced.

  As shown in FIG. 2, the reproduction processing execution unit 220 includes an inspection result receiving unit 221, a moving image processing unit 222, and an input / output processing unit 223.

  The inspection result receiving unit 221 determines whether or not the moving image data of the moving image including the inappropriate subject transmitted by the management server 300 via the communication network 500 (hereinafter simply referred to as inappropriate moving image data). The test result information shown is received. When the inspection result information received by the inspection result receiving unit 221 indicates that the moving image data is not inappropriate, the moving image processing unit 222 sends the moving image data acquisition unit of the redirect processing execution unit 210 to the input / output processing unit 223. The moving image data inputted by 211 is inputted. In addition, when the inspection result information indicates that the moving image data is inappropriate, the moving image processing unit 222 performs image processing on the moving image data input by the moving image data acquisition unit 211, and sends it to the input / output processing unit 223. Then, input the moving image data after image processing. For example, an input device (not shown) such as a keyboard and an output device (not shown) such as a display and a speaker are connected to the input / output processing unit 223. The input / output processing unit 223 transmits transmission request information requesting transmission of a moving image to the moving image distribution server 400 according to an operation performed on the input device by the user, or a moving image based on the moving image data input by the moving image processing unit 222. Is input to the output device and played back.

  The client terminal 200 is, for example, a computer mounted with a CPU (Central Processing Unit) that executes processing according to program control, a mobile communication terminal, or the like.

  As shown in FIG. 2, the management server 300 in the moving image monitoring system 100 according to the first embodiment of the present invention includes a feature data receiving unit 310, a subject database 320, a feature data collating unit 330, and an inspection result transmitting unit 340. .

  The feature data receiving unit 310 receives the feature data transmitted by the feature data transmitting unit 213 of the client terminal 200 via the communication network 500 and inputs the feature data to the feature data matching unit 330. In the subject database 320, feature amounts corresponding to inappropriate images (for example, subjects) are registered in advance. The feature data matching unit 330 determines whether or not a feature amount that matches the feature amount indicated by the feature data input by the feature data transmission unit 213 is registered in the subject database 320. When the feature data matching unit 330 determines that a feature amount that matches the feature amount indicated by the feature data is registered in the subject database 320, the feature data matching unit 330 performs the following processing. That is, the feature data matching unit 330 indicates that the image is an inappropriate image, and the position indicating the subject range of the feature amount registered in the subject database 320 in the frame in which the feature amount indicated by the feature data is extracted. Test result information including the information is generated. That is, the position information indicates an inappropriate subject range in the frame. Then, the feature data matching unit 330 inputs the generated inspection result information to the inspection result transmission unit 340. Note that the feature data collating unit 330 specifies, for example, a rectangular range as an inappropriate subject range, and generates position information corresponding to coordinates indicating each of the four corners of the specified range.

  For example, the image recognition service GAZIRU (registered trademark) employs the feature data matching unit 330 to determine whether or not a feature value that matches the feature value indicated by the feature data is registered in the subject database 320. The recognized image recognition technology is used.

  Next, the operation of the moving image monitoring system 100 according to the first embodiment of the present invention will be described. FIG. 3 is a flowchart illustrating the operation of the client terminal 200 in the moving image monitoring system 100 according to the first embodiment of this invention. As illustrated in FIG. 3, the moving image data acquisition unit 211 of the redirect process execution unit 210 of the client terminal 200 receives moving image data from the moving image distribution server 400 via the communication network 500 (step S101). Then, the moving image data acquisition unit 211 inputs the moving image data received in step S101 to the feature data conversion unit 212 and the moving image processing unit 222 of the reproduction processing execution unit 220.

  Based on the moving image data input by the moving image data acquisition unit 211, the feature data conversion unit 212 sequentially selects frames that are 1-second intervals at the time of reproduction, and sequentially generates feature data corresponding to the characteristics of the selected frames (step). S102). Then, the feature data conversion unit 212 sequentially inputs the generated feature data to the feature data transmission unit 213.

  The feature data transmission unit 213 transmits the feature data input by the feature data conversion unit 212 to the management server 300 (step S103).

  Further, when the inspection result receiving unit 221 transmits the inspection result information corresponding to the feature data transmitted in the process of step S103 and received via the communication network 500 (Y in step S104), the following Perform the process. That is, the feature data transmission unit 213 starts transmitting feature data to the management server 300 based on a frame subsequent to the frame corresponding to the feature data transmitted in step S103 (step S105).

  When the inspection result information received by the inspection result receiving unit 221 indicates that the moving image data is not inappropriate (N in Step S106), the moving image processing unit 222 sends the moving image data acquisition unit to the input / output processing unit 223. The moving image data input by 211 is input, and the process proceeds to step S108.

  When the inspection result information received by the inspection result receiving unit 221 in the process of step S104 indicates that the moving image data is inappropriate (Y in step S106), the moving image processing unit 222 displays the moving image data acquisition unit 211. Is subjected to image processing (step S107), the image data after image processing is input to the input / output processing unit 223, and the process proceeds to step S108.

  An image process performed on the moving image data by the moving image processing unit 222 based on the inspection result information will be described. In this example, the inspection result information indicating that the moving image data is inappropriate includes position information. The position information indicates a range in which an inappropriate subject is reproduced when the moving image data is reproduced. Therefore, the moving image processing unit 222 performs image processing for arranging the substitute image in the range indicated by the position information on the moving image data. Note that, for example, a cvCopy function in OpenCV is used for image processing for arranging a substitute image in a range indicated by position information in moving image data.

  The input / output processing unit 223 reproduces a moving image based on the moving image data input by the moving image processing unit 222 (step S108).

  Next, the operation of the management server 300 in the moving image monitoring system 100 according to the first embodiment of this invention will be described. FIG. 4 is a flowchart illustrating the operation of the management server 300 in the moving image monitoring system 100 according to the first embodiment of this invention.

  As shown in FIG. 4, the feature data receiving unit 310 of the management server 300 receives the feature data transmitted by the feature data transmitting unit 213 of the client terminal 200 in the process of step S <b> 103 via the communication network 500 (step S <b> 103). In step S201, the received feature data is input to the feature data matching unit 330.

  The feature data matching unit 330 determines whether or not a feature amount that matches the feature amount indicated by the feature data input by the feature data receiving unit 310 is registered in the subject database 320 (step S202). If the feature data matching unit 330 determines that a feature amount that matches the feature amount indicated by the feature data is registered in the subject database 320 (Y in step S202), the moving image data is inappropriate. Is determined, and inspection result information including the position information and indicating that the moving image data is inappropriate is generated (step S203). Further, when the feature data matching unit 330 determines that a feature amount that matches the feature amount indicated by the feature data is not registered in the subject database 320 (N in step S202), the moving image data is not inappropriate. Judgment and inspection result information indicating that the moving image data is not inappropriate are generated (step S204).

  The inspection result transmitting unit 340 transmits the inspection result information generated by the feature data matching unit 330 in the process of step S203 or step S204 to the client terminal 200 (step S205).

  According to the present embodiment, the client terminal 200 generates feature data corresponding to the feature of the moving image data received from the moving image distribution server 400 and transmits it to the management server 300. Then, the management server 300 determines whether or not the video data is inappropriate based on the transmitted feature data, and transmits inspection result information corresponding to the determination result to the client terminal 200. The client terminal 200 performs image processing on the moving image data according to the transmitted inspection result information, and reproduces the moving image after the image processing. Accordingly, since moving image data is not transmitted and received between the client terminal 200 and the management server 300, while suppressing an increase in the processing load on the client terminal 200 and the management server 300 and the increase in the communication load on the communication network 500. The moving image data distributed from the moving image distribution server 400 can be monitored.

  Further, according to the present embodiment, when the management server 300 determines that the video data is inappropriate, the management server 300 transmits position information corresponding to the inappropriate subject range to the client terminal 200. Then, the client terminal 200 performs image processing on the moving image data based on the transmitted position information and reproduces the moving image. Therefore, according to the present embodiment, inappropriate reproduction of the subject is prevented. Further, it is possible to perform image processing only on an inappropriate subject range and reproduce the image while leaving the remaining portion unchanged.

  Furthermore, according to the present embodiment, the feature amount to be compared with the feature data for determining that the video data is inappropriate is registered in the subject database 320 of the management server 300. Therefore, the moving image data distributed to the plurality of client terminals 200 can be monitored by updating the registered content of the subject database 320. In addition, inappropriate reproduction of a subject in the plurality of client terminals 200 can be prevented.

Embodiment 2. FIG.
Next, a moving image monitoring system 10 according to a second embodiment of the present invention will be described with reference to the drawings. FIG. 5 is a block diagram illustrating a configuration example of the moving image monitoring system 10 according to the second embodiment of this invention. As shown in FIG. 5, the moving image monitoring system 10 according to the second embodiment of the present invention includes a reception / playback terminal 20 and a management terminal 30.

  The reception / playback terminal 20 corresponds to the client terminal 200 in the first embodiment of the present invention shown in FIG. 1 and FIG. The management terminal 30 corresponds to the management server 300 in the first embodiment of the present invention shown in FIG. 1 and FIG.

  As shown in FIG. 5, the reception / playback terminal 20 includes a data generation unit 21, a data transmission unit 22, an image processing unit 23, and a moving image playback unit 24.

  The data generation unit 21 corresponds to the feature data conversion unit 212 in the first embodiment of the present invention shown in FIG. The data transmission unit 22 corresponds to the feature data transmission unit 213 in the first embodiment of the present invention shown in FIG. The image processing unit 23 corresponds to the moving image processing unit 222 in the first embodiment of the present invention shown in FIG. The moving image reproducing unit 24 corresponds to the input / output processing unit 223 in the first embodiment of the present invention shown in FIG.

  The data generation unit 21 generates feature data corresponding to the characteristics of the frames constituting the moving image based on the moving image data received from the moving image distribution device (corresponding to the moving image distribution server 400 shown in FIGS. 1 and 2). The data transmission unit 22 transmits the feature data generated by the data generation unit 21. The image processing unit 23 performs image processing on the moving image data. The moving image reproduction unit 24 reproduces a moving image based on the moving image data.

  As shown in FIG. 5, the management terminal 30 includes a storage unit 32 and an information transmission unit 34.

  The storage unit 32 corresponds to the subject database 320 in the first embodiment of the present invention shown in FIG. The information transmission unit 34 corresponds to the inspection result transmission unit 340 in the first embodiment of the present invention shown in FIG.

  The storage unit 32 stores determination data for determining whether the moving image is inappropriate. The information transmission unit 34 has inappropriate video data corresponding to the feature data when determination data matching the feature data transmitted by the data transmission unit 22 of the reception / playback terminal 20 is stored in the storage unit 32. The determination result information indicating this is transmitted to the reception / playback terminal 20.

  When determination result information indicating that the moving image data is inappropriate is transmitted, the image processing unit 23 performs image processing on the moving image data, and the moving image reproduction unit 24 performs image processing on the image processing unit 23. Play a video based on the applied video data.

  According to the present embodiment, it is possible to monitor a distributed video while favorably suppressing an increase in processing load and communication load.

DESCRIPTION OF SYMBOLS 10,100 Movie monitoring system 20 Reception playback terminal 21 Data generation part 22 Data transmission part 23 Image processing part 24 Video reproduction part 30 Management terminal 32 Storage part 34 Information transmission part 200 Client terminal 210 Redirection process execution part 211 Video data acquisition part 212 Feature data conversion unit 213 Feature data transmission unit 220 Playback processing execution unit 221 Inspection result reception unit 222 Movie processing unit 223 Input / output processing unit 300 Management server 310 Feature data reception unit 320 Subject database 330 Feature data collation unit 340 Inspection result transmission unit 400 Video distribution server 500 Communication network

Claims (7)

Data generating means for generating feature data corresponding to the characteristics of the frames constituting the moving image based on the moving image data received from the moving image distribution device;
Data transmitting means for transmitting the feature data generated by the data generating means;
Image processing means for performing image processing on the moving image data;
A receiving and reproducing terminal including a moving image reproducing means for reproducing a moving image based on the moving image data, and a storage means in which determination data for determining whether or not the video is inappropriate is stored;
When the determination data that matches the feature data transmitted by the data transmission means of the reception / playback terminal is stored in the storage means, the moving image data corresponding to the feature data is inappropriate. A management terminal including information transmission means for transmitting the determination result information to the reception reproduction terminal;
When determination result information indicating that the video data is inappropriate is transmitted,
The image processing means performs image processing on the moving image data,
The moving image monitoring system, wherein the moving image reproduction unit reproduces a moving image based on the moving image data on which the image processing unit has performed the image processing. The management terminal determines whether the determination data that matches the feature data generated by the data generation unit of the reception / playback terminal is stored in the storage unit, and matches the feature data in the determination And determining means for generating position information indicating an inappropriate subject range in the moving image according to the feature data when it is determined that the determination data is stored in the storage means,
The information transmission means transmits determination result information including the position information to the reception / playback terminal when the determination means generates the position information,
The moving image monitoring system according to claim 1, wherein the image processing means of the reception / playback terminal performs the image processing on the inappropriate subject range based on the position information. The moving image monitoring system according to claim 2, wherein the image processing unit performs the image processing to replace the inappropriate subject range with another image based on the position information. The reception / playback terminal includes reception means for receiving the determination result information transmitted by the information transmission means of the management terminal,
The data transmission unit transmits feature data of a frame subsequent to the frame according to the feature data when the reception unit receives the determination result information according to the transmitted feature data. The moving image monitoring system according to claim 3. The moving image monitoring system according to any one of claims 1 to 4, wherein the data generation unit generates feature data in accordance with a local feature amount of the frame. A data generation step of generating feature data corresponding to the characteristics of the frames constituting the video based on the video data received from the video distribution device;
A transmission step of transmitting the feature data generated in the data generation step;
An image processing step of performing image processing on the moving image data;
A video playback step for playing a video based on the video data,
Storage means for storing determination data for determining whether or not the video is inappropriate is stored, and the determination data that matches the feature data transmitted in the transmission step is stored in the storage means. A determination result indicating that the moving image data is inappropriate from a management terminal including information transmission means for transmitting determination result information indicating that the moving image data corresponding to the feature data is inappropriate When information is sent,
In the image processing step, image processing is performed on the moving image data,
A moving image monitoring method, wherein the moving image reproduction step reproduces a moving image based on the moving image data subjected to the image processing in the image processing step. On the computer,
A data generation process for generating feature data corresponding to the characteristics of the frames constituting the video based on the video data received from the video distribution device;
A transmission process for transmitting the feature data generated by the data generation process;
Image processing for performing image processing on the moving image data;
And a video playback process for playing a video based on the video data,
Storage means for storing determination data for determining whether or not the video is inappropriate is stored, and the determination data that matches the feature data transmitted in the transmission process is stored in the storage means. A determination result indicating that the moving image data is inappropriate from a management terminal including information transmission means for transmitting determination result information indicating that the moving image data corresponding to the feature data is inappropriate When information is sent,
In the image processing, the moving image data is subjected to image processing,
A moving image monitoring program for reproducing a moving image based on moving image data subjected to the image processing in the image processing processing in the moving image reproduction processing.

Images