JP2018033103A - Frame rate determination device, frame rate determination method and computer program for frame rate determination - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a frame rate determination device capable of determining the frame rate of video data according to medium processing to be applied.SOLUTION: A frame rate determination device has a frame rate lowering section (21) for generating a low rate video data by lowering the frame rate of video data obtained from input devices (2-1 through 2-n), a medium processing section (23) for acquiring first processing results of each picture of executing predetermined medium processing for each picture of video data obtained during a predetermined period, and second processing results of each picture of executing predetermined medium processing for each picture of low rate video data obtained during a predetermined period, and a judgment section (24) determining a frame rate set for the video data according to the difference of first number of times of change when the first processing results change in the acquisition time order of each picture, and second number of times of change when the second processing results change.SELECTED DRAWING: Figure 3

Description

  The present invention relates to, for example, a frame rate determination device, a frame rate determination method, and a frame rate determination computer program that determine the frame rate of moving image data to be subjected to media processing.

  It has been studied to transmit media data such as moving image data generated by an input device such as a video camera to a server via a network and process the media data at the server. In such a case, generally, the higher the frame rate of moving image data, the larger the amount of data. Therefore, if the frame rate of moving image data is excessive, the load on the network or the processing load on the server becomes larger than necessary. Become. In particular, as the number of input devices increases, the network load and server processing load increase. For this reason, it is preferable that the frame rate of the moving image data obtained by each input device is adjusted so that the frame rate is appropriately lowered within a range where the result of media processing on the moving image data is appropriate.

  On the other hand, when evaluating an object that performs periodic motion, a technique for reducing the data size of the moving image data of the object used for the evaluation while maintaining the image quality necessary and sufficient for the evaluation has been proposed. (For example, refer to Patent Document 1). For example, in the technique disclosed in Patent Document 1, frame image data of a moving image obtained by imaging a myocardial cell sheet is sequentially input, and T frame difference data is generated by detecting a frame difference. Based on the T frame difference data, an actual working period in which the cardiomyocyte sheet is actually moving due to pulsation and a stationary period in which the cardiomyocyte sheet is stationary are determined. In the actual operation period, frame image data is recorded for each frame, and in the still period, thinning is performed and one frame image data is recorded for every predetermined number of frames.

JP 2011-2111278 A

  However, the subject represented in the moving image data to be processed does not always perform a periodic motion. In the technique described in Patent Document 1, since the still period is determined based on the frame difference data, in order to determine a thinning rate when recording moving image data, Difference processing between frames needs to be executed. Therefore, the original frame rate is maintained until difference processing between frames is executed.

  In one aspect, an object of the present invention is to provide a frame rate determination device capable of determining a frame rate of moving image data according to applied media processing.

  In one aspect, the frame rate determination device generates a low rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount from the moving image data obtained from the input device; The first processing result for each picture obtained by executing predetermined media processing on each picture of moving image data obtained during a predetermined period, and each picture of low-rate moving image data obtained during a predetermined period A media processing unit that acquires a second processing result for each picture that has undergone predetermined media processing, a first number of changes in which the first processing result changes in the order of acquisition time of each picture, and a second processing A determination unit that determines a frame rate set for the moving image data in accordance with a difference from a second change count in which the result changes in the order of acquisition time of each picture.

  In another aspect, the frame rate determination method generates low-rate moving image data having a reduced frame rate according to a predetermined rate decrease amount from moving image data obtained from an input device, The first processing result for each picture obtained by executing predetermined media processing on each picture of the moving image data obtained in the above is acquired, and for each picture of the low-rate moving image data obtained during the predetermined period A second processing result for each picture that has undergone predetermined media processing is acquired, the first processing result changes in the order of acquisition time of each picture, and the second processing result is for each picture. Including determining a frame rate set for the moving image data in accordance with a difference from the second change count that changes in the order of acquisition time.

  In yet another aspect, the frame rate determination computer program generates low-rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount from the moving image data obtained from the input device, Each picture of the low-rate moving image data obtained during the predetermined period is obtained by obtaining a first processing result for each picture obtained by performing predetermined media processing on each picture of the moving image data obtained during the predetermined period. The second processing result for each picture for which predetermined media processing has been executed is acquired, and the first processing result changes in the order of acquisition time of each picture, and the second processing result is The computer is configured to determine the frame rate set for the moving image data in accordance with the difference from the second change count that changes in the acquisition time order of each picture. It includes instructions for.

  As one aspect, it is possible to determine the frame rate of moving image data according to applied media processing.

1 is a schematic configuration diagram of a media processing system including a server in which a frame rate determination device is mounted according to one embodiment. It is a schematic block diagram of a server. It is a functional block diagram of the control part of a server. It is a figure which shows an example of a frame rate database. (A) and (b) are comparative examples of media processing results obtained for each picture of moving image data and media processing results obtained for each picture of low-rate moving image data. It is an operation | movement flowchart of a frame rate determination process. (A) is a figure which shows an example of the prior investigation list | wrist of the value obtained as a result of media processing, (b) is a figure which shows an example of the inspection list | wrist of the value of the media processing result obtained during a test | inspection period. is there. (A)-(c) is a figure which respectively shows an example of the group of the media processing specified with the moving image data to which attention is paid.

  Hereinafter, the frame rate determination apparatus will be described with reference to the drawings. This frame rate determination device acquires moving image data from an input device such as a video camera or a surveillance camera or a terminal having the input device. The frame rate determination device is configured such that every time moving image data is obtained, the media specified for each of the moving image data and the low-rate moving image data obtained by reducing the frame rate of the moving image data. Execute the process. This frame rate determination apparatus determines whether the difference between the processing result for each picture of the low-rate moving image data and the processing result for the corresponding picture of the original moving image data is always within the allowable range during the inspection period. If the difference between the processing results is within an allowable range for each picture of the low-rate moving image data, the frame rate determination device changes the processing result for each picture of the original moving image data in the order of acquisition time of each picture. Count the number of times. Similarly, the frame rate determination apparatus counts the number of times the processing result for each picture of the low-rate moving image data changes in the order of acquisition time of each picture. Hereinafter, the number of times the processing result changes is simply referred to as the number of changes. If the difference between the number of changes for the original moving image data and the number of changes for the low-rate moving image data is within an allowable range, the frame rate determination device reduces the set frame rate of the moving image data related to the media processing. Rate The frame rate of moving image data. At this time, the frame rate determination device determines the frame rate set for the moving image data obtained by the input device for each combination of the input device and the designated media processing. Thereby, this frame rate determination apparatus can determine an appropriate frame rate for moving image data in accordance with not only the contents of the designated media processing but also the installation environment of the input apparatus. In this specification, the optimum frame rate or optimum rate for moving image data means the lowest frame rate for moving image data in a range that does not affect the result of media processing applied to the moving image data. Say.

  In the embodiment described below, the frame rate determination device is mounted on a server that can communicate with another device via a communication network, and a moving image obtained by the input device or a terminal having the input device via the communication network. Image data is acquired. Note that the present invention is not limited to this example, and the frame rate determination device may be mounted on, for example, a stand-alone computer connected to one or more input devices.

  FIG. 1 is a schematic configuration diagram of a media processing system including a server in which a frame rate determination device is mounted according to one embodiment. The media processing system 1 includes input devices 2-1 to 2-n (n is an integer of 1 or more) and a server 3. The server 3 is connected to each of the input devices 2-1 to 2-n via a wired or wireless communication network 4 so that they can communicate with each other. Note that the communication network 4 can be, for example, a communication network using an Internet line. Alternatively, the communication network 4 may be a communication network using a dedicated line laid in a specific facility.

  The input devices 2-1 to 2-n are installed in different places, for example. Each input device may be installed outdoors or indoors. Each input device generates moving image data at a predetermined frame rate. Therefore, each input device can be, for example, a video camera or a camera capable of continuous shooting at a predetermined frame rate. Each input device shoots the shooting range to generate moving image data including pictures that are periodically generated at a set frame rate.

  The input devices 2-1 to 2-n may be devices having the same specifications or may be devices having different specifications. For example, the angle of view of the input device 2-1 and the angle of view of the input device 2-2 may be the same or different from each other. Further, for example, the maximum frame rate of moving image data generated by the input device 2-1 and the maximum frame rate of moving image data generated by the input device 2-2 may be the same, or They may be different from each other.

  Each of the input devices 2-1 to 2-n converts the moving image data so that the frame rate of the generated moving image data becomes the designated frame rate, and then converts the moving image data to the communication network 4. To the server 3 via. Note that before the frame rate determination process is performed by the frame rate determination device mounted on the server 3, each input device receives the moving image data so that the frame rate of the moving image data becomes a maximum frame rate that can be set. Generate. Each input device may transmit moving image data having the highest frame rate to the server 3 via the communication network 4. On the other hand, each input device receives a notification of the optimum rate from the server 3 or, if an optimum rate is set by the worker, the moving image data has a moving image so that the frame rate becomes the optimum rate. Convert image data. Each input device may transmit the moving image data converted to the optimum rate to the server 3 via the communication network 4.

  When each input device transmits moving image data to the server 3, the input device or identification information for specifying that the input device has been generated by the input device (hereinafter referred to as a media ID for convenience) is used as moving image data. Together with the server 3.

  Each input device may transmit frame rate information representing the frame rate set for the moving image data to the server 3 together with the moving image data or separately from the moving image data.

In addition, any of the input devices 2-1 to 2-n may be connected to a terminal (not shown). In this case, the input device connected to the terminal transmits the moving image data generated at the maximum frame rate to the terminal. The terminal may convert the moving image data so that the frame rate of the moving image data becomes the designated frame rate, and then transmit the converted data to the server 3 via the communication network 4. At that time, the terminal may attach the media ID to the moving image data.
Alternatively, any of the input devices 2-1 to 2-n may be connected to the communication network 4 via a gateway (not shown). In this case, the input device connected to the gateway transmits the moving image data generated at the maximum frame rate to the gateway, and the gateway sets the frame rate of the moving image data to the specified frame rate. Convert moving image data. The gateway may transmit the converted moving image data to the server 3 via the communication network 4.

  FIG. 2 is a schematic configuration diagram of the server 3 in which the frame rate determination device is mounted. As illustrated in FIG. 2, the server 3 includes a communication interface unit 11, a user interface unit 12, a memory 13, a control unit 14, and a storage device 15. Further, the server 3 may have a peripheral device connection port (not shown).

  The communication interface unit 11 is an example of a notification unit, and includes a communication interface for connecting the server 3 to the communication network 4 and its control circuit. The communication interface unit 11 receives moving image data from each of the input devices 2-1 to 2 -n via the communication network 4, and passes the received moving image data to the control unit 14. Further, the communication interface unit 11 may transmit the set frame rate information of the moving image data received from the control unit 14 to a designated input device among the input devices via the communication network 4. Thereby, the designated input device can automatically set the frame rate of the moving image data generated by the device itself to the frame rate indicated in the set frame rate information.

  The user interface unit 12 includes, for example, operation input devices such as a keyboard and a mouse, and a display device such as a liquid crystal display. Then, for example, the user interface unit 12 outputs an operation signal for starting the frame rate determination process to the control unit 14 in accordance with a user operation. Alternatively, the user interface unit 12 outputs, to the control unit 14, an operation signal for designating media processing applied to moving image data obtained by the input device for each input device in accordance with a user operation. . Note that the user interface unit 12 may be omitted. In this case, for example, various operation signals may be transmitted to the control unit 14 from the communication network 4 or a maintenance terminal (not shown) connected via a peripheral device connection port. .

  The memory 13 is an example of a storage unit, and includes, for example, a readable / writable semiconductor memory and a read-only semiconductor memory. The memory 13 stores the intermediate data generated when the designated media processing is performed on the moving image data, the processing result, or the like for a certain period.

  The control unit 14 is an example of a frame rate determination device, and includes, for example, one or a plurality of processors and their peripheral circuits. Then, the control unit 14 controls the entire server 3. Each time the control unit 14 receives a picture included in the moving image data from any of the input devices 2-1-2-n via the communication interface unit 11, the control unit 14 performs media processing designated for the picture. Run. Further, the control unit 14 executes a frame rate determination process on the moving image data received from the input device 2-1-2-n for the designated input device.

  The storage device 15 is another example of a storage unit and includes, for example, a hard disk device. The storage device 15 may further include other storage devices such as an optical recording medium and its access device. The storage device 15 stores a computer program for executing various media processing executed by the control unit 14, for example. The storage device 15 also performs media processing performed on the moving image data generated by the input device 2-1 to 2-n for each of the input devices 2-1 to 2-n and a frame rate for designating the frame rate of the moving image data. Remember the database. Further, the storage device 15 stores a computer program for executing a frame rate determination process executed by the control unit 14 and various data used in the frame rate determination process. Furthermore, the storage device 15 may store moving image data acquired from each input device for a certain period.

  FIG. 3 is a functional block diagram of the control unit 14 relating to the frame rate determination process. As illustrated in FIG. 3, the control unit 14 includes a frame rate reduction unit 21, a selection unit 22, a media processing unit 23, and a determination unit 24.

  Each of these units included in the control unit 14 is, for example, a functional module realized by a computer program executed on a processor included in the control unit 14. Alternatively, these units included in the control unit 14 may be mounted on the server 3 as one or a plurality of integrated circuits that realize the functions of the units.

  The frame rate reduction unit 21 generates low-rate moving image data by reducing the frame rate of moving image data (hereinafter referred to as focused moving image data) received from an input device that is a target of frame rate determination processing. . For example, the frame rate reduction unit 21 reduces the frame rate by thinning out pictures from moving image data according to a set rate reduction amount.

Further, the frame rate reduction unit 21 may change the rate reduction amount as will be described in detail later.
The frame rate reduction unit 21 outputs the low rate moving image data to the media processing unit 23.

  The selection unit 22 selects media processing to be performed on the moving image data of interest and the corresponding low-rate moving image data. In the present embodiment, the selection unit 22 refers to the frame rate database stored in the storage device 15 and selects media processing to be executed on the moving image data and the corresponding low-rate moving image data.

  FIG. 4 is a diagram illustrating an example of a frame rate database. In each column in the leftmost column of the frame rate database 400, media processing IDs for specifying media processing are registered. In each column in the center column of the frame rate database 400, a media ID for moving image data to be subjected to media processing specified by the media processing ID registered in the left column is registered. In each column of the rightmost column of the frame rate database 400, a frame rate (frame per second, fps) applied to the moving image data specified by the media ID described in the same row as the column is registered. Is done. Note that the frame rate may not be registered for moving image data before the frame rate determination process is applied.

  The selection unit 22 refers to the frame rate database and specifies a media processing ID corresponding to the media ID of the moving image data of interest. Then, the selection unit 22 reads a media processing program corresponding to the identified media processing ID from the storage device 15 and develops it in the memory 13. The selection unit 22 notifies the media processing unit 23 of the memory address where the media processing program is expanded.

  The media processing unit 23 executes media processing designated for the moving image data of interest and the corresponding low-rate moving image data. For example, each time the media processing unit 23 obtains a picture included in the moving image data of interest, the media processing unit 23 gives the picture to the address of the memory 13 where the designated media processing program is expanded. Execute the process. Similarly, each time the media processing unit 23 acquires a picture included in the low-rate moving image data, the media processing unit 23 is designated by giving the picture to the address of the memory 13 where the designated media processing program is expanded. Perform media processing.

  For example, when the designated media processing is processing for detecting a person appearing on a picture, the media processing unit 23 receives, for example, a histogram of Oriented Gradients (HOG) feature quantity as input, and learns for human detection. RealAdaBoost discriminator is used. Then, the media processing unit 23 uses the discriminator to detect a human region that is a region where a person is shown on the picture of interest. In this case, for example, the media processing unit 23 calculates a plurality of HOG feature amounts in a window set on the picture of interest. And the media processing part 23 should just determine whether the window is a human area | region by inputting each HOG feature-value into a RealAdaBoost discriminator. The media processing unit 23 can detect the human region at an arbitrary position on the picture of interest by repeating the above processing while changing the position and size of the window. Then, the media processing unit 23 counts the number of human areas detected on the picture of interest, thereby obtaining the number of people shown in the picture of interest.

  Alternatively, the media processing unit 23 may detect a human region from the picture of interest using another classifier such as a support vector machine or a deep neural network learned for human detection instead of the RealAdaBoost classifier. . Alternatively, the media processing unit 23 may detect the human area from the picture of interest based on another technique for detecting the human area from the image.

  Furthermore, the media processing unit 23 may detect a human area or the like using the picture of interest and the past picture. For example, the media processing unit 23 performs a difference process between the picture of interest and the immediately preceding picture in order to detect a human region in which a moving person is shown, and the absolute value of the difference in luminance value is greater than or equal to a predetermined value. A set of pixels may be detected. The media processing unit 23 detects the pixel set as a human area when the number of pixels included in the pixel set is included in a predetermined range corresponding to the size of the person on the picture. Good. Further, the media processing unit 23 may associate a human area detected from each of a predetermined number of pictures in the past with a human area detected from the picture of interest using tracking processing using an optical flow or the like. Good. Thereby, the media processing unit 23 may determine the number of people who move toward a specific direction (for example, an entrance / exit within the shooting range of the corresponding input device) among the detected people. Further, the media processing unit 23 may identify a human area that has already been detected and a human area that is newly detected in the target human area, and obtain the number of newly detected persons.

  Similarly, when the designated media processing is processing for detecting a human face in a picture, the media processing unit 23 recognizes a classifier such as a RealAdaBoost classifier or a deep neural network learned for face detection. Is used. Then, the media processing unit 23 extracts a feature amount from the window while changing the position and size of the window on the target picture, and inputs the extracted feature amount to the discriminator. What is necessary is just to detect the face area which is the area where the face is reflected. Alternatively, the media processing unit 23 detects a skin color region having a skin color from the picture of interest, and detects a facial region by detecting feature points of each part of the face in the skin color region by, for example, template matching. Also good.

  Further, when the designated media processing is biometric authentication processing based on a person's face shown on the picture, the media processing unit 23 extracts, for example, feature points of each part of the person's face from the picture of interest. . Then, the media processing unit 23 stores, for each of a plurality of registered users, the degree of coincidence between the feature points of each part of the registered user's face and the extracted feature points stored in the storage device 15. Based on this, a matching score is calculated. Note that the media processing unit 23 may calculate a matching score for each registered user in accordance with any of various methods used for face authentication. Then, when the maximum value of the collation score is equal to or greater than a predetermined threshold, the media processing unit 23 determines that the registered user corresponding to the maximum value is reflected in the picture of interest.

  Similarly, the media processing unit 23 executes the designated media processing for the low-rate moving image data corresponding to the moving image data of interest. The media processing unit 23 stores the result of media processing for each picture included in the moving image data of interest and the result of media processing for each picture included in the corresponding low-rate moving image data in the memory 13. . At that time, the media processing unit 23 stores the result of media processing of each picture in the memory 13 together with information indicating the reproduction time of the picture for each of the moving image data and low-rate moving image data of interest.

  For example, when the designated media processing is detection of a person or a face, the media processing unit 23 stores the number of detected people or the number of faces in the memory 13 as a result of the media processing. Further, when the designated media processing is biometric authentication processing based on a face, the media processing unit 23 saves whether or not the authentication is successful and the maximum value of the matching score in the memory 13 as a result of the media processing. To do.

  The determination unit 24 determines a frame rate applied to the moving image data of interest based on the result of media processing on the moving image data of interest and the corresponding low-rate moving image data. In the present embodiment, the determination unit 24 counts the number of changes in which the processing result for each picture of the moving image data within the examination period changes in the order of acquisition time of each picture. Similarly, the determination unit 24 counts the number of times the processing result for each picture of the low-rate moving image data within the examination period changes in the order of acquisition time of each picture. Then, the determination unit 24 determines the frame rate applied to the moving image data of interest according to the difference in the number of changes.

  For example, for each picture included in the low-rate moving image data obtained within a certain inspection period, the determination unit 24 performs the media processing result for the picture and the media processing for the corresponding picture of the moving image data of interest. Compare the results. The determination unit 24 determines that the difference between the processing result of each picture of the low-rate moving image data and the processing result of the corresponding picture of the moving image data of interest is within an allowable range, and the low-rate moving image data and the It is determined whether or not the difference in the number of changes between moving image data is within an allowable range. When the above determination condition is satisfied, the determination unit 24 registers a frame rate corresponding to the current rate decrease amount in the frame rate database as a provisional optimum rate. Then, the determination unit 24 notifies the frame rate reduction unit 21 that the rate reduction amount is further reduced by a certain value. Then, the determination unit 24 performs the same processing as described above on the moving image data and the low-rate moving image data obtained within the examination period after changing the rate decrease amount.

  On the other hand, it is assumed that the difference between the result of media processing for any picture of the low-rate moving image data and the result of media processing for the corresponding picture of the moving image data of interest falls outside the allowable range. In this case, the determination unit 24 determines the provisional optimum rate set at that time as the optimum rate for the set of moving image data and media processing of interest, and uses the frame rate for the set. Set. In addition, it is assumed that the difference between the number of changes in the processing result for the moving image data of interest within the examination period and the number of changes in the processing result for the corresponding low-rate moving image data are out of the allowable range. Also in this case, the determination unit 24 determines that the provisional optimum rate set at that time is the optimum rate for the set of moving image data and media processing of interest, and uses the frame rate for the set. Set as follows. Thereby, the determination part 24 can determine an optimal rate appropriately.

  The determination unit 24 notifies the corresponding input device via the communication interface unit 11 of the information indicating the optimum rate, for example, the value representing the optimum rate itself, or the value representing the amount of rate decrease from the highest frame rate. . Alternatively, the determination unit 24 notifies the terminal or gateway to which the input device is connected via the communication interface unit 11 of information indicating the optimum rate. The determination unit 24 registers the set optimum rate in a column corresponding to a set of media ID and media processing ID corresponding to moving image data of interest in the frame rate database. Further, the determination unit 24 may transmit the set optimum rate to the maintenance terminal together with the corresponding set of media ID and media processing ID.

  It should be noted that the inspection period is the installation of the input device corresponding to the moving image data of interest so that the number of changes in the processing result of the moving image data of interest is at least once, preferably a plurality of times during the inspection period. Set according to the environment. Alternatively, the inspection period may be a period until the number of changes in the processing result of the moving image data of interest acquired during the inspection period reaches a predetermined number. The predetermined number of times is set to about 1 to 10 times, for example.

  In addition, for example, when the media processing is human detection, the allowable range for the difference between the processing results is the number of people detected from the picture of interest in the low-rate moving image data and the moving image data of interest. It is set as the difference from the number of people detected from the corresponding picture. The difference is, for example, 0-1. When the media processing is face detection, the allowable range for the difference in processing results is set in the same manner.

  When the media processing is biometric authentication, the allowable range is either the maximum value of the matching score in the target picture of the low-rate moving image data or the maximum value of the matching score in the corresponding picture of the moving image data of interest. It is set to a value obtained by multiplying either one by 0.05 to 0.1. However, it is determined that the difference between the processing results is included in the allowable range that the authentication result for the picture of interest in the low-rate moving image data matches the authentication result of the corresponding picture of the moving image data of interest. It may be added as a condition for.

  The allowable range for the difference in the number of changes is set according to the content of the media processing and the required processing system, and is 0 to 1, for example.

  FIG. 5A and FIG. 5B are comparative examples of media processing results obtained for each picture of moving image data and media processing results obtained for each picture of low-rate moving image data, respectively. It is. In this example, both the allowable range for the difference in the number of changes and the allowable range for the difference in the result of media processing are set to 0.

  For each of FIG. 5A and FIG. 5B, the numeric columns 501 to 502 shown on the upper side are media processing for each picture arranged in the order of acquisition time of moving image data obtained during the examination period. Represents the result. Numerical values columns 511 to 512 shown on the lower side represent the result of media processing for each picture arranged in the order of acquisition time of low-rate moving image data. The result of media processing for a picture of low-rate moving image data and the result of media processing for a corresponding picture of moving image data (that is, a picture obtained at the same time) are shown in the same rectangle.

  In FIG. 5A, for moving image data, the number of changes in the processing result is three (that is, when time t2 → t3, when t4 → t5, and when t6 → t7). Also for the low-rate moving image, the number of processing result changes is three (that is, at time t1 → t3, at t3 → t5, and at t5 → t7). In addition, for each set of a picture of low-rate moving image data and a corresponding picture of moving image data, the result of media processing is the same for low-rate moving image data and moving image data. Therefore, in this case, both the difference in the number of changes and the difference in the result of media processing for each set of corresponding pictures are within the allowable range. Therefore, the determination unit 24 further reduces the frame rate of the moving image data.

  On the other hand, in FIG. 5B as well, the result of media processing is the same between the low-rate moving image data and the moving image data for each set of pictures of the low-rate moving image data and corresponding pictures of the moving image data. . However, for moving image data, the processing result changes four times (that is, when the time t1 → t2, t2 → t3, t4 → t5, t6 → t7). For a low-rate moving image, the number of changes in the processing result is three. Therefore, in this case, the difference in the number of changes is out of the allowable range. Therefore, the determination unit 24 sets the provisional optimum rate for the moving image data as the optimum rate for the moving image data.

  FIG. 6 is an operation flowchart of a frame rate determination process executed by the control unit 14.

  The frame rate reduction unit 21 sets the rate reduction amount to an initial value for the moving image data of interest (step S101). Thereafter, the control unit 14 resets the inspection period (step S102).

  The control unit 14 receives a picture of moving image data of interest via the communication interface unit 11. Then, the selection unit 22 specifies the media processing designated for the moving image data (step S103). Then, the media processing unit 23 executes the designated media processing for the received picture, and stores the obtained processing result in the storage device 15 (step S104).

  On the other hand, the frame rate reduction unit 21 generates low-rate moving image data in which the frame rate of the received moving image data is reduced by a set rate reduction amount (step S105). Then, the media processing unit 23 executes the designated media processing for the picture of the low-rate moving image data, and stores the obtained processing result in the storage device 15 (step S106).

  When the media processing is performed on the picture of the low-rate moving image data, the determination unit 24 determines whether or not the difference between the processing result for the picture and the processing result for the corresponding picture of the moving image data is within an allowable range ( Step S107). If the difference is within the allowable range (step S107—Yes), the determination unit 24 determines whether or not the inspection period has ended (step S108). If the inspection period has not ended (No at Step S108), the control unit 14 repeats the processes after Step S103.

  On the other hand, if the inspection period has ended in step S108 (step S108-Yes), the determination unit 24 counts the number of changes C1 of the processing result for the moving image data of interest during the inspection period. Further, the determination unit 24 counts the number of changes C2 of the processing result for the low-rate moving image data during the examination period (step S109). Then, the determination unit 24 determines whether or not the difference between the number of changes C1 and the number of changes C2 is within an allowable range (step S110).

  If the difference between the number of changes C1 and the number of changes C2 is within the allowable range (step S110—Yes), the determination unit 24 sets the frame rate corresponding to the current rate decrease amount to a temporary optimal rate. Then, the determination unit 24 writes the provisional optimum rate in the column of the frame rate database corresponding to the set of moving image data and media processing of interest. The frame rate reduction unit 21 increases the current rate reduction amount by a certain value (step S111). Then, the control part 14 repeats the process after step S102.

  On the other hand, when the difference between the processing results is out of the allowable range in step S107 (step S107-No), the determination unit 24 sets the provisional optimum rate to the optimum rate for the moving image data of interest (step S107). S112). Similarly, when the difference between the number of changes C1 and the number of changes C2 is out of the allowable range in step S110 (step S110), the determination unit 24 uses the provisional optimum rate as the optimum rate for the moving image data of interest. (Step S112). Then, the control unit 14 ends the frame rate determination process.

  As described above, this frame rate determination device is capable of processing both moving image data obtained from an input device and low-rate moving image data obtained by reducing the frame rate of moving image data. Perform the same media processing. The frame rate determination apparatus repeatedly reduces the frame rate of moving image data within a range in which the difference in the number of changes in processing results between moving image data and low-rate moving image data during the inspection period is within an allowable range. Determine the optimal rate. As a result, this frame rate determination device can optimize the moving image data according to the applied media processing for each combination of the input device that generates moving image data and the media processing, regardless of the installation environment of the input device. The frame rate can be determined. The frame rate determination device reduces the frame rate of moving image data in accordance with the optimal frame rate of the input device, so that the load on the communication network and the server on which the frame rate determination device is mounted The processing load can be reduced.

  In addition, for a set of input device and media processing subject to frame rate determination processing, the optimum rate for the set of media processing included in the set and another input device has already been registered in the frame rate database. There is. Therefore, as a modified example, the frame rate reduction unit 21 sets the initial rate reduction amount according to the optimum rate for the media processing registered in the frame rate database in step S101 of the operation flowchart shown in FIG. A value may be set. And the control part 14 may perform a frame rate determination process similarly to said embodiment. In this case, the control unit 14 can shorten the period required for the frame rate determination process.

  In this modification, in some cases, the rate decrease amount set as the initial value is excessive, and even with the initial value, the number of changes in the processing result for moving image data and the processing result for low-rate moving image data The difference from the number of changes may deviate from the allowable range. In such a case, the control unit 14 may reduce the rate reduction amount by a certain value in step S111 of the operation flowchart shown in FIG. When the difference between the number of changes in the processing result for the moving image data and the corresponding number of changes in the processing result for the low-rate moving image data falls within an allowable range, the control unit 14 decreases the rate at that time. The frame rate corresponding to the amount may be set as the optimum rate. At this time, for each picture of the low-rate moving image data, the control unit 14 indicates that the difference between the processing result of the picture and the processing result of the corresponding picture of the moving image data is within an allowable range. You may add to the conditions which make the frame rate according to the amount of fall the optimal rate.

  In addition, the value of the media processing result obtained for the moving image data of interest obtained in the inspection period may be biased to a part of the distribution range of values obtained as a result of the media processing. In such a case, it is not known whether the difference between the processing result for the picture included in the low-rate moving image data and the processing result for the corresponding moving image data is within the allowable range for the entire distribution range of the value. . Therefore, according to another modification, the control unit 14 saves the processing results obtained during a certain period in advance in the preliminary survey list before starting the adjustment of the frame rate for the moving image data of interest. It may be left. The fixed period is set according to the installation environment of the input device that generates the moving image data of interest, for example, and is set for several hours or several days, for example. Then, the determination unit 24 selects one of the processing results for the moving image data obtained during the examination period among the plurality of values registered in the preliminary survey list in step S108 in the operation flowchart shown in FIG. Find the percentage of those that match the value of. When the ratio becomes a predetermined ratio (for example, 0.8 to 0.9) or more, the determination unit 24 may determine that the inspection period has ended.

  FIG. 7A is a diagram showing an example of a preliminary survey list showing a list of values obtained as a result of media processing, and FIG. 7B is a list of values of media processing results obtained during the examination period. It is a figure which shows an example of the test | inspection list | wrist which shows. The left column of the preliminary survey list 700 represents the media ID that identifies the moving image data of interest, and the central column of the preliminary survey list 700 indicates the media processing ID that identifies the media processing specified for the moving image data of interest. Represent. Each value included in the right column of the pre-investigation list 700 represents the result of media processing on the picture of the moving image data of interest obtained during the pre-investigation period. Similarly, the left column of the examination list 710 represents a media ID that identifies moving image data of interest, and the middle column of the examination list 710 represents a media processing ID that identifies media processing designated for the moving image data of interest. Represents. Each value included in the right column of the inspection list 710 represents the result of media processing for the picture of the moving image data of interest obtained during the inspection period. In the right column of the preliminary survey list 700 and the inspection list 710, the value on the right side indicates a new processing result.

  In this example, in the inspection list 710, values corresponding to all the processing result values included in the preliminary survey list 700 are obtained when the values of the 10th processing result are obtained. Therefore, the control unit 14 may determine that the inspection period has ended when the value of the 10th processing result is obtained.

  According to this modification, the frame rate determination apparatus performs processing of low-rate moving image data corresponding to the processing result of the moving image data for the entire distribution range of the processing result value of the media processing for the moving image data of interest. It can be checked whether the results match. Therefore, according to this modification, the frame rate determination apparatus can determine the optimum rate for the entire distribution range of values of the media processing result for the moving image data of interest. In addition, according to this modification, the frame rate determination device can end the inspection period if a value equal to or greater than a predetermined ratio among a plurality of values registered in the preliminary survey list is obtained, so that the fixed period set in advance The inspection period may be shorter than the inspection period. Therefore, in some cases, the frame rate determination apparatus can reduce the time required to obtain the optimum frame rate for the set of moving image data and media processing of interest.

  In addition, a plurality of media processes may be designated for moving image data of interest. In this case, it is preferable that the processing result of the media processing unit to be stored at the time of executing the frame rate determination processing is determined according to how the plurality of media processing is applied.

  FIG. 8A to FIG. 8C are diagrams illustrating examples of sets of moving image data of interest and designated media processing. In the example shown in FIG. 8A, the media processing set 801 includes three media processing A, B, and C. Each media processing A, B, and C is applied in series to the moving image data of interest, and finally one processing result is obtained. Therefore, in this case, at the time of executing the frame rate determination process, the media processing unit 23 may save one processing result finally obtained for each of the moving image data of interest and the corresponding low-rate moving image data. .

  In this example, for example, the media processing A is processing for detecting an object moving from moving image data. For example, SIFT feature points are detected for each picture included in the moving image data, and the detected feature points are detected. A moving object is detected by applying a tracking process to. Further, the media processing B is, for example, processing for detecting a face, and the face detection processing as described above is applied to an area on a picture in which a moving object detected as a processing result of the media processing A is captured. By detecting the face. The media processing C is, for example, gender determination processing, and the feature amounts of characteristic parts such as eyes, nose and mouth obtained from the processing result of the media processing B from the region where the face is reflected on the picture are obtained. The gender determination result is obtained by calculating and applying principal component analysis to the calculated feature amount.

  In the example shown in FIG. 8B, the media processing set 802 includes three media processing A, B, and C. In this example, media processing A is first applied to moving image data of interest, and media processing B and media processing C are applied in parallel to the obtained processing result. Therefore, two processing results are finally obtained for one input moving image data. Therefore, in this case, at the time of executing the frame rate determination process, the media processing unit 23 connects the two finally obtained processing results for each of the moving image data of interest and the corresponding low-rate moving image data. Save the data.

In the example shown in FIG. 8C, the media processing set 803 includes five media processing A, B, C, D, and E. In this example, two pieces of moving image data are designated as moving image data of interest. First, media processing A is applied to one of the moving image data of interest, and media processing B is applied to the other of the moving image data of interest. Then, the processing result of the media processing A and the processing result of the media processing B are respectively input to the media processing C, and the media processing C outputs one processing result. The processing results output from the media processing C are input to the media processing D and E, and are processed in parallel by the media processing D and the media processing E. Therefore, two processing results are finally obtained for the two input moving image data. Therefore, in this case, at the time of executing the frame rate determination process, the media processing unit 23 saves one of the two moving image data of interest by connecting the two finally obtained processing results into one data. To do. Further, the media processing unit 23 saves the other one of the two moving image data of interest by connecting two finally obtained processing results into one data. In the example shown in FIG. 8C, one of the moving image data input to the media processing A is stored by connecting the processing result of the media processing D and the processing result of the media processing E. Similarly, for the other moving image data input to the media processing B, a result obtained by connecting the processing result of the media processing D and the processing result of the media processing E is stored.
In this way, by determining the processing result to be stored, the control unit 14 can determine whether the moving image data of interest and the low-rate moving image data corresponding to the moving image data of interest correspond to each set of the moving image data of interest and the designated media processing. The difference in processing results can be evaluated appropriately.

  Note that a plurality of media processes may be separately performed on moving image data obtained from one input device. Alternatively, the same media processing may be performed on moving image data obtained from a plurality of input devices. In this case, for each combination of the moving image data and the media processing, the control unit 14 executes any one of the processes in the above-described embodiment and the modified example, and sets an optimal frame rate for the moving image data. do it. When a plurality of media processes are separately executed for one moving image data, the optimum rate may be different for each media process. In such a case, the frame rate set for the moving image data may be the highest frame rate among the optimum rates set for each media process.

  Further, depending on the communication bandwidth of the communication network 4, the processing capability of the server 3, or the transmission frequency of moving image data from the input device, the frame rate of moving image data transmitted from the input device is higher than the optimum rate. It may be possible. Therefore, for example, when the rate reduction amount is an initial value, the control unit 14 has a difference in the number of changes in the processing result between the moving image data of interest and the corresponding low-rate moving image data in the inspection period within an allowable range. Even if the processing results of the pictures match, it is not necessary to increase the rate reduction amount. Then, the control unit 14 may set the frame rate set for the moving image data of interest as the frame rate corresponding to the initial value of the rate decrease amount. Even in this case, the control unit 14 can determine the frame rate set for the moving image data of interest according to the media processing specified for the moving image data.

  Further, in the above-described embodiments and modifications, when the applied media processing is performed only for individual pictures, the pictures of the low-rate moving image data and the corresponding pictures of the moving image data are the same. is there. For this reason, the processing results for these two pictures are the same. Therefore, in this case, the determination unit 24 does not have to consider the difference in processing results for individual pictures. That is, the process of step S107 in the operation flowchart shown in FIG. 6 may be omitted. Then, the determination unit 24 determines whether the difference between the number of changes in the processing result for the moving image data of interest and the number of changes in the processing result for the corresponding low-rate moving image data is within an allowable range. What is necessary is just to determine an optimal rate. In this case, the determination unit 24 does not need to refer to the difference between the processing results of individual pictures, so that the amount of calculation can be reduced.

  Further, in the above-described embodiment and modification, the frame rate determination process is executed by the control unit of one server, but the frame rate determination process may be executed by a plurality of servers. In this case, the processing of each unit of the control unit in the above-described embodiment and modification may be executed by different servers connected to be communicable with each other.

  Further, a computer program for frame rate determination that causes a computer to realize the functions of the server control unit according to the above-described embodiment or modification may be provided in a form stored in a computer-readable medium. Such a medium can be, for example, a magnetic recording medium, an optical recording medium, or a semiconductor memory.

  All examples and specific terms listed herein are intended for instructional purposes to help the reader understand the concepts contributed by the inventor to the present invention and the promotion of the technology. It should be construed that it is not limited to the construction of any example herein, such specific examples and conditions, with respect to showing the superiority and inferiority of the present invention. Although embodiments of the present invention have been described in detail, it should be understood that various changes, substitutions and modifications can be made thereto without departing from the spirit and scope of the present invention.

The following supplementary notes are further disclosed regarding the embodiment described above and its modifications.
(Appendix 1)
A frame rate reduction unit that generates low rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount from the moving image data obtained from the input device;
A first processing result for each picture obtained by executing predetermined media processing on each picture of the moving image data obtained during a predetermined period, and each of the low-rate moving image data obtained during the predetermined period A media processing unit for obtaining a second processing result for each picture obtained by performing the predetermined media processing on a picture;
The difference between the first number of changes in which the first processing result changes in the order of acquisition time of each picture and the second number of changes in which the second processing result changes in the order of acquisition time of each picture. A determination unit that determines a frame rate set for the moving image data in response,
A frame rate determination apparatus.
(Appendix 2)
The determination unit obtains a frame rate set for the moving image data from the input device when a difference between the first change count and the second change count is within a predetermined allowable range. The frame rate determination device according to appendix 1, wherein the frame rate is reduced by the predetermined rate reduction amount from the frame rate of the moving image data at that time.
(Appendix 3)
When the difference between the first number of changes and the second number of changes is within the predetermined allowable range, the frame rate reduction unit increases the predetermined rate reduction amount,
The media processing unit acquires the first processing result for each picture of the moving image data obtained during the predetermined period after the predetermined rate decrease amount is increased, and the corresponding low-rate moving image Obtaining the second processing result for each picture of the image data;
When the difference between the first change count and the second change count for the predetermined period after the predetermined rate decrease amount is increased is within the allowable range, the determination unit The frame rate set for the image data is determined to be a frame rate that is reduced by the predetermined rate reduction amount after the increase from the frame rate of the moving image data obtained from the input device. Frame rate determination device.
(Appendix 4)
The determination unit, when the difference between the first number of changes and the second number of changes for the predetermined period after the predetermined rate decrease amount is outside the predetermined allowable range, The frame rate set for the moving image data is determined to be a frame rate that is reduced by the predetermined rate reduction amount before the increase from the frame rate of the moving image data obtained from the input device, or 2 4. The frame rate determination device according to 3.
(Appendix 5)
A third image obtained by executing the predetermined media processing on each of a plurality of pictures of the moving image data obtained from the input device during a certain period before the predetermined period. A storage unit for storing the processing result of
The determination unit has a ratio of a processing result that matches one of the first processing results for each picture of the moving image data obtained during the predetermined period among a plurality of the third processing results. The frame rate determination apparatus according to any one of appendices 1 to 4, wherein when the ratio is reached, it is determined that the predetermined period has ended.
(Appendix 6)
The frame rate reduction unit generates second low-rate moving image data in which the frame rate is reduced according to a second rate reduction amount from the moving image data obtained from the input device,
The media processing unit performs a second media process on each picture of the moving image data obtained during a second predetermined period, performs a third processing result for each picture, and the second predetermined process. Obtaining a fourth processing result for each picture obtained by performing the second media processing on each picture of the second low-rate moving image data obtained during the period;
The determination unit includes: a third change number in which the third processing result changes in the order of acquisition time of the picture; and a fourth change number in which the fourth process result changes in the order of acquisition time of the picture. The moving image data when the frame rate set for the moving image data when the second media processing is applied is obtained from the input device when the difference is within the predetermined allowable range The frame rate determination device according to any one of appendices 1 to 5, wherein the frame rate is determined to be a frame rate that is reduced from the frame rate by the second rate decrease amount.
(Appendix 7)
The frame rate determination device according to any one of appendices 1 to 6, further comprising a notification unit that notifies the input device of information representing the set frame rate for the moving image data.
(Appendix 8)
The determination unit has the difference between the first change count and the second change count in the predetermined period within a predetermined allowable range, and the low-rate moving image data obtained within the predetermined period When the difference between the second processing result of the picture and the first processing result of the corresponding picture of the moving image data is within a second allowable range, The frame rate determination apparatus according to appendix 1, wherein the frame rate is a frame rate that is reduced by the predetermined rate reduction amount from the frame rate of the moving image data obtained at the time.
(Appendix 9)
For each combination of media processing with any one of the plurality of input devices, the storage device further stores a frame rate set for moving image data obtained by the input device,
When the second frame rate for the combination of the moving image data obtained from the second input device and the predetermined media processing is stored in the storage unit, the frame rate reduction unit calculates the predetermined rate reduction amount. The frame rate determination device according to attachment 1, wherein the frame rate determination device sets the value according to the second frame rate.
(Appendix 10)
From the moving image data obtained from the input device, low-rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount is generated,
Obtaining a first processing result for each picture obtained by performing a predetermined media process on each picture of the moving image data obtained during a predetermined period;
Obtaining a second processing result for each picture for which the predetermined media processing has been performed on each picture of the low-rate moving image data obtained during the predetermined period;
The difference between the first number of changes in which the first processing result changes in the order of acquisition time of each picture and the second number of changes in which the second processing result changes in the order of acquisition time of each picture. In response, a frame rate set for the moving image data is determined.
A frame rate determination method.
(Appendix 11)
From the moving image data obtained from the input device, low-rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount is generated,
Obtaining a first processing result for each picture obtained by performing a predetermined media process on each picture of the moving image data obtained during a predetermined period;
Obtaining a second processing result for each picture for which the predetermined media processing has been performed on each picture of the low-rate moving image data obtained during the predetermined period;
The difference between the first number of changes in which the first processing result changes in the order of acquisition time of each picture and the second number of changes in which the second processing result changes in the order of acquisition time of each picture. In response, a frame rate set for the moving image data is determined.
A computer program for determining a frame rate for causing a computer to execute the above.

DESCRIPTION OF SYMBOLS 1 Media processing system 2-1 to 2-n Input device 3 Server 4 Communication network 11 Communication interface part 12 User interface part 13 Memory 14 Control part 15 Storage apparatus 21 Frame rate reduction part 22 Selection part 23 Media processing part 24 Determination part

Claims (8)

A frame rate reduction unit that generates low rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount from the moving image data obtained from the input device;
A first processing result for each picture obtained by executing predetermined media processing on each picture of the moving image data obtained during a predetermined period, and each of the low-rate moving image data obtained during the predetermined period A media processing unit for obtaining a second processing result for each picture obtained by performing the predetermined media processing on a picture;
The difference between the first number of changes in which the first processing result changes in the order of acquisition time of each picture and the second number of changes in which the second processing result changes in the order of acquisition time of each picture. A determination unit that determines a frame rate set for the moving image data in response,
A frame rate determination apparatus.   The determination unit obtains a frame rate set for the moving image data from the input device when a difference between the first change count and the second change count is within a predetermined allowable range. The frame rate determination device according to claim 1, wherein the frame rate is reduced by the predetermined rate reduction amount from the frame rate of the moving image data at that time. When the difference between the first number of changes and the second number of changes is within the predetermined allowable range, the frame rate reduction unit increases the predetermined rate reduction amount,
The media processing unit acquires the first processing result for each picture of the moving image data obtained during the predetermined period after the predetermined rate decrease amount is increased, and the corresponding low-rate moving image Obtaining the second processing result for each picture of the image data;
When the difference between the first change count and the second change count for the predetermined period after the predetermined rate decrease amount is increased is within the allowable range, the determination unit The frame rate set for the image data is determined to be a frame rate that is reduced by the predetermined rate reduction amount after the increase from the frame rate of the moving image data obtained from the input device. The frame rate determination device described.   The determination unit, when the difference between the first number of changes and the second number of changes for the predetermined period after the predetermined rate decrease amount is outside the predetermined allowable range, The frame rate set for the moving image data is determined to be a frame rate that is reduced by the predetermined rate decrease amount before the increase from the frame rate of the moving image data obtained from the input device. Or the frame rate determination apparatus of 3. A third image obtained by executing the predetermined media processing on each of a plurality of pictures of the moving image data obtained from the input device during a certain period before the predetermined period. A storage unit for storing the processing result of
The determination unit has a ratio of a processing result that matches one of the first processing results for each picture of the moving image data obtained during the predetermined period among a plurality of the third processing results. The frame rate determination device according to any one of claims 1 to 4, wherein when the ratio is reached, it is determined that the predetermined period has ended.   The determination unit has the difference between the first change count and the second change count in the predetermined period within a predetermined allowable range, and the low-rate moving image data obtained within the predetermined period When the difference between the second processing result of the picture and the first processing result of the corresponding picture of the moving image data is within a second allowable range, The frame rate determination apparatus according to claim 1, wherein the frame rate is reduced by the predetermined rate reduction amount from the frame rate of the moving image data obtained at the time. From the moving image data obtained from the input device, low-rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount is generated,
Obtaining a first processing result for each picture obtained by performing a predetermined media process on each picture of the moving image data obtained during a predetermined period;
Obtaining a second processing result for each picture for which the predetermined media processing has been performed on each picture of the low-rate moving image data obtained during the predetermined period;
The difference between the first number of changes in which the first processing result changes in the order of acquisition time of each picture and the second number of changes in which the second processing result changes in the order of acquisition time of each picture. In response, a frame rate set for the moving image data is determined.
A frame rate determination method. From the moving image data obtained from the input device, low-rate moving image data in which the frame rate is reduced according to a predetermined rate reduction amount is generated,
Obtaining a first processing result for each picture obtained by performing a predetermined media process on each picture of the moving image data obtained during a predetermined period;
Obtaining a second processing result for each picture for which the predetermined media processing has been performed on each picture of the low-rate moving image data obtained during the predetermined period;
The difference between the first number of changes in which the first processing result changes in the order of acquisition time of each picture and the second number of changes in which the second processing result changes in the order of acquisition time of each picture. In response, a frame rate set for the moving image data is determined.
A computer program for determining a frame rate for causing a computer to execute the above.

Images