JP5549431B2 - Server apparatus, moving vector detection method and program - Google Patents

Description

  The present invention relates to a server device, a movement vector detection method, and a program for detecting an area moved between two frames.

  With the development of the network, an environment that can connect to the network everywhere has been established. For this reason, a client device that is reduced in function and reduced in weight as much as possible so that it can be used everywhere is prepared, and processing for operations input to the client device is performed by a server device connected via a network. Server systems are becoming practical.

  In such a client-server system, a screen indicating the processing result of the operation input to the client device is generated by the server device. For this reason, in order to display the processing result performed in the server device in the client device, the server device transmits the generated screen information to the client device at predetermined time intervals.

  At this time, depending on the data size of frame data, which is screen data sent at a predetermined interval, and the degree of network congestion, after the operation information is input to the client device, the frame data reflecting this operation information is displayed on the client device. There may be a delay before it is done. In order to reduce this delay, it is necessary to make the data size of the frame data transmitted from the server device to the client device as small as possible.

  Therefore, various methods for reducing the data size of transmitted frame data have been adopted. For example, a method such as differential transmission is used in which only a portion changed from the previously generated frame data is transmitted.

  In the differential transmission method, usually, in the case of movement of the window portion on the frame as shown in FIG. 27, the region changed from the temporally previous frame in the temporally subsequent frame, that is, in FIG. As shown in the figure, the area corresponding to the window part before the movement in the temporally subsequent frame and the area of the window part after the movement are recognized as the difference area, and the data from the difference area is used as the difference data from the server device to the client. Sending to the device. However, when the image of the moving window part does not change, it is not necessary to send the entire data including the window part before moving and the window part after moving as difference data.

  If the image of the window portion does not change, the image of the window portion before movement already stored in the client device may be used as the image of the window portion after movement. In other words, for the window part after movement, the image data of the window part before movement may be copied to the storage position of the window part after movement on the frame data stored in the client device. Then, only the remaining data as shown in FIG. 29 is transmitted as difference data from the server device, and the window portion is moved by rewriting the window portion before movement to the difference data on the frame data stored in the client device. The reproduced image can be reproduced. That is, as shown in FIG. 30, only the position information before the movement of the window part, the information about the movement destination of the window part, and the remaining difference data are transmitted, and the frame data after the movement of the window part is transmitted to the client device. Can be reproduced on the side.

  Here, the position information before the movement of the window portion may be a combination of the coordinates of the upper left end point and the coordinates of the lower right end point of the window portion, or the coordinates of the upper left end point and the size of the window portion (the length of the window portion in the horizontal direction). And the length in the vertical direction). That is, it is only necessary to transmit four numerical data as position information before the window portion is moved. Further, as the information on the movement destination of the window part, for example, if a vector connecting the coordinates of the upper left end point of the window part before the movement and the coordinates of the upper left end point of the window part after the movement, that is, information on the movement vector is transmitted. good. That is, as the information on the movement destination of the window portion, it is only necessary to transmit two numerical data that are coordinate values of the movement vector. Therefore, instead of transmitting the data of the window part after movement, it is only necessary to transmit six numerical data, and the data size of the data to be transmitted can be considerably reduced.

  For example, Patent Document 1 discloses a method for detecting a moving region such as a window portion, and a method for detecting a moving vector of the region and remaining difference data.

JP 2009-194626 A

  However, the methods so far including the method disclosed in Patent Document 1 are natural moving images in which a moving image transmitted from a server device to a client device is a moving image generated by information captured through a lens such as a camera. It is not considered whether there is an artificial moving image that is a moving image generated only by digital processing of the server device.

  Unlike an artificial video, a natural video has color fluctuations and position fluctuations. In other words, the conventional methods including the method disclosed in Patent Document 1 do not take these fluctuations into account, so that there is a problem in that the accuracy of movement vector calculation is reduced when a natural moving image is handled.

  Therefore, the present invention has been made in view of the above-described problems, and an object thereof is to provide a server device, a movement vector detection method, and a program that detect a movement vector with high accuracy.

  In order to solve the above-described problem, the server device according to the present invention includes a moving image type determining unit that determines whether moving image data is a natural moving image in which color and position fluctuations exist, and a continuous frame constituting the moving image data. By comparing a certain first frame with a second frame, it is possible to move the moving area, which is an image area that exists in both of the two frames and whose position on the frame is moving. Movement vector detection means for detecting a movement vector indicating a magnitude and direction, wherein the movement vector detection means detects the movement vector based on a determination result by the moving image type determination means.

  In the moving vector detection method according to the present invention, the moving image type determining step for determining whether the moving image data is a natural moving image in which color and position fluctuations exist, and the first frame that is a continuous frame constituting the moving image data. The size and direction of movement of the moving area, which is an image area that exists in both of the two frames and whose position on the frame is moving, by comparing the second frame and the second frame A movement vector detection step for detecting a movement vector indicating the movement vector, wherein the movement vector detection step detects the movement vector based on a determination result obtained by the moving image type determination step.

  The program according to the present invention includes a moving image type determining unit that determines whether the moving image data is a natural moving image in which color and position fluctuations exist, and a continuous frame constituting the moving image data. By comparing the first frame with the second frame, the magnitude of movement of the moving area, which is an image area that exists in both of the two frames and whose position on the frame is moving, is It is made to function as a movement vector detection means which detects the movement vector which shows a direction, The said movement vector detection means detects the said movement vector based on the determination result by the said moving image kind determination means.

  According to the present invention, it is possible to accurately detect a movement vector.

It is a figure which shows the structural example of the client server system which concerns on embodiment of this invention. It is a figure which shows the structural example of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the movement of the area | region in two frames. It is a figure which shows the structural example of the movement vector detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the production | generation method of a hash value. It is a figure explaining the hash table produced | generated by the movement vector detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the vector calculated in the movement vector detection part of the server apparatus which concerns on embodiment of this invention. It is a figure which shows the processing operation example in the movement vector detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the vector calculated in the movement vector detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the vector calculated in the movement vector detection part of the server apparatus which concerns on embodiment of this invention. It is a figure which shows the processing operation example in the movement vector detection part of the server apparatus which concerns on embodiment of this invention. It is a figure which shows the structural example of the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the comparison area determined in the movement area detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the area | region compared in the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the area | region compared in the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the comparison performed in the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure which shows the processing operation example in the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure which shows the structural example of the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the initial stage area | region set in the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the comparison area set in the movement area detection part of the server apparatus which concerns on embodiment of this invention. It is a figure which shows the processing operation example in the movement area | region detection part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the process example in the difference data part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the process example in the difference data part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the process example in the difference data part of the server apparatus which concerns on embodiment of this invention. It is a figure explaining the process example in the difference data part of the server apparatus which concerns on embodiment of this invention. It is a figure which shows the processing operation example in the server apparatus which concerns on embodiment of this invention. It is a figure explaining the movement of a window part. It is a figure explaining a difference area. It is a figure explaining a difference area. It is a figure explaining the reproduction | regeneration of the flame | frame by the positional information on a movement area | region, a movement vector, and difference data.

  Next, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

<Configuration example of client server system>
FIG. 1 is a configuration example of a client server system according to an embodiment of the present invention. The client server system according to the present embodiment includes a server device 100 and a client device 200. Server apparatus 100 and client apparatus 200 are connected via a network. In FIG. 1, there is one client device 200, but a plurality of client devices 200 may be used.

<Configuration Example of Server Device 100>
FIG. 2 is a diagram illustrating a configuration example of the server apparatus 100 according to the embodiment of the present invention. The server device 100 according to the present embodiment includes a storage unit 110, a movement vector detection unit 120, a movement region detection unit 130, a difference data generation unit 140, and a transmission unit 150. With such a configuration, the server apparatus 100 compares two temporally continuous frames as shown in FIG. 3, moves the moving area in each frame, and the moving distance and movement of the moving area. A movement vector indicating a direction is detected. Then, difference data is generated using the movement vector, the movement area, and the previous frame in time, and the movement vector information, the movement area information, and the difference data are transmitted to the client apparatus 200. Hereinafter, of the two temporally consecutive frames to be compared, the temporally previous frame is referred to as a pre-movement frame, and the temporally subsequent frame is referred to as an after-movement frame. That is, the server apparatus 100 uses the pre-movement frame and the post-movement frame stored in the storage unit 110 to detect a movement vector by the movement vector detection unit 120, detects a movement region by the movement region detection unit 130, and performs the difference. The data generation unit 140 generates difference data, and the transmission unit 150 transmits the movement vector information, the movement region information, and the difference data to the client device 200.

  The storage unit 110 records a pre-movement frame, a post-movement frame, coordinate values / pixel values of edge points / feature points described later, a hash table, and the like. The number of storage units 110 may be one, or may be prepared for each data to be stored. For example, the storage unit 110 includes a RAM, a hard disk, etc., data that needs to be read and written frequently is stored in the RAM, and data that does not need to be read and written frequently is stored in the hard disk. You may make it do.

<Configuration Example of Moving Vector Detection Unit 120>
The movement vector detection unit 120 detects a movement vector based on the pre-movement frame and the post-movement frame stored in the storage unit 110. FIG. 4 is a diagram illustrating a configuration of the movement vector detection unit 120 of the server device 100 according to the embodiment of the present invention. The movement vector detection unit 120 includes an edge point extraction unit 121, a hash table generation unit 122, a feature point extraction unit 123, a feature point pair extraction unit 124, a vector calculation unit 125, a movement vector determination unit 126, a moving image A seed determination unit 127.

  The edge point extraction unit 121 extracts, as edge points, pixels that have a greater change in pixel value than the surrounding pixels in each of the pre-movement frame and post-movement frame stored in the storage unit 110. There are various ways to extract edge points. For example, when attention is paid to a certain pixel, the pixel value of the pixel is compared with the pixel values of two pixels adjacent in two directions orthogonal to the pixel (for example, left and up), and the difference in pixel value is If it is greater than or equal to a predetermined value, the pixel is extracted as an edge point. At the time of comparison, the sum of differences in the R, G, and B components may be compared, or may be compared in each of the R, G, and B components. When comparing each of the R, G, and B components, the pixel is extracted as an edge point when the difference between any one of the R, G, and B components is equal to or greater than a predetermined value. If the difference between two components is greater than or equal to a predetermined value, the pixel may be extracted as an edge point, or when the difference between all components is greater than or equal to a predetermined value In addition, the pixel may be extracted as an edge point.

  The moving image transmitted from the server device 100 to the client device 200 can be two types of moving images, natural moving images and artificial moving images. The natural moving image is a moving image generated by information captured through a lens such as a camera, for example, and the artificial moving image is a moving image generated only by digital processing such as the server device 100, for example.

  Even if the natural moving image looks the same with the naked eye, it passes through the lens, so the color fluctuates slightly between successive frames. On the other hand, there is no such fluctuation in the artificial video. For this reason, if the same value is used for the natural video and the artificial video as the predetermined value used when extracting the edge point, in the natural video, the point to be extracted as the edge point is caused by the above fluctuation. It may happen that it is not extracted as an edge point.

  Therefore, it is preferable to use different values for the natural video and the artificial video for the predetermined value used when extracting the edge point. For example, the predetermined value used in the natural moving image is made smaller than the predetermined value used in the artificial moving image. By doing so, it is possible to reduce the failure to extract edge points in natural moving images.

  Note that the moving image type determination unit 127 that determines whether the moving image transmitted from the server device 100 to the client device 200 is a natural moving image or an artificial moving image will be described in detail below.

  The hash table generation unit 122 quantifies the edge points extracted by the edge point extraction unit 121 with the hash values shown below, and shows the number of appearances for each hash value, the coordinate value of the edge point having the hash value, and the like Generate a table. The hash table is generated for each of the pre-movement frame and the post-movement frame.

  In order to generate a hash table, the hash table generation unit 122 first generates a hash value for all edge points. Here, there are various ways of generating the hash value. For example, when attention is paid to an edge point, the distance D1 between the edge point and the edge point to the left of the edge point, and the distance D2 between the edge point and the edge point to the left of the edge point , D1 + n × D2 is used as the hash value of the edge point. Here, n is an arbitrary integer. At this time, for example, as shown in FIG. 5, when there are three edge points and the coordinates of the respective edge points are (81, 50), (83, 50), (86, 50), n = If 16, the hash value of the edge point at the coordinates (86, 50) is 3 + 16 × 5 = 83.

  As illustrated in FIG. 6, the hash table generation unit 122 generates, for each hash value, a table that summarizes the number of appearances of the hash value in each frame and the coordinate values of the edge points at which the hash value is generated. At this time, for example, the hash table generation unit 122 may generate the hash values of the edge points in the raster scan order.

  The feature point extraction unit 123 extracts, as feature points, edge points having a hash value whose appearance count is 1 from the hash tables of the pre-movement frame and the post-movement frame generated by the hash table generation unit 123. For example, in the case of the hash table shown in FIG. 6, edge points having coordinate values (86, 50) and (114, 121) from the hash table of the pre-movement frame are Edge points whose coordinate values are (288, 148) and (32, 384) are extracted as feature points.

  The feature point pair extraction unit 124 extracts a feature point pair having the same hash value as a feature point pair from the hash table of the pre-movement frame and the hash table after the movement. For example, in the hash table illustrated in FIG. 6, the feature point pair extraction unit 124 includes the feature point at the coordinates (86, 50) of the pre-movement frame having the same hash value as 83, and the post-movement The feature points at the coordinates (288, 148) of the frame are extracted as feature point pairs.

  That the hash values generated as described above are the same means that the positional relationship between the feature point of the feature point pair, the one left edge point of the feature point, and the two left edge points is before the movement. It is likely that the frame and the post-movement frame are the same. In other words, the feature point of the feature point pair, the one left edge point and the two left edge points of the feature point are not moved at all in the pre-movement frame and the post-movement frame, or are the same distance in the same direction. There is a high possibility of moving. Therefore, the feature points of the feature point pair extracted as described above are likely to be points that have not moved at all or points within the moving region.

  The vector calculation unit 125, for each feature point pair extracted by the feature point pair extraction unit 124, the position (coordinate) of the feature point in the pre-movement frame, the position (coordinate) of the feature point in the post-movement frame, The vector which connects is calculated. For example, if it is a feature point pair of the hash table shown in FIG. 6, a vector is calculated as shown in FIG. As described above, since there is a high possibility that the feature point pair includes a point in the movement region, the vector calculated by the vector calculation unit 125 is a movement vector that is a vector indicating movement of the movement region. High possibility.

  The movement vector determination unit 126 is a vector that indicates the movement of the movement region, except for the zero vector (0, 0), among the vectors calculated by the vector calculation unit 125 for each feature point pair. It is determined as a movement vector.

  As described above, the feature point of the feature point pair is highly likely to be a point that has not moved or a point in the moving region. The vectors calculated by the vector calculation unit 125 may include many zero vectors that are vectors indicating movement with respect to feature point pairs that have not moved at all. For this reason, first, the zero vector is removed from the vector calculated by the vector calculation unit 125.

  There may be vectors of various values in the remaining vectors. Among the remaining vectors, there are also vectors calculated from feature point pairs in which the positional relationship between the one left edge point and the two left edge points of the feature points is different between the pre-movement frame and the post-movement frame. However, each such vector should have a different value. Therefore, it is highly likely that the most frequently used vector among the remaining vectors is a movement vector. Therefore, among the remaining vectors, the most frequently occurring vector is determined as the movement vector. For example, in the case of the hash table as shown in FIG. 6, since the vector is calculated as shown in FIG. 7 for each feature point pair, the vector (202, 98) is determined as the movement vector.

  In this way, a movement vector indicating the magnitude and direction of movement of the moving area can be detected. In the above method, the movement vector is detected using the point where the three edge points maintain the same positional relationship in the pre-movement frame and the post-movement frame. Therefore, it is possible to detect the movement vector more reliably than the method using one point or two points.

<Example of Processing Operation in Movement Vector Detection Unit 120>
FIG. 8 is a diagram illustrating an example of a processing operation in the movement vector detection unit 120 of the server device 100 according to the embodiment of the present invention. First, an edge point is extracted by the edge point extraction unit 121 in each of the pre-movement frame and the post-movement frame (S101). Then, the hash table generation unit 122 gives a hash value to each edge point extracted by the edge point extraction unit 121, and has the number of appearances of the hash value in the pre-movement frame and the post-movement frame and the hash value thereof. A hash table showing the coordinate values of the edge points is generated (S102). Next, the feature point extraction unit 123 uses the hash table generated by the hash table generation unit 122 to extract an edge point having a hash value that appears once in each frame as a feature point (S103). ). Next, the feature point pair extraction unit 124 extracts a feature point pair having the same hash value from the hash table of the pre-movement frame and the hash table after the movement as a feature point pair (S104). Next, for each of the feature point pairs extracted by the feature point pair extraction unit 124, the vector calculation unit 125 sets the feature point position (coordinates) in the pre-movement frame and the feature point position (coordinates) in the post-movement frame. Are calculated (S105). Finally, among the vectors calculated by the vector calculation unit 125 for each feature point pair, the movement vector determination unit 126 selects the most frequently occurring vector other than the zero vector (0, 0) and indicates the movement of the movement region. Is determined as a movement vector (S106). With such processing operation, the movement vector detection unit 120 detects a movement vector.

<Consideration for color fluctuation>
In the above, the feature point extraction unit 123 extracts an edge point having a hash value whose appearance count is 1 as a feature point. However, in the natural moving image, as described above, it is difficult to extract edge points due to color fluctuation. Similarly, in the extraction of feature points, there may be fewer points extracted as feature points due to this color fluctuation.

  Therefore, when the moving image type determining unit 127 determines that the moving image transmitted from the server device 100 to the client device 200 is a natural moving image, the feature point extracting unit 123 has an appearance count equal to or less than a predetermined threshold. An edge point having a hash value may be extracted as a feature point. For example, when the predetermined threshold is set to 2, if the hash table shown in FIG. 6 is used, an edge point whose coordinate value is (86, 50) or (114, 121) from the hash table of the pre-movement frame, etc. In addition to edge points where the number of appearances of hash values is 1, edge points where the number of appearances of hash values is 2, such as edge points whose coordinate values are (23, 30) or (531, 321), are also feature points. Extracted. Further, from the hash table of the post-movement frame, in addition to the edge point where the number of appearances of the hash value is 1, such as the edge point whose coordinate value is (288, 148), (32, 384), the coordinate value is (242, 102) and (634, 179) are also extracted as feature points.

  At this time, the feature point pair extraction unit 124 generates a pair from the hash table of the pre-movement frame and the hash table after the movement using feature points having the same hash value, and uses the generated pair as a feature point. Extract as a pair. That is, for a hash value whose appearance count is 1 in both the pre-movement frame and the post-movement frame, the feature point pair extraction unit 124 extracts a feature point pair by the above-described method, For either or both of the post-movement frames, for a hash value whose appearance count is greater than 1, the feature point of the pre-movement frame having the hash value and the feature point of the post-movement frame having the hash value Generate pairs for all combinations between and.

  For example, in the hash table shown in FIG. 6, the hash value 84 has an appearance count of 2 in the pre-movement frame and an appearance count of 1 in the post-movement frame. In such a case, the feature point pair extraction unit 124 pairs the feature point at the coordinates (23, 30) of the pre-movement frame and the feature point at the coordinates (32, 384) of the post-movement frame, and sets the pre-movement frame. The feature points at the coordinates (531, 321) of the image and the feature points at the coordinates (32, 384) of the post-movement frame are paired, and these two pairs are extracted as feature point pairs.

  For example, in the hash table shown in FIG. 6, the hash value 551 has an appearance count of 2 in both the pre-movement frame and the post-movement frame. In such a case, the feature point pair extraction unit 124 pairs the feature point at the coordinates (603, 200) of the pre-movement frame and the feature point at the coordinates (39, 399) of the post-movement frame, and sets the pre-movement frame. The feature point at the coordinates (603, 200) and the feature point at the coordinates (421, 405) of the post-movement frame are paired, and the feature point at the coordinates (39, 399) of the pre-movement frame and the post-movement frame Pair the feature points at the coordinates (39, 399), and pair the feature points at the coordinates (39, 399) of the pre-movement frame with the feature points at the coordinates (421, 405) of the post-movement frame. Four pairs are extracted as feature point pairs.

  The vector calculation unit 125 then, for each feature point pair extracted by the feature point pair extraction unit 124, the position (coordinates) of the feature point in the pre-movement frame and the position (coordinates) of the feature point in the post-movement frame. ) Are calculated. For example, if it is a feature point pair of the hash table shown in FIG. 6, a vector is calculated as shown in FIG. The movement vector determination unit 126 is a vector that indicates the movement of the moving region, except the zero vector (0, 0), among the vectors calculated by the vector calculation unit 125 for each feature point pair. Is determined as a movement vector.

  In this way, it is possible to extract more feature points that are candidates for points in the moving region. For this reason, it is possible to determine the movement vector using more feature points, that is, more samples, and even moving images that cause color fluctuations like natural movies can move accurately. It becomes possible to determine the vector.

  It should be noted that an artificial moving image whose appearance count is not more than a predetermined threshold may be extracted as a feature point. In such a case, the color of the natural video is more likely to fluctuate than that of the artificial video. Therefore, the threshold for the number of appearances in the natural video may be set larger than the threshold for the number of appearances in the artificial video.

<Consideration for position fluctuation>
In addition, in a natural moving image, in addition to the above-described color fluctuation, there are fluctuations in pixel positions between successive frames. Therefore, when the moving image type determining unit 127 determines that the moving image transmitted from the server apparatus 100 to the client device 200 is a natural moving image, the feature point pair extracting unit 124 moves the hash table of the pre-movement frame and the moving table. In addition to feature point pairs having the same hash value, feature point pairs whose hash values differ by ± 1 may be extracted as feature point pairs from the subsequent hash table.

  For example, in the hash table shown in FIG. 6, the feature point pair extraction unit 124 uses the feature point (hash value 83) at the coordinates (86, 50) of the pre-movement frame and the coordinates (288, 148) of the post-movement frame. ) In the feature point (hash value 83), and the feature point (hash value 83) in the pre-movement frame coordinates (86, 50) and the feature point in the post-movement frame coordinates (32, 384). A pair with (hash value 84) is also extracted as a feature point pair. In the hash table shown in FIG. 6, the feature point pair extraction unit 124 uses the feature point (hash value 249) at the coordinates (31, 112) of the pre-movement frame and the coordinates (233, 200) of the post-movement frame. ) And a feature point pair (hash value 250) are also extracted as feature point pairs.

  The vector calculation unit 125 then, for each feature point pair extracted by the feature point pair extraction unit 124, the position (coordinate) of the feature point in the pre-movement frame and the position (coordinate) of the feature point in the post-movement frame. The vector which connects is calculated. For example, if it is a feature point pair of the hash table shown in FIG. 6, a vector is calculated as shown in FIG. The movement vector determination unit 126 is a vector that indicates the movement of the moving region, except the zero vector (0, 0), among the vectors calculated by the vector calculation unit 125 for each feature point pair. Is determined as a movement vector.

  By doing so, it is possible to extract a feature point pair even for a moving image in which position fluctuation occurs, such as a natural moving image. Therefore, by doing in this way, it is possible to accurately determine a movement vector even for a moving image in which position fluctuation occurs, such as a natural moving image.

  In the present embodiment, as described above, when attention is paid to a certain edge point, the distance D1 between the edge point and the edge point one left of the edge point, and the edge point and the edge point 2 Using the distance D2 to the left edge point, D1 + n × D2 is used as the hash value of the edge point. For this reason, when the value of the distance D2 from the two left edge points is shifted by 1 due to the fluctuation of the position, the hash value is shifted by n. Therefore, the feature point pair extraction unit 124, from the hash table of the pre-movement frame and the hash table after the movement, in addition to feature point pairs having the same hash value, also includes feature point pairs whose hash values are different by ± n. You may make it extract as a feature point pair. This also makes it possible to perform calculation in consideration of the influence of position fluctuation, and to calculate the movement vector more accurately.

  Further, it is also possible to consider color fluctuation as well as position fluctuation. That is, when the moving image type determining unit 127 determines that the moving image transmitted from the server device 100 to the client device 200 is a natural moving image, the feature point extracting unit 123 has the number of appearances equal to or less than a predetermined value. An edge point having a hash value is extracted as a feature point, and the feature point pair extraction unit 124, in addition to the feature point pair having the same hash value, from the hash table of the pre-movement frame and the hash table after the movement. Alternatively, feature point pairs whose hash values differ by ± 1 may be extracted as feature point pairs. In this way, both position fluctuation and color fluctuation can be taken into account, and the movement vector can be calculated more accurately.

<Movie Type Discriminating Unit 127>
The moving image type determination unit 127 determines whether the moving image transmitted from the server device 100 to the client device 200 is a natural moving image or an artificial moving image. There are various methods for performing this determination. For example, as shown below, a method based on color fluctuation and a method based on position fluctuation are conceivable.

  (1) As a method for determining whether the moving image transmitted from the server device 100 to the client device 200 is a natural moving image or an artificial moving image, first, whether or not there is a color fluctuation in the transmitted moving image, A method of discriminating based on the above is conceivable. As described above, there is no color fluctuation in an artificial moving image that is artificially generated only by digital processing. Therefore, it is possible to discriminate between a natural moving image and an artificial moving image by determining a moving image in which color fluctuation exists as a natural moving image and determining a moving image in which no color fluctuation exists as an artificial moving image.

  Therefore, for example, the moving image type determination unit 127 selects the color of each feature point of the feature point pair connected by the movement vector determined by the movement vector detection unit 120 among the feature point pairs extracted by the movement vector detection unit 120. And the ratio of the feature point pairs having the same color among the feature point pairs connected by these movement vectors may be calculated. Here, when there is no color fluctuation in the moving image, the colors should match in these feature point pairs. Therefore, the moving image type determination unit 127 determines that the transmitted moving image is a natural moving image when the ratio is greater than a predetermined value, and the transmitted moving image is an artificial moving image when the ratio is equal to or less than the predetermined value. Therefore, it is possible to distinguish between a natural moving image and an artificial moving image.

  (2) Further, for example, the moving image type determination unit 127 includes a color difference in the feature point pair connected by the movement vector determined by the movement vector detection unit 120 among the feature point pairs extracted by the movement vector detection unit 120. And the standard deviation of the distribution of these color differences may be calculated. Here, when there is no color fluctuation in the moving image, the colors match in these feature point pairs. Therefore, when there is no color fluctuation, the standard deviation of the color difference distribution should be zero. For this reason, when the standard deviation is larger than a predetermined value, the moving image type determination unit 127 determines that the transmitted moving image is a natural moving image, and when the standard deviation is equal to or smaller than the predetermined value, By discriminating that the video is an artificial video, it is possible to discriminate between a natural video and an artificial video.

  (3) As a method of determining whether the moving image transmitted from the server apparatus 100 to the client device 200 is a natural moving image or an artificial moving image, it is based on whether or not there is a position fluctuation in the transmitted moving image. It is also possible to discriminate them. As described above, there is no position fluctuation in an artificial moving image that is artificially generated only by digital processing. Therefore, it is possible to discriminate between a natural moving image and an artificial moving image by determining a moving image in which position fluctuation exists as a natural moving image and determining a moving image in which position fluctuation does not exist as an artificial moving image.

  Therefore, for example, the moving image type discrimination unit 127 hashes the feature points of the feature point pairs connected by the movement vector determined by the movement vector detection unit 120 among the feature point pairs extracted by the movement vector detection unit 120. It is preferable to calculate the difference between the values and calculate the standard deviation of the distribution of these hash value differences. Here, when there is no position fluctuation in the video, the hash values match in these feature point pairs. Therefore, when there is no position fluctuation, the standard deviation of the hash value difference distribution should be zero. is there. For this reason, when the standard deviation is larger than a predetermined value, the moving image type determination unit 127 determines that the transmitted moving image is a natural moving image, and when the standard deviation is equal to or smaller than the predetermined value, By discriminating that the video is an artificial video, it is possible to discriminate between a natural video and an artificial video.

<Example of processing operation during video type discrimination>
FIG. 11 illustrates processing performed in the motion vector detection unit 120 according to the present embodiment when determining whether the moving image transmitted from the server device 100 to the client device 200 is a natural moving image or an artificial moving image. It is a figure which shows an operation example.

  First, the processing operation shown in FIG. 8 is performed on moving image data of a predetermined number of frames (for example, several to several tens of frames) of a transmission moving image (S201). The processing operation performed at this time may be a processing operation considering color fluctuation or a processing operation not considering it. In addition, the processing operation performed at this time may be a processing operation that considers the fluctuation of the position, or may be a processing operation that does not consider it.

  Then, the moving image type determining unit 127 determines whether the transmitted moving image is a natural moving image or an artificial moving image based on the moving image data of the first predetermined number of frames of the transmitted moving image (S202). At this time, the moving image type determination unit 127 may use, for example, any one of the methods (1) to (3). When the moving image type determination unit 127 uses the above method (3), in step S201, the processing operation shown in FIG. 8 is performed in consideration of the position fluctuation. That is, in step S104 in FIG. 8, the feature point pair extraction unit 124 determines that the hash value is ± 1 in addition to the feature point pair having the same hash value from the hash table of the pre-movement frame and the hash table after the movement. A pair of feature points that are different from each other is also extracted as a feature point pair.

  As described above, in the present embodiment, it is possible to determine whether the transmission video is a natural video or an artificial video using the first predetermined number of video data of the transmission video. After that, as described above, it is possible to calculate the movement vector in consideration of the color fluctuation and the position fluctuation. Therefore, it is possible to calculate an accurate movement vector.

<First Configuration Example of Moving Area Detection Unit 130>
The movement area detection unit 130 detects a movement area based on the pre-movement frame and the post-movement frame stored in the storage unit 110 and the movement vector detected by the movement vector detection unit 120. FIG. 12 is a diagram illustrating a first configuration example of the movement area detection unit 130 of the server device 100 according to the embodiment of the present invention. The moving area detecting unit 130 of the first configuration example includes a comparison area determining unit 131 and a moving area determining unit 132.

  The comparison area determination unit 131 determines a comparison area used for detecting the movement area. Of the points in the pre-movement frame, when the movement vector detected by the movement vector detection unit 120 is used for movement, the point that moves out of the frame is not a point in the movement area. Therefore, there should be a moving area within the area occupied by the points that are removed, that is, the points that remain in the frame when moved by the movement vector. Therefore, as shown in FIG. 13, among the points of the pre-movement frame, an area formed by points remaining in the frame when moved by the movement vector is set as a comparison area of the pre-movement frame. For the same reason, as shown in FIG. 13, among the points of the post-movement frame, an area formed by points remaining in the frame when moved by the inverse vector of the movement vector is set as a comparison area of the post-movement frame.

  The movement area determination unit 132 determines the movement area by comparing the pre-movement frame and the post-movement frame using the comparison area determined by the comparison area determination unit 131. At this time, the movement area determination unit 132 according to the present embodiment corresponds to (A1) a comparison between the comparison area of the pre-movement frame and the comparison area of the post-movement frame, and (A2) a comparison area of the pre-movement frame and the comparison area. Three comparisons are made: comparison with the area of the post-movement frame to be performed, and (A3) comparison of the comparison area of the post-movement frame and the area of the pre-movement frame corresponding to the comparison area.

  The comparison of (A1), that is, the comparison between the comparison area of the pre-movement frame and the comparison area of the post-movement frame is performed by comparing the point of the comparison area of the pre-movement frame and the post-movement frame at the coordinate position obtained by moving the point by the movement vector. This is a comparison with this point. That is, when the movement vector is (X, Y), as shown in FIG. 13, the point (x, y) in the comparison area of the pre-movement frame and the point (x + X, y + Y) in the comparison area of the post-movement frame. ). If the points to be compared are points in the moving region, they are considered to have the same pixel value. Further, if the points to be compared are points outside the moving region, those points are considered to have different pixel values.

  Further, the comparison of (A2), that is, the comparison between the comparison area of the pre-movement frame and the area of the post-movement frame corresponding to the comparison area, the movement at the point of the comparison area of the pre-movement frame and the coordinate position of that point It is a comparison with the point of the back frame. That is, a comparison is made between the point (x, y) in the comparison area of the pre-movement frame and the point (x, y) in the post-movement frame. Therefore, when the comparison area is determined as shown in FIG. 13, in the comparison of (A2), the area surrounded by the oblique lines shown in FIG. 14 is compared.

  The points in the movement area of the pre-movement frame should have moved to different coordinates in the post-movement frame. Therefore, in the comparison of (A2), when the points to be compared are in the moving region, the points are considered to have different pixel values. On the other hand, if the point to be compared is a point outside the moving region, the point is not moved, and it is considered that those points have the same pixel value.

  The same applies to the comparison in (A3), that is, the comparison between the comparison area of the post-movement frame and the area of the pre-movement frame corresponding to the comparison area. For example, when the comparison area is determined as shown in FIG. 13, in the comparison of (A3), the areas indicated by the oblique lines in FIG. 15 are compared.

  Since a rectangular area is desired as the moving area, each comparison is performed for each line. At this time, in the comparison of (A1), whether or not (Condition A1) “The number of pixels having the same pixel value is equal to or greater than a predetermined value” is satisfied for each line. Further, in the comparison of (A2) and the comparison of (A3), it is confirmed for each line whether (Condition A2) “Different pixel values but the number of pixels is equal to or greater than a predetermined value” holds. In addition, (Condition A1) may be “the ratio of pixels having the same pixel value is equal to or greater than a predetermined value”. Further, (Condition A2) may be set as “the ratio of pixels having different pixel values is a predetermined value or more”. Alternatively, “the number (ratio) of pixels having the same pixel value is equal to or less than a predetermined value” may be used.

  Each comparison is made for each line in order from the border line of the comparison area, that is, from the four sides of the top, bottom, left, and right, inward. Then, in all the comparisons (A1), (A2), and (A3), if a line that satisfies the condition is found, the confirmation of the condition for each line is stopped. That is, when a line that satisfies all of the conditions (condition A1), (condition A2), and (condition A3) is found, the confirmation of the condition for each line ends. As illustrated in FIG. 16, the movement area determination unit 132 determines that the area surrounded by the found line is a movement area.

  As described above, by determining a region that satisfies all three conditions as a moving region, it is possible to detect the moving region more accurately. That is, it is possible not only to simply compare the comparison areas but also to more accurately detect the movement area by comparing before and after movement in the same area.

<Example of Processing Operation in First Configuration Example of Moving Area Detection Unit 130>
FIG. 17 is a diagram illustrating an example of a processing operation in the first configuration example of the movement area detection unit 130 of the server device 100 according to the present embodiment. First, the comparison area determination unit 131 determines a comparison area in each of the pre-movement frame and the post-movement frame (S301). Then, the movement area determination unit 132 compares the comparison area of the pre-movement frame and the comparison area of the post-movement frame, compares the comparison area of the pre-movement frame and the area of the post-movement frame corresponding to the comparison area, By comparing the comparison area of the post-movement frame with the area of the pre-movement frame corresponding to the comparison area, the movement area is determined (S302). With such processing operation, the moving area detection unit 130 of the first configuration example detects the moving area.

  The line determined as the frame of the moving area by the first configuration of the moving area determining unit 130 described above does not satisfy (condition A1) “the number of pixels having the same pixel value is equal to or greater than a predetermined value”. Must not. Here, if the predetermined value is too small, the condition becomes loose, and an area larger than the original moving area may be determined as the moving area. If a region larger than the original moving region is determined as the moving region, the difference data becomes large as will be described in detail below. In addition, processing such as copying of the moving area is performed when reproducing the frame after movement, etc., but if a large area is determined as the moving area, the amount of data to be copied becomes large, and the amount of processing at the time of copying is large. It will become bigger. For this reason, it is necessary to increase the above-mentioned predetermined value to some extent and make the conditions strict. However, if it is too severe, an area that is smaller than the original moving area may be determined as the moving area. Therefore, it is also possible to determine a candidate region for the moving region by a method other than the method described above and compare only the lines in the region outside the candidate region under the above conditions. By doing so, even if (Condition A1) is made stricter, it is not smaller than this candidate area, so that an area smaller than the original moving area is not determined as the moving area.

<Second Configuration Example of Moving Area Detection Unit 130>
FIG. 18 is a diagram illustrating a second configuration example of the movement area detection unit 130 of the server device 100 according to the embodiment of the present invention. The second configuration example of the moving region detection unit 130 is configured by adding an initial region setting unit 133 and a candidate region determination unit 134 to the first configuration example. With this configuration, it is possible to determine a moving area candidate area by a method different from the above-described method, and to compare only the area outside the candidate area for each line under the above-described conditions to determine the moving area. become.

  The initial region setting unit 133 sets an initial region for detecting a candidate region for the moving region. Since the rectangular area is to be obtained as the moving area, the rectangular area is set as the initial area. There are various ways to set the initial area. As described above, among the feature point pairs extracted by the movement vector detection unit 120, the feature points of the feature point pair connected by the movement vector determined by the movement vector detection unit 120 are points in the movement region. There is a high possibility. Therefore, for example, a rectangular area including all these points may be set as the initial area. For example, as shown in FIG. 19, when feature points of feature point pairs connected by movement vectors are distributed, an area as shown in FIG. 19 may be set as an initial area. For example, the comparison area may be set as the initial area.

  When setting the initial region, as described above, instead of setting the rectangular region including all the feature points of the feature point pairs connected by the movement vector determined by the movement vector detection unit 120 as the initial region, Of these feature point pairs, a rectangular region including only feature points of feature point pairs having no color difference among the feature point pairs may be set as an initial region. If the points of a feature point pair are points in the moving region, they should be the same color. Therefore, by doing this, an initial rectangle is set after removing a feature point pair that is not a point in the moving region from a feature point pair connected by the moving vector determined by the moving vector detecting unit 120. As a result, an initial region closer to the moving region can be obtained.

  Further, among the feature point pairs connected by the motion vector determined by the motion vector detection unit 120, a rectangular region including only the feature points of the feature point pair whose color difference in the feature point pair is less than a predetermined value is an initial region. You may make it set as. As described above, in the case of a natural moving image, there is a color fluctuation. Therefore, if the color difference in the feature point pair is equal to or less than the color difference due to the color fluctuation, the point of the feature point pair is highly likely to be a point in the moving region. Therefore, by doing this, a feature point pair that is not a point in the moving region is removed from the feature point pair connected by the movement vector determined by the movement vector detecting unit 120 even for a natural moving image. Thus, an initial rectangle can be set, and an initial area closer to the moving area can be obtained.

  The candidate area determination unit 134 determines a candidate area for the movement area by excluding points outside the movement area from the initial area set by the initial area setting unit 133. For example, as described above, a rectangular area that includes all feature points of a feature point pair connected by a movement vector is likely to include points in the movement area, and thus is suitable as an initial area. It may also contain outside points. Therefore, the candidate area determination unit 134 removes points outside the moving area from such an initial area.

  Therefore, the candidate area determination unit 134 also (B1) compares the initial area of the pre-movement frame and the initial area of the post-movement frame (B2) the initial area of the pre-movement frame and the same as the movement area determination unit 132 described above. Comparison with the area of the post-movement frame corresponding to the initial area (B3) The three comparisons of the initial area of the post-movement frame and the area of the pre-movement frame corresponding to the initial area are performed.

  Since it is desired to obtain a rectangular area as the moving area, the candidate area determining unit 134 also performs the comparison for each line. At this time, in the comparison of (B1), it is confirmed for each line whether (Condition B1) “Pixels with the same pixel value continue for a predetermined value or more” holds. In the comparison of (B2) and the comparison of (B3), it is confirmed for each line whether (Condition B2) “Different pixel values but the number of pixels is equal to or larger than a predetermined value” is satisfied. Further, (Condition B1) may be set as “pixels having the same pixel value continue for a predetermined ratio or more”. Further, (Condition B2) may be set as “the pixel value is different, but the pixel ratio is equal to or greater than a predetermined value”. Alternatively, “the number (ratio) of pixels having the same pixel value is equal to or less than a predetermined value” may be used.

  Also in the candidate area determination unit 134, the comparison is performed for each line in order from the frame line of the initial area, that is, from the four sides on the upper, lower, left, and right sides. Then, in all the comparisons (B1), (B2), and (B3), when a line that satisfies the condition is found, the confirmation of the condition for each line is stopped. That is, when a line that satisfies all of the conditions (Condition B1), (Condition B2), and (Condition B3) is found, the confirmation of the condition for each line ends. The candidate area determination unit 132 determines that the area surrounded by the found line is a candidate area for the movement area.

  In this configuration, the comparison area determination unit 131 determines, as a comparison area, an area obtained by removing the candidate area determined by the candidate area unit 134 from the comparison area obtained by the above-described method. That is, the hatched area shown in FIG. 20 is determined as the comparison area.

  Further, in this configuration, the moving region determining unit 132 uses the comparison region determined by the comparative region determining unit 131 to determine the moving region by the same method as the method in the first configuration example described above. In the method described above, the comparison is performed line by line from the outer frame of the comparison area to the inner side, but the comparison is performed line by line from the frame line of the candidate area to the frame line of the comparison area. Also good. At this time, if a line that does not hold even one of the above (Condition A1), (Condition A2), or (Condition A3) is found, the comparison is stopped and the line surrounded by one of the found lines is enclosed. It is preferable to determine the area to be moved as the moving area.

  As described above, a region that satisfies all three conditions is first detected as a candidate region, and further, a region that satisfies all three other conditions is determined as a moving region, so that the moving region can be detected more accurately. It becomes possible.

<Example of Processing Operation in Second Configuration Example of Moving Area Detection Unit 130>
FIG. 21 is a diagram illustrating an example of a processing operation in the second configuration example of the movement area detection unit 130 of the server device 100 according to the present embodiment. First, the initial area setting unit 133 sets an initial area in each of the pre-movement frame and the post-movement frame (S401). Then, the candidate area determination unit 134 uses this initial area to determine a candidate area for the movement area in each of the pre-movement frame and the post-movement frame (S402). Next, the comparison area determination unit 131 determines a comparison area in each of the pre-movement frame and the post-movement frame using this candidate area (S403). Then, the movement area determination unit 132 compares the comparison area of the pre-movement frame and the comparison area of the post-movement frame, compares the comparison area of the pre-movement frame and the area of the post-movement frame corresponding to the comparison area, By comparing the comparison area of the post-movement frame with the area of the pre-movement frame corresponding to the comparison area, the movement area is determined (S404). Through such processing operation, the moving area detection unit 130 of the second configuration example detects the moving area.

<Other processing>
The difference data generation unit 140 includes the pre-movement frame and the post-movement frame stored in the storage unit 110, the movement vector detected by the movement vector detection unit 120, and the movement region detected by the movement region detection unit 120. To generate differential data. For example, as shown in FIG. 22, the difference data generation unit 130 generates a frame in which the movement area of the pre-movement frame is copied to the position moved by the movement vector of the pre-movement frame, and this frame and the post-movement frame To generate difference data. By doing in this way, as shown in FIG. 23, even when the moving area cannot be accurately detected, a difference area is detected as shown in FIG. 24, and the data of this difference area can be used as difference data. Therefore, if the difference data, the movement vector, and the pre-movement frame are used, the post-movement frame can be reproduced. FIG. 25 shows the difference area when the moving area can be accurately detected. Compared to the case shown in FIG. 24, the area of the difference region is small. That is, by accurately detecting the movement area, the difference area becomes smaller, and the difference data transmitted to the client device 200 can be made smaller. That is, according to the present invention, since the moving area can be detected more accurately, the difference area can be reduced, and the difference data transmitted to the client device 200 can be further reduced. For this reason, the time for detecting the difference area can be shortened, and the time required for compressing the difference data can be shortened.

  The transmission unit 150 transmits the information on the movement vector detected by the movement vector detection unit 120, the information on the movement region detected by the movement region detection unit 130, and the difference data generated by the difference data generation unit 140 to the client. To device 200.

  In this way, by transmitting not only the data of the entire frame after movement but also the movement vector, movement area information, and difference data, the amount of data to be transmitted is reduced, and the load on the network is reduced. Further, since the amount of data to be transmitted is reduced, the transmission time through the network can be shortened. Further, in the present invention, since the moving area can be detected more accurately, the difference data becomes smaller. Thus, the load on the network is further reduced, and the transmission time through the network can be further shortened.

<Example of Processing Operation of Server Device 100>
FIG. 26 is a diagram showing processing operations in the server apparatus 100 according to the embodiment of the present invention. First, the server apparatus 100 detects a movement vector by the movement vector detection unit 120 (S501). Then, based on the movement vector detected by the movement vector detection unit 120, the movement region detection unit 130 detects the movement region (S502). Then, based on the movement vector detected by the movement vector detection unit 120 and the movement region detected by the movement region detection unit 130, the difference data generation unit 140 generates difference data (S503). Then, the information on the movement vector detected by the movement vector detection unit 120, the information on the movement region detected by the movement region detection unit 130, and the difference data generated by the difference data generation unit 140 are transmitted by the transmission unit 150. The data is transmitted to the client device 200 (S504). Through such processing operation, the server apparatus 100 transmits the information on the movement vector, the information on the movement area, and the difference data so that the client apparatus 200 can reproduce the post-movement frame.

  As described above, the server apparatus 100 uses the information transmitted from the server apparatus 100 and the pre-movement frame stored in the client apparatus 200 in the client apparatus 200 without transmitting the post-movement frame. It becomes possible to reproduce the frame after movement.

  The processing operation in the above-described embodiment can be executed by hardware, software, or a combined configuration of both.

  When processing by software is executed, the program is loaded from a ROM (Read Only Memory) storing a program recording a processing sequence to a memory (RAM) in a computer incorporated in dedicated hardware. The program can be read and executed, or the program can be installed and executed on a general-purpose computer capable of executing various processes.

  For example, the program can be recorded in advance on a hard disk or ROM as a recording medium. Alternatively, the program is stored on a removable recording medium such as a magnetic disk such as a floppy (registered trademark) disk, an optical disk such as a CD (Compact Disc) or DVD (Digital Versatile Disc), or a magneto-optical disk such as an MO (Magneto Optical) disk. It is possible to store (record) temporarily or permanently.

  Such a removable recording medium can be provided as so-called package software.

  The program is installed on the computer from the above-described removable recording medium, transferred wirelessly from the download site to the computer, or transferred to the computer via a network such as a LAN (Local Area Network) or the Internet. On the other hand, the computer can receive the transferred program and install it on a recording medium such as a built-in hard disk.

  In addition to being executed in time series in accordance with the processing operations described in the above embodiment, the processing capability of the apparatus that executes the processing, or a configuration to execute in parallel or individually as necessary Is also possible.

  In addition, the system described in the above embodiment can be configured to have a logical set configuration of a plurality of devices or to mix the functions of each device.

  The present invention has been described above by the preferred embodiments of the present invention. While the invention has been described with reference to specific embodiments thereof, various modifications and changes can be made to these embodiments without departing from the broader spirit and scope of the invention as defined in the claims. is there.

  Further, in the server device according to the present invention, when the moving vector detection unit determines that the moving image data is a natural moving image by the moving image type determining unit, the movement vector detecting unit considers color fluctuations in the two frames. A movement vector may be detected.

  Further, in the server device according to the present invention, when the moving vector detecting unit determines that the moving image data is a natural moving image by the moving image type determining unit, the movement vector detecting unit considers the fluctuation of the position in the two frames. A movement vector may be detected.

  Further, in the server device according to the present invention, the movement vector detecting unit extracts an edge point extracting unit that extracts, as an edge point, a pixel having a large change in pixel value compared to surrounding pixels in each of the two frames. A hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is digitized as a hash value by a predetermined method using the edge points extracted by the edge point extraction unit, and the two frames A feature point for extracting, as a feature point, an edge point having a hash value with an appearance count of 1 from each of the two hash tables generated by the hash table generation unit and the hash table generation unit generated for each Extracting means; a hash table of the first frame; and a second hash table of the first frame. A feature point pair extracting unit that extracts a pair of feature points having the same hash value as a feature point pair, and the feature point pair extracted by the feature point pair extracting unit. A vector calculating means for calculating a vector connecting the feature point in the frame and the feature point in the second frame; and a vector calculated for each feature point pair by the vector calculating means except for the zero vector, You may make it have a movement vector determination means which determines a frequent vector as a movement vector.

  Further, in the server device according to the present invention, the feature point extracting unit, when the moving image type determining unit determines that the moving image data is a natural moving image, the two frames generated by the hash table generating unit. You may make it extract from each hash table the edge point which has a hash value whose appearance frequency is below a predetermined value as a feature point.

  In the server device according to the present invention, when the moving image type determining unit determines that the moving image data is a natural moving image, the feature point pair extracting unit and the second frame hash table and the second frame A pair of feature points whose hash values differ by a predetermined value may be extracted from the hash table as feature point pairs.

  Further, in the server device according to the present invention, the moving image type discriminating unit is configured such that the ratio of the feature point pairs whose colors match among the feature point pairs connected by the movement vector determined by the movement vector detecting unit is greater than a predetermined value. When it is large, the moving image data may be determined to be a natural moving image.

  Further, in the server device according to the present invention, when the moving image type discriminating unit has a standard deviation of the color difference distribution in the feature point pair connected by the movement vector determined by the movement vector detecting unit greater than a predetermined value, You may make it discriminate | determine that the said moving image data is a natural moving image.

  In the server device according to the present invention, when the moving image type discriminating unit has a standard deviation of a difference distribution of hash values in the feature point pair connected by the movement vector determined by the movement vector detecting unit larger than a predetermined value. The moving image data may be determined as a natural moving image.

  Further, in the movement vector detection method according to the present invention, the movement vector detection step extracts an edge point extraction step that extracts, as an edge point, a pixel having a large change in pixel value as compared with surrounding pixels in each of the two frames. And the hash table showing the number of appearances for each hash value and the coordinate value of the edge point having the hash value as a hash value by converting the edge point extracted in the edge point extraction step into a hash value by a predetermined method. A hash table generation step generated for each of the two frames and an edge point having a hash value with the appearance count of 1 are extracted as feature points from each of the two hash tables generated by the hash table generation step. Feature point extraction step and hash of the first frame A feature point pair extraction step for extracting a feature point pair having the same hash value as a feature point pair from the table and the second hash table, and a feature point pair extracted by the feature point pair extraction step. On the other hand, a vector calculation step for calculating a vector connecting the feature point in the first frame and the feature point in the second frame, and among the vectors calculated for each feature point pair by the vector calculation step In addition to the zero vector, a movement vector determination step for determining the most frequently occurring vector as a movement vector may be included.

  In the movement vector detection method according to the present invention, the feature point extracting step may be performed when the moving image type determining step determines that the moving image data is a natural moving image, and the feature table extracting step generates the 2 generated by the hash table generating step. An edge point having a hash value whose number of appearances is a predetermined value or less may be extracted as a feature point from the hash table of each of the two frames.

  In the movement vector detection method according to the present invention, when the moving image type determining step determines that the moving image data is a natural moving image, the feature point pair extracting step includes the hash table of the first frame and the A pair of feature points whose hash values differ by a predetermined value may be extracted from the second hash table as a feature point pair.

  In the movement vector detection method according to the present invention, the moving image type determination step includes a predetermined ratio of the feature point pairs having the same color among the feature point pairs connected by the movement vector determined in the movement vector detection step. When the value is larger than the value, the moving image data may be determined to be a natural moving image.

  In the moving vector detection method according to the present invention, the moving image type determining step may be performed when the standard deviation of the color difference distribution in the feature point pair connected by the moving vector determined by the moving vector detecting step is larger than a predetermined value. The moving image data may be determined as a natural moving image.

  Further, in the movement vector detection method according to the present invention, the moving image type determination step has a standard deviation of a difference distribution of hash values in a feature point pair connected by the movement vector determined in the movement vector detection step greater than a predetermined value. In some cases, the moving image data may be determined to be a natural moving image.

  In the program according to the present invention, the movement vector detection unit extracts an edge point extraction unit that extracts, as an edge point, a pixel having a large change in pixel value compared to surrounding pixels in each of the two frames. The edge points extracted by the edge point extraction means are digitized as hash values by a predetermined method, and a hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is displayed for each of the two frames. And a feature point extraction that extracts an edge point having a hash value with the appearance count of 1 as a feature point from each of the two hash tables generated by the hash table generation unit Means, the hash table of the first frame and the second hash table. A feature point pair extracting unit that extracts a pair of feature points having the same hash value as a feature point pair, and the feature point pair extracted by the feature point pair extracting unit. A vector calculating means for calculating a vector connecting the feature point in the frame and the feature point in the second frame; and a vector calculated for each feature point pair by the vector calculating means except for the zero vector, You may make it have a movement vector determination means which determines a frequent vector as a movement vector.

  Further, in the program according to the present invention, the feature point extracting unit, when the moving image type determining unit determines that the moving image data is a natural moving image, each of the two frames generated by the hash table generating unit. From the hash table, an edge point having a hash value whose appearance count is equal to or less than a predetermined value may be extracted as a feature point.

  In the program according to the present invention, the feature point pair extraction unit, when the moving image type determining unit determines that the moving image data is a natural moving image, the hash table of the first frame and the second frame A pair of feature points whose hash values differ by a predetermined value may be extracted as a feature point pair from the hash table.

  Further, in the program according to the present invention, the moving image type discriminating unit is configured such that, among the feature point pairs connected by the movement vector determined by the movement vector detecting unit, the ratio of the feature point pairs having the same color is larger than a predetermined value. In some cases, the moving image data may be determined to be a natural moving image.

  In the program according to the present invention, when the moving image type discriminating unit has a standard deviation of the color difference distribution in the feature point pair connected by the movement vector determined by the movement vector detecting unit greater than a predetermined value, You may make it discriminate | determine that moving image data is a natural moving image.

  Further, in the program according to the present invention, when the moving image type discriminating unit has a standard deviation of a difference distribution of hash values in the feature point pair connected by the movement vector determined by the movement vector detecting unit larger than a predetermined value, You may make it discriminate | determine that the said moving image data is a natural moving image.

DESCRIPTION OF SYMBOLS 100 Server apparatus 110 Storage part 120 Movement vector detection part 121 Edge point extraction part 122 Hash table production | generation part 123 Feature point extraction part 124 Feature point pair extraction part 125 Vector calculation part 126 Movement vector determination part 127 Moving image kind discrimination | determination part 130 Movement area | region detection part 130 Unit 131 comparison region determining unit 132 moving region determining unit 133 initial region setting unit 134 candidate region determining unit 140 difference data generating unit 150 transmitting unit 200 client device

Claims (14)

A video type discriminating means for discriminating whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. A movement vector detecting means for detecting a movement vector indicating the magnitude and direction of movement of the moving area that is a moving area;
The movement vector detection means, in each of the two frames, an edge point extraction means for extracting a pixel having a large change in pixel value as compared to surrounding pixels as an edge point;
A hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is digitized as a hash value by a predetermined method using the edge points extracted by the edge point extraction unit, and the two frames A hash table generating means to generate for each;
When the moving image type determining unit determines that the moving image data is a natural moving image, a hash whose number of appearances is equal to or less than a predetermined value from the hash tables of the two frames generated by the hash table generating unit. Feature point extraction means for extracting edge points having values as feature points;
A feature point pair extraction unit that extracts a pair of feature points having the same hash value as a feature point pair from the hash table of the first frame and the second hash table;
Vector calculation means for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted by the feature point pair extraction means;
Of the vector calculated for each feature point pair by the vector calculating means, other than a zero vector, the server characterized by Rukoto that Yusuke and motion vector determining means for determining a vector of the most frequent as the movement vector, the apparatus. The movement vector detection means includes
The movement vector is detected in consideration of color fluctuations in the two frames when the moving image type determining unit determines that the moving image data is a natural moving image. Server device. The movement vector detection means includes
The movement vector is detected in consideration of fluctuations in position in the two frames when the moving image type determining unit determines that the moving image data is a natural moving image. Server device. A video type discriminating means for discriminating whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. A movement vector detecting means for detecting a movement vector indicating the magnitude and direction of movement of the moving area that is a moving area;
The movement vector detection means includes
In each of the two frames, an edge point extracting unit that extracts, as an edge point, a pixel having a large change in pixel value compared to surrounding pixels;
A hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is digitized as a hash value by a predetermined method using the edge points extracted by the edge point extraction unit, and the two frames A hash table generating means to generate for each;
Feature point extracting means for extracting, as a feature point, an edge point having a hash value whose appearance count is 1 from each of the two hash tables generated by the hash table generating means;
When the moving image type determining means determines that the moving image data is a natural moving image,
A feature point pair extraction unit that extracts a pair of feature points having different hash values by a predetermined value from the hash table of the first frame and the second hash table;
Vector calculation means for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted by the feature point pair extraction means;
Of the vector calculated for each feature point pair by the vector calculating means, other than a zero vector, wherein the Surusa over server that has a motion vector determining means for determining a vector of the most frequent as the movement vector, the apparatus. A video type discriminating means for discriminating whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. A movement vector detecting means for detecting a movement vector indicating the magnitude and direction of movement of the moving area that is a moving area;
The movement vector detection means includes
The movement vector is detected based on a determination result by the moving image type determination unit,
In each of the two frames, an edge point extracting unit that extracts, as an edge point, a pixel having a large change in pixel value compared to surrounding pixels;
A hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is digitized as a hash value by a predetermined method using the edge points extracted by the edge point extraction unit, and the two frames A hash table generating means to generate for each;
Feature point extracting means for extracting, as a feature point, an edge point having a hash value whose appearance count is 1 from each of the two hash tables generated by the hash table generating means;
A feature point pair extraction unit that extracts a pair of feature points having the same hash value as a feature point pair from the hash table of the first frame and the second hash table;
Vector calculation means for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted by the feature point pair extraction means;
Among the vectors calculated for each feature point pair by the vector calculation means, there is a movement vector determination means for determining the most frequent vector as a movement vector other than the zero vector,
The moving image type discrimination means includes
When the standard deviation of the color difference distribution in the feature point pair connected by the movement vector determined by the movement vector detection means is larger than a predetermined value, the moving image data is determined to be a natural moving image. to Lusa over server apparatus. The moving image type discrimination means includes
The moving image data is determined to be a natural moving image when the standard deviation of the distribution of hash value differences in the feature point pair connected by the moving vector determined by the moving vector detecting means is larger than a predetermined value. The server device according to claim 4 . A video type determination step for determining whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. A movement vector detecting step for detecting a movement vector indicating the magnitude and direction of movement of the moving area, which is a moving area,
The moving vector detection step includes:
In each of the two frames, an edge point extraction step of extracting a pixel having a large change in pixel value as compared to surrounding pixels as an edge point;
A hash table showing the number of appearances for each hash value and the coordinate value of the edge point having the hash value is converted into a hash value by a predetermined method and the hash table extracted in the edge point extraction step is the two frames. A hash table generation step to generate for each;
When the moving image type determining step determines that the moving image data is a natural moving image, a hash whose number of appearances is equal to or less than a predetermined value from the hash tables of the two frames generated by the hash table generating step. A feature point extraction step for extracting edge points having values as feature points;
A feature point pair extracting step of extracting a pair of feature points having the same hash value as a feature point pair from the hash table of the first frame and the second hash table;
A vector calculation step for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted in the feature point pair extraction step;
A movement vector determining step of determining, as a movement vector, the most frequently occurring vector other than the zero vector among the vectors calculated for each feature point pair in the vector calculation step. Method. A video type determination step for determining whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. A movement vector detecting step for detecting a movement vector indicating the magnitude and direction of movement of the moving area, which is a moving area,
The moving vector detection step includes:
In each of the two frames, an edge point extraction step of extracting a pixel having a large change in pixel value as compared to surrounding pixels as an edge point;
A hash table showing the number of appearances for each hash value and the coordinate value of the edge point having the hash value is converted into a hash value by a predetermined method and the hash table extracted in the edge point extraction step is the two frames. A hash table generation step to generate for each;
A feature point extracting step of extracting, as a feature point, an edge point having a hash value whose appearance count is 1 from each of the two hash tables generated by the hash table generating step;
When the moving image type determining step determines that the moving image data is a natural moving image, a feature point having a hash value that differs by a predetermined value from the hash table of the first frame and the second hash table. A feature point pair extraction step for extracting pairs as feature point pairs;
A vector calculation step for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted in the feature point pair extraction step;
Wherein among the vector calculation step of vector calculated for each feature point pair, other than zero vectors, moves you, comprising: the motion vector determining step of determining a vector of the most frequent as the movement vector, the dynamic Vector detection method. A video type determination step for determining whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. A movement vector detecting step for detecting a movement vector indicating the magnitude and direction of movement of the moving area, which is a moving area,
The moving vector detection step includes:
The movement vector is detected based on a determination result obtained by the moving image type determination step,
In each of the two frames, an edge point extraction step of extracting a pixel having a large change in pixel value as compared to surrounding pixels as an edge point;
A hash table showing the number of appearances for each hash value and the coordinate value of the edge point having the hash value is converted into a hash value by a predetermined method and the hash table extracted in the edge point extraction step is the two frames. A hash table generation step to generate for each;
A feature point extracting step of extracting, as a feature point, an edge point having a hash value whose appearance count is 1 from each of the two hash tables generated by the hash table generating step;
A feature point pair extracting step of extracting a pair of feature points having the same hash value as a feature point pair from the hash table of the first frame and the second hash table;
A vector calculation step for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted in the feature point pair extraction step;
A movement vector determining step for determining a most frequent vector as a movement vector other than the zero vector among the vectors calculated for each feature point pair in the vector calculation step;
The moving image type determining step includes:
When the standard deviation of the color difference distribution in the feature point pair connected by the movement vector determined by the movement vector detection step is larger than a predetermined value, it is determined that the moving image data is a natural moving image. that moving vector detection method be. The moving image type determining step includes:
The moving image data is determined to be a natural moving image when the standard deviation of the distribution of hash value differences in the feature point pairs connected by the moving vector determined by the moving vector detection step is larger than a predetermined value. The movement vector detection method according to claim 8 . Server device
A video type discriminating means for discriminating whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. Function as a movement vector detecting means for detecting a movement vector indicating the magnitude and direction of movement of the moving area,
The movement vector detection means includes
In each of the two frames, an edge point extracting unit that extracts, as an edge point, a pixel having a large change in pixel value compared to surrounding pixels;
A hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is digitized as a hash value by a predetermined method using the edge points extracted by the edge point extraction unit, and the two frames A hash table generating means to generate for each;
When the moving image type determining unit determines that the moving image data is a natural moving image, a hash whose number of appearances is equal to or less than a predetermined value from the hash tables of the two frames generated by the hash table generating unit. Feature point extraction means for extracting edge points having values as feature points;
A feature point pair extraction unit that extracts a pair of feature points having the same hash value as a feature point pair from the hash table of the first frame and the second hash table;
Vector calculation means for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted by the feature point pair extraction means; of the vector calculated for each feature point pair by means, other than a zero vector, program characterized Rukoto that Yusuke and motion vector determining means for determining a vector of the most frequent as the movement vector, a. Server device
A video type discriminating means for discriminating whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. Function as a movement vector detecting means for detecting a movement vector indicating the magnitude and direction of movement of the moving area,
The movement vector detection means includes
In each of the two frames, an edge point extracting unit that extracts, as an edge point, a pixel having a large change in pixel value compared to surrounding pixels;
A hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is digitized as a hash value by a predetermined method using the edge points extracted by the edge point extraction unit, and the two frames A hash table generating means to generate for each;
Feature point extracting means for extracting, as a feature point, an edge point having a hash value whose appearance count is 1 from each of the two hash tables generated by the hash table generating means;
When the moving image type determining means determines that the moving image data is a natural moving image,
A feature point pair extraction unit that extracts a pair of feature points having different hash values by a predetermined value from the hash table of the first frame and the second hash table;
Vector calculation means for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted by the feature point pair extraction means;
Of the vector calculated for each feature point pair by the vector calculating means, other than a zero vector, features and to Help program that has a motion vector determining means for determining a vector of the most frequent as the movement vector, the . Server device
A video type discriminating means for discriminating whether the video data is a natural video with fluctuations in color and position;
By comparing the first frame and the second frame, which are consecutive frames constituting the moving image data, the position on the frame is an image area that exists in both of the two frames. Function as a movement vector detecting means for detecting a movement vector indicating the magnitude and direction of movement of the moving area,
The movement vector detection unit detects the movement vector based on a determination result by the moving image type determination unit,
In each of the two frames, an edge point extracting unit that extracts, as an edge point, a pixel having a large change in pixel value compared to surrounding pixels;
A hash table indicating the number of appearances for each hash value and the coordinate value of the edge point having the hash value is digitized as a hash value by a predetermined method using the edge points extracted by the edge point extraction unit, and the two frames A hash table generating means to generate for each;
Feature point extracting means for extracting, as a feature point, an edge point having a hash value whose appearance count is 1 from each of the two hash tables generated by the hash table generating means;
A feature point pair extraction unit that extracts a pair of feature points having the same hash value as a feature point pair from the hash table of the first frame and the second hash table;
Vector calculation means for calculating a vector connecting the feature point in the first frame and the feature point in the second frame for each of the feature point pairs extracted by the feature point pair extraction means;
Among the vectors calculated for each feature point pair by the vector calculation means, there is a movement vector determination means for determining the most frequent vector as a movement vector other than the zero vector,
The moving image type discrimination means includes
When the standard deviation of the color difference distribution in the feature point pair connected by the movement vector determined by the movement vector detection means is larger than a predetermined value, the moving image data is determined to be a natural moving image. to Help program. The moving image type discrimination means includes
The moving image data is determined to be a natural moving image when the standard deviation of the distribution of hash value differences in the feature point pair connected by the moving vector determined by the moving vector detecting means is larger than a predetermined value. The program according to claim 12 .

Images