JP2011040808A - Information processor, control method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processor reducing an area to be detected as an updating area. <P>SOLUTION: The information processor (100) compares a frame to be encoded and an encoded reference frame, and detects an image area included in both the frame to be encoded and the reference frame, and moving its position on a screen, as a moving area (step S4). Then, the image of the detected moving area is copied to a coordinate position after movement on the reference frame, and motion compensation is executed (step S6). Then, the image area different in the reference frame after copying and the frame to be encoded is detected as the updating area (step S8). On the basis of the relation between the coordinate position of the detected moving area and the coordinate position of the detected updating area, the detected updating area is divided further. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for controlling user interface display.

  In recent years, there is a thin client system in which a server device and a thin client terminal are connected via a network.

  The thin client system is equipped with the minimum necessary functions in the thin client terminal and executes most of the processing on the server device side, and the thin client terminal side receives the processing result executed on the server device side, and the processing result Is displayed on the screen of the thin client terminal. This makes it possible to prevent information leakage from the terminal and to facilitate application management on the terminal side.

  In the thin client system, drawing data of an application program executed on the server device is transferred to the thin client terminal and displayed on the screen of the thin client terminal.

  For this reason, if the processing amount of drawing data increases or the number of thin client terminals connected to the server device increases, the processing load on the server device for transferring drawing data to the thin client terminal increases. However, problems such as a decrease in response time and a limitation on the number of connected thin client terminals may occur.

  For this reason, as a drawing data display control technique by software, only a rectangular update area including a drawing data portion that has been updated in one screen is cut out, and this is compressed and transferred as necessary. There is a document disclosing the technology (for example, see Patent Document 1). As a result, the amount of data associated with the transfer of drawing data on the computer screen can be reduced, and the processing load on the server device can be reduced.

  In addition, in a moving picture coding system such as MPEG (Moving Picture Expert Group), a frame to be coded is compared with a reference frame that has already been coded, and a movement vector and a movement area are detected. Motion compensation is performed based on the movement vector and the movement area to reduce the transmission code amount. Note that a technique called a full search method is generally used to detect a movement vector and a movement area.

  However, the full search method requires a very large amount of computation. For this reason, there is a document that discloses a high-speed technique that first searches coarsely and widely, narrows according to the evaluation result, and searches with high accuracy (see, for example, Patent Document 2).

JP-T-2004-503862 JP 2004-129099 A

  Even in the thin client system disclosed in Patent Document 1 described above, when the drawing data on the computer screen is transferred from the server device to the thin client terminal, the server device detects the movement of an image area such as a window, The amount of transfer data can be reduced by performing motion compensation based on the detected image area.

  However, the distance that an image region such as a window moves within one frame time may reach 100 pixels or more, but in many cases, it is about several pixels to several tens of pixels.

  For this reason, large windows or the like generally overlap before and after movement as shown in FIG. FIG. 1 shows a window A ′ before and after movement. When the reference frame including the window A ′ before the movement and the encoding target frame including the window A ′ after the movement are compared, as shown in FIG. 1, the movement area B ′ before and after the movement can be detected. . If the movement area B ′ before and after movement can be detected, motion compensation can be performed based on the movement area B ′ before and after movement. The coordinate position of the start point of the movement area B ′ before the movement is (X1 ′, Y1 ′), and the coordinate position of the end point is (X2 ′, Y2 ′). Further, the coordinate position of the start point of the movement area B ′ after the movement is (nX1 ′, nY1 ′), and the coordinate position of the end point is (nX2 ′, nY2 ′).

  It should be noted that when the reference frame subjected to motion compensation is compared with the encoding target frame and a rectangular update area including a drawing data portion that has been drawn and updated in one screen is cut out, the entire screen is set as one update area. When the detection area is set, as shown in FIG. 2, the entire area corresponding to the movement area B ′ before the movement is detected as the update area D ′. As a result, the region overlapping the motion compensation (copy) region is also detected as the update region D ′, and the code amount increases. The update area D ′ indicates a rectangular area surrounded by a dotted line portion shown in FIG.

  The window A ′ shown in FIGS. 1 and 2 is assumed to be Windows (registered trademark) XP, and the outer frame portion C ′ is also detected as the movement region B ′. For this reason, the outer size of the window A ′ is the same as the outer size of the moving area B ′.

  However, as shown in FIG. 3, when the window A is Windows Vista (registered trademark) or the like, the outer frame portion C is transparent like transparent glass, and therefore overlaps the back of the outer frame portion C of the window A. The image is seen through the outer frame portion C. In the example of FIG. 3, the background image can be seen through the outer frame portion C of the window A before the movement. As a result, as shown in FIG. 3, when the reference frame including the window A before the movement is compared with the encoding target frame including the window A after the movement, the movement area before and after the movement excluding the outer frame portion C is compared. B will be detected. For this reason, the outer size of the window A is different from the outer size of the moving area B. The coordinate position of the start point of the movement area B before the movement shown in FIG. 3 is (X1, Y1), and the coordinate position of the end point is (X2, Y2). Further, the coordinate position of the start point of the movement area B after the movement is (nX1, nY1), and the coordinate position of the end point is (nX2, nY2).

  When a reference frame that has been subjected to motion compensation based on the moving area B before and after the movement is compared with the encoding target frame, and when a rectangular update area that includes a drawing data portion that has been updated in one screen is cut out When the entire screen is set as one update area detection area, an area including the movement area B before and after the movement is detected as the update area D as shown in FIG. The update area D indicates an area surrounded by a dotted line portion shown in FIG. As a result, as shown in FIG. 4, an update area D that is larger than the update area D ′ shown in FIG. 2 is detected, and the code amount further increases.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an information processing apparatus, a control method, and a program that can reduce an area to be detected as an update area, which is the problem described above.

  In order to achieve this object, the present invention has the following features.

<Information processing device>
An information processing apparatus according to the present invention includes:
The encoding target frame to be encoded is compared with the encoded reference frame, and the position on the screen is moved in an image area that exists in both the encoding target frame and the reference frame. A moving area detecting means for detecting a moving image area as a moving area;
Motion compensation means for copying the image of the movement area detected by the movement area detection means to the coordinate position after movement on the reference frame;
Update area detecting means for detecting an image area different between the reference frame after copying and the encoding target frame as an update area;
The update area detecting means includes
The update area is further divided based on the relationship between the coordinate position of the movement area detected by the movement area detection means and the coordinate position of the update area.

<Control method>
The control method according to the present invention includes:
A control method performed by an information processing apparatus,
The encoding target frame to be encoded is compared with the encoded reference frame, and the position on the screen is moved in an image area that exists in both the encoding target frame and the reference frame. A moving area detecting step of detecting a moving image area as a moving area;
A motion compensation step of copying the image of the movement region detected in the movement region detection step to the coordinate position after movement on the reference frame;
An update area detecting step of detecting an image area that is different between the reference frame after copying and the encoding target frame as an update area, and
The update region detection step includes
The update area is further divided based on the relationship between the coordinate position of the movement area detected in the movement area detection step and the coordinate position of the update area.

<Program>
The program according to the present invention is:
A program to be executed by an information processing apparatus,
The encoding target frame to be encoded is compared with the encoded reference frame, and the position on the screen is moved in an image area that exists in both the encoding target frame and the reference frame. Moving area detection processing for detecting a moving image area as a moving area;
A motion compensation process for copying the image of the moving area detected in the moving area detection process to the coordinate position after movement on the reference frame;
An update area detection process for detecting, as an update area, an image area that is different between the reference frame after copying and the encoding target frame,
The update area detection process includes:
The update area is further divided based on the relationship between the coordinate position of the movement area detected by the movement area detection process and the coordinate position of the update area.

  According to the present invention, the area to be detected as the update area can be reduced.

It is a figure which shows the case where window A '(Windows XP) including outer frame C' which is not transparent is moved to the lower right. It is a figure which shows the case where the whole screen is set to the update area | region detection area. It is a figure which shows the case where the window A (Windows Vista) containing the transparent outer frame C is moved to the lower right. It is a figure which shows the case where the whole screen is set to the update area | region detection area. It is a figure which shows the system configuration example of the thin client system of this embodiment. 3 is a diagram illustrating an internal configuration example of a server apparatus 100 that configures a thin client system. FIG. It is a figure which shows a series of processing operation examples in a thin client system. It is a 1st figure for demonstrating a series of processing operation examples in a thin client system. It is a 2nd figure for demonstrating a series of processing operation examples in a thin client system. 3 is a diagram illustrating a configuration example of an update area detection area setting unit 24. FIG. 6 is a diagram illustrating an example of processing operation of an update area detection area setting unit 24. FIG. FIG. 10 is a first diagram for explaining an example of processing operation of an update area detection area setting unit 24; FIG. 10 is a second diagram for explaining an example of the processing operation of the update area detection area setting unit 24. 3 is a diagram illustrating a configuration example of an update region detection unit 25. FIG. 6 is a diagram illustrating an example of processing operation of an update area detection unit 25. FIG. FIG. 10 is a first diagram for explaining an example of processing operation of the update region detection unit 25 (when the update region 1 is divided). FIG. 10 is a second diagram for explaining an example of processing operation of the update region detection unit 25 (in the case where the update region 1 is divided). FIG. 11 is a third diagram for explaining an example of processing operation of the update region detection unit 25 (in the case where the update region 1 is divided). FIG. 10 is a fourth diagram for explaining an example of processing operation of the update region detection unit 25 (when the update region 2 is divided). FIG. 10 is a fifth diagram for explaining an example of processing operation of the update region detection unit 25 (when the update region 2 is divided). It is a 6th figure for demonstrating the processing operation example (when dividing the update area | region 2) of the update area | region detection part 25. FIG. FIG. 14 is a seventh diagram for explaining an example of processing operation of the update region detection unit 25 (when the update regions 1 and 2 are divided). It is a figure which shows the processing operation example of the update area | region detection part 25 of 2nd Embodiment. It is a figure for demonstrating the processing operation example of the update area | region detection part 25 of 2nd Embodiment.

<Outline of Thin Client System of this Embodiment>
First, an outline of the thin client system of this embodiment will be described with reference to FIGS.

  The thin client system of the present embodiment is configured by connecting the server device 100 and thin client terminals 200-1 to 200-n (n is an arbitrary integer) via a network (NW).

  As illustrated in FIG. 7, the server apparatus 100 according to the present embodiment compares the encoding target frame to be encoded with the encoded reference frame, and sets both the encoding target frame and the reference frame. An image area that is an existing image area and whose position on the screen is moving is detected as a movement area (step S4). Then, the image of the movement area detected in step S4 is copied to the coordinate position after movement on the reference frame, and motion compensation is performed (step S6). Next, an image area that is different between the copied reference frame and the encoding target frame is detected as an update area (step S8). At this time, the update area detected in step S8 is further divided based on the relationship between the coordinate position of the movement area detected in step S4 and the coordinate position of the update area detected in step S8.

  Thereby, the server apparatus 100 can reduce the area detected as the update area and reduce the amount of transfer data output to the thin client terminals 200-1 to 200-n. For this reason, it is possible to reduce the load on the network NW and perform efficient data transfer. Hereinafter, the thin client system of this embodiment will be described in detail with reference to the accompanying drawings.

<System configuration example of thin client system>
First, a system configuration example of the thin client system of the present embodiment will be described with reference to FIG.

  The thin client system of the present embodiment is configured by connecting the server device 100 and the thin client terminals 200-1 to 200-n (n is an arbitrary integer) via a network NW.

  The server device 100 is a device that transmits various information managed by the server device 100 to the thin client terminals 200-1 to 200-n.

  The thin client terminals 200-1 to 200-n are devices that receive various types of information sent from the server device 100 and display the received information on the screens of the thin client terminals 200-1 to 200-n.

<Internal configuration example of server device 100>
Next, an example of the internal configuration of the server device 100 of this embodiment will be described with reference to FIG.

  The server device 100 according to the present embodiment includes a storage unit 10, a control unit 20, and a communication unit 30.

<Storage unit 10>
The storage unit 10 stores various information. The storage unit 10 of the present embodiment includes an encoding target frame storage unit 11 and a reference frame storage unit 12.

  The encoding target frame storage unit 11 stores an encoding target frame to be encoded. The control unit 20 stores the video signal to be encoded in the encoding target frame storage unit 11 as an encoding target frame. For example, a color video signal on a display screen of a computer (not shown) is analog or digital captured, and the captured video signal for one screen is stored in the encoding target frame storage unit 11 as an encoding target frame. In addition, in order to generate a color video signal of a computer display screen (not shown), data is captured from image information stored inside the computer, and the captured video signal for one screen is used as an encoding target frame. It can also be stored in the frame storage unit 11.

  The reference frame storage unit 12 stores an encoded reference frame used when encoding the encoding target frame stored in the encoding target frame storage unit 11.

<Control unit 20>
The control unit 20 controls the server device 100 in an integrated manner. The control unit 20 of the present embodiment includes a movement vector detection unit 21, a movement region detection unit 22, a motion compensation unit 23, an update region detection region setting unit 24, an update region detection unit 25, and a region encoding unit 26. And comprising.

  The movement vector detection unit 21 compares the encoding target frame and the reference frame, and detects one main movement vector. The main movement vector means a dominant one of one or more movement vectors. For example, when a map scroll and a mouse cursor move simultaneously on the computer screen, the map scroll occupies most of the movement vector, so the movement vector by the map scroll is detected as the main movement vector. In addition, the detection method of a movement vector is not specifically limited, All the detection methods are applicable. Note that by comparing the encoding target frame and the reference frame over the entire screen, a movement vector in a relatively large object unit such as a window can be detected as a main movement vector.

  The movement area detection unit 22 is the same or similar image area that exists in both the encoding target frame and the reference frame, and the position on the screen changes by the movement vector detected by the movement vector detection unit 21. The detected image area is detected as a moving area. Note that the detection method of the moving region is not particularly limited, and any detection method can be applied as long as motion compensation described later can be performed.

  The motion compensation unit 23 copies the movement region detected by the movement region detection unit 22 to the movement destination indicated by the movement vector in the reference frame, and generates a reference frame after motion compensation. Note that the motion compensation method is not particularly limited, and for example, any motion compensation method defined in MPEG or the like is applicable.

  The update area detection area setting unit 24 sets one or more update area detection areas on the frame.

  The update region detection unit 25 detects, as the update region, a rectangular image region that is different between the reference frame after motion compensation and the encoding target frame for each update region detection region set by the update region detection region setting unit 24. . Further, the update area detection unit 25 further divides the update area based on the relationship between the detected position coordinates of the update area and the coordinate position of the movement area detected by the movement area detection unit 22.

  The region encoding unit 26 generates a code that is encoded by an arbitrary encoding method using the update region detected by the update region detection unit 25 as an image.

  The communication unit 30 performs communication control of various types of information stored in the storage unit 10. For example, the code information generated for the encoding target frame stored in the encoding target frame storage unit 11 is transmitted to the thin client terminals 200-1 to 200-n. Note that the code information generated for one encoding target frame includes the update region code information generated by the region encoding unit 26, and the moving region information detected by the moving vector detecting unit 21 and the moving region detecting unit 22. (Coordinate information and movement vector information of the movement source area) are included. The thin client terminals 200-1 to 200-n receive the code information transmitted from the server device 100, and decode the encoding target frame based on the received code information.

<A series of processing operations in the thin client system>
Next, a series of processing operations in the thin client system of this embodiment will be described with reference to FIG.

  The control unit 20 captures a frame to be encoded, and stores the captured frame in the encoding target frame storage unit 11 as an encoding target frame (step S1).

  The motion vector detection unit 21 compares the encoding target frame stored in the encoding target frame storage unit 11 with the reference frame stored in the reference frame storage unit 12, and detects one main movement vector. (Step S2).

  For example, if the movement vector 303 shown in FIG. 8 is dominant from the comparison between the encoding target frame 301 shown in FIG. 8 and the reference frame 302, the movement vector 303 is detected as a main movement vector. The movement vector detection unit 21 temporarily stores information on the detected movement vector 303 in the storage unit 10.

  Assume that the outer frame portion C of the window A included in the encoding target frame 301 and the reference frame 302 shown in FIG. 8 is transparent like transparent glass. As a result, as shown in FIG. 8, when the reference frame 302 including the window A before movement and the encoding target frame 301 including the window A after movement are compared, the movement of the part excluding the outer frame part C is compared. Vector 303 will be detected.

  Next, when a movement vector is detected from the encoding target frame (step 3 / Yes), the moving area detection unit 22 is the same or similar that exists in both the encoding target frame and the reference frame. An image area which is an image area and whose position on the screen is changed by the movement vector 303 detected by the movement vector detection unit 21 is detected as a movement area (step S4).

  For example, in the encoding target frame 301 and the reference frame 302 shown in FIG. 8, when the region 304 of the encoding target frame 301 and the region 305 of the reference frame 302 are the same or similar image regions, the moving region detection unit 22 detects the image areas 304 and 305 as the movement area B. The movement area detection unit 22 temporarily stores the detected coordinate information of the movement area B in the storage unit 10.

  Assume that the outer frame portion C of the window A included in the encoding target frame 301 and the reference frame 302 shown in FIG. 8 is transparent like transparent glass. As a result, the areas 304 and 305 excluding the outer frame part C are detected as the movement area B. The coordinate position of the start point of the movement area B before the movement is (X1, Y1), and the coordinate position of the end point is (X2, Y2). Further, the coordinate position of the start point of the movement area B after the movement is (nX1, nY1), and the coordinate position of the end point is (nX2, nY2).

  Next, when a movement area is detected (step S5 / Yes), the motion compensation unit 23 copies the image of the movement area B before movement on the reference frame to the position after movement indicated by the movement vector 303. Compensation is performed, and the reference frame stored in the reference frame storage unit 12 is updated (step S6).

  For example, in the case of FIG. 8, the region 305 is copied to the position corresponding to the region 304 in the encoding target frame 301 on the reference frame 302. As a result, a reference frame 306 after motion compensation is generated as shown in FIG.

  However, when the movement vector is not detected (step S3 / No), the movement area detection and the motion compensation are not executed. In addition, even if a movement vector is detected, if the detection of the movement area fails (step S5 / No), motion compensation is not executed.

  Next, the update area detection area setting unit 24 sets one or more update area detection areas for detecting the update area on the frame (step S7). A method for setting the update area detection area will be described later.

  Next, the update region detection unit 25 detects, as update regions, image regions that differ between the reference frame after motion compensation and the encoding target frame for each update region detection region set by the update region detection region setting unit 24. (Step S8). At this time, the update region detection unit 25 is based on the relationship between the coordinate position of the movement region detected by the movement region detection unit 22 in step S4 and the coordinate position of the update region detected by the update region detection unit 25 in step S8. In addition, the update area detected in step S8 is further divided. A detection method of the update area detection will be described later. Thereby, for example, in the case of FIG. 9B, six update regions 307 to 312 are detected. The update areas 307 to 312 indicate rectangular areas surrounded by dotted lines shown in FIG. The update area detection unit 25 temporarily stores the coordinate information of the detected six update areas 307 to 312 in the storage unit 10.

  Next, the region encoding unit 26 generates a code obtained by encoding the update region detected by the update region detection unit 25 as an image (step S9). The area encoding unit 26 temporarily stores the generated code information in association with the coordinate information of the update areas 307 to 312 stored in the storage unit 10.

  The communication unit 30 reads the information generated for the encoding target frame, that is, the coordinate information of each update region, the code information and the movement region information (the coordinate information and the movement vector information of the movement source region) from the storage unit 10. The read information is output to the thin client terminals 200-1 to 200-n (step S10).

  For example, in the case of FIG. 9B, the coordinate information of the update areas 307 to 312 and the code information thereof, the coordinate information of the movement area 305 and the information of the movement vector 303 are used as the code information related to the encoding target frame 301 as the thin client terminal 200. Output to -1 to n.

  When the thin client terminals 200-1 to 200-n receive the code information transmitted from the server device 100 and decode the frame based on the received code information, the thin client terminals 200-1 to 200-n perform a procedure reverse to the encoding. For example, when decoding the encoding target frame 301 of FIG. 8B from the reference frame 302 and the code information, first, the moving area 305 of the reference frame 302 is copied to the location indicated by the moving vector 303, and FIG. The motion-compensated reference frame 306 shown in a) is generated, and the encoding target frame 301 is generated by reflecting the update regions 307 to 312 decoded from the code information to the reference frame 306.

  In this way, the server apparatus 100 compares, for example, the encoding target frame 301 to be encoded shown in FIG. 8 with the encoded reference frame 302, and the encoding target frame 301 and the reference frame 302 are compared. An image area that exists in both areas and whose position on the screen is moving is detected as a movement area B. Then, the detected movement area B is copied to the position after movement on the reference frame 302, and a reference frame 306 after motion compensation shown in FIG. 9A is generated. Then, a different image area is detected between the generated reference frame 306 after copying shown in FIG. 9A and the encoding target frame 301 shown in FIG. 8B, and the position of the detected update area is detected. Based on the relationship between the coordinates and the coordinate position of the moving area B, the detected update area is further divided to detect update areas 307 to 312 shown in FIG. Then, the coordinate information of the update areas 307 to 312, the code information and the movement area information (coordinate information and movement vector information 303 of the movement area B before movement) are output to the thin client terminals 200-1 to 200 -n. Thereby, the server apparatus 100 can reduce the area detected as the update area and reduce the amount of transfer data output to the thin client terminals 200-1 to 200-n. For this reason, it is possible to reduce the load on the network NW and perform efficient data transfer.

<Update area detection area setting unit 24>
Next, the update area detection area setting unit 24 will be described with reference to FIG.

  The update area detection area setting unit 24 includes a moving area presence / absence determination unit 241 and a division unit 242 as shown in FIG.

  The moving region presence / absence determining unit 241 determines whether or not a moving region has been detected by the moving region detecting unit 22.

  The dividing unit 242 determines the necessity of screen division and sets an update region detection area by screen division according to the determination result of the moving region presence / absence determining unit 241.

<Example of processing operation of update area detection area setting unit 24>
Next, an example of processing operation of the update area detection area setting unit 24 will be described with reference to FIG.

  The moving region presence / absence determining unit 241 reads out the detection result of the moving region detecting unit 22 from the storage unit 10 and analyzes the detection result to determine whether or not a moving region has been detected (step S71).

  When the moving area is not detected (step S71 / No), the dividing unit 242 sets the entire screen as one update area detection area (step S72).

  If a movement area is detected (step S71 / Yes), the coordinate information of the movement area after movement included in the detection result of the movement area detection unit 22 is read from the storage unit 10, and the starting point of the movement area after movement is read. Then, a search process separation line is provided at the coordinate position of the end point, the screen is divided into three by the search process separation line (step S73), and each divided area is set as one update area detection area (step S74). Thereafter, the update area detection unit 25 detects the update area for each update area detection area.

  For example, as shown in FIG. 3, when the window A including the transparent outer frame portion C moves in the lower right direction, if the entire screen is set as one update area detection area, as shown in FIG. An update area D that includes the moving area B is detected. On the other hand, in the present embodiment, as shown in FIG. 12, a search processing separation line is provided at the coordinate position (nX1, nY1) of the start point and the coordinate position (nX2, nY2) of the end point of the movement area B after movement, Since the screen is divided into three by the search processing separation line, and the update area is detected in each update area detection area, as shown in FIG. 13, the total area of the areas to be detected as the update area can be reduced. The amount of codes can be reduced.

  In the MPEG standard or the like, encoding is performed in block units (for example, 4 × 4 pixel units, 8 × 8 pixel units, etc.). For this reason, the coordinate positions (nX1, nY1) and (nX2, nY2) of the moving region where the search processing separation line is provided may not exist on the boundary line of the block unit. In this case, an adjustment unit (not shown) is provided, and the coordinate positions (nX1, nY1) and (nX2, nY2) for providing the search processing separation lines are set so that the search processing separation lines exist on the boundary line of the block unit. Will be adjusted.

  For example, the adjustment of the coordinate positions (nX1, nY1) and (nX2, nY2) where the search process separation lines are provided is uniquely adjusted in the → direction shown in FIG. Adjust so that there is a separation line.

<Update area detector 25>
Next, the update region detection unit 25 will be described with reference to FIG.

  For each update region detection region set by the update region detection region setting unit 24, the update region detection unit 25 performs a motion-compensated reference frame (306 shown in FIG. 9A) and an encoding target frame (306). An image area that differs from 301) shown in FIG. 8B is detected as an update area. As illustrated in FIG. 14, the update area detection unit 25 includes an update area extraction unit 251 and a division unit 252.

  The update region extraction unit 251 calculates, for each update region detection region, a difference between pixel values at the same position in the reference frame after motion compensation and the encoding target frame, the calculated difference, and a predetermined threshold value. , Compare. Then, the update region extraction unit 251 extracts an image region including a pixel value in which a difference larger than a predetermined threshold is detected as an update region. Note that the predetermined threshold value can be arbitrarily changed, and can be set to 0, for example.

  The dividing unit 252 determines whether there is an update area including the coordinate position of the movement area B before and after the movement in the update area extracted by the update area extraction unit 251, and the coordinate position of the movement area B before and after the movement If there is an update area including the update area, a separation line is provided at the coordinate position where the area variation amount (ΔY) in the Y direction of the update area is the largest, and the update area is further divided via the separation line. The region variation amount (ΔY) means the variation amount of a region constituted by a pixel value in which the difference in pixel value between the reference frame after motion compensation and the encoding target frame is larger than a predetermined threshold (including 0). To do.

<Example of processing operation of update area detection unit 25>
Next, an example of the processing operation of the update region detection unit 25 will be described with reference to FIG.

  The update region extraction unit 251 calculates, for each update region detection region, a difference between pixel values at the same position in the reference frame after motion compensation and the encoding target frame, the calculated difference, and a predetermined threshold value. , Compare. Then, the update region extraction unit 251 extracts an image region including a pixel value in which a difference larger than a predetermined threshold is detected as an update region (Step S81).

  The dividing unit 252 determines whether there is an update area including the coordinate position of the movement area B before and after the movement in the update area extracted by the update area extraction unit 251 (step S82). If there is an update area that includes the B coordinate position (step S82 / Yes), a separation line is provided at the coordinate position where the area variation (ΔY) in the Y direction of the update area is the largest, and the update is performed via the separation line. The area is further divided (step S83).

  In the present embodiment, as shown in FIG. 12, a search processing separation line is provided at the start point (nX1, nY1) and end point (nX2, nY2) of the movement area B after movement, and the screen is divided into three by the search processing separation line. Then, the update area extraction unit 251 detects the update area for each update area detection area. For this reason, as shown in FIG. 13, the total area of the areas detected as update areas can be reduced, and the amount of codes can be reduced accordingly.

  Furthermore, in this embodiment, the dividing unit 252 determines that the update area extracted by the update area extraction unit 251 includes an update area including the coordinate position of the movement area B before and after the movement, in the Y direction of the update area. A separation line is provided at a coordinate position where the region variation amount (ΔY) is the largest, and the update region is further divided through the separation line.

  For example, as shown in FIG. 16, the update area 1 includes the coordinate positions (X1, Y1), (nX1, nY1), (nX2, nY1) of the movement area B before and after the movement, and divides the update area. (Xa, Ya) <(nX2, nY1) ≦ (Xb, Yb) and (Xa, Ya) ≦ (X1, Y1) <(Xb, Yb). The coordinate position of the start point obtained when extracting the update area 1 is (Xa, Ya), the coordinate position of the end point is (Xb, Yb), and the detection rectangle of the update area 1 is the start point (Xa , Ya) and an end point (Xb, Yb) (a rectangular region surrounded by a dotted line portion shown in FIG. 16). The coordinate position of the start point of the movement area B before movement is (X1, Y1), the coordinate position of the end point is (X2, Y2), and the movement area B before movement is its start point (X1, Y1) , The region surrounded by the end points (X2, Y2). The coordinate position of the start point of the movement area B after movement is (nX1, nY1), the coordinate position of the end point is (nX2, nY2), and the movement area B after movement is its start point (nX1, nY1) , A region surrounded by end points (nX2, nY2). For this reason, as shown in FIG. 17, a separation line is provided at the coordinate position (Xα, Yβ) where the region variation amount (ΔY) in the Y direction of the update region 1 is the largest, and the update region 1 is defined via the separation line. Further, as shown in FIG. 18, the update area 1 is divided into an update area 1 ′ and an update area 5.

  Further, as shown in FIG. 19, the update area 2 includes the coordinate positions (X1, nY1), (nX1, nY1), (nX1, nY2) of the movement area B before and after the movement, and divides the update area. (Xa, Ya) <(nX1, nY2) ≦ (Xb, Yb) and (Xa, Ya) ≦ (X1, nY1) <(Xb, Yb). The coordinate position of the start point obtained when extracting the update area 2 is (Xa, Ya), the coordinate position of the end point is (Xb, Yb), and the detection rectangle of the update area 1 is its start point (Xa , Ya) and an end point (Xb, Yb) (a rectangular region surrounded by a dotted line portion shown in FIG. 19). The coordinate position of the start point of the movement area B before movement is (X1, Y1), the coordinate position of the end point is (X2, Y2), and the movement area B before movement is its start point (X1, Y1) , The region surrounded by the end points (X2, Y2). The coordinate position of the start point of the movement area B after movement is (nX1, nY1), the coordinate position of the end point is (nX2, nY2), and the movement area B after movement is its start point (nX1, nY1) , A region surrounded by end points (nX2, nY2). The coordinate information (X1, nY1) used in the division condition is obtained from the coordinate information of the movement area B before and after the movement. For this reason, as shown in FIG. 20, a separation line is provided at the coordinate position (Xα, Yβ) where the region variation amount (ΔY) in the Y direction of the update region 2 is the largest, and the update region 2 is defined via the separation line. Further, as shown in FIG. 21, the update area 2 is divided into an update area 2 ′ and an update area 6.

  Thereby, as shown in FIG. 22, the update region detection unit 25 can further reduce the total area of the regions detected as the update region, compared with FIG. 13, and can reduce the code amount accordingly. In addition, since unnecessary update area information is not transmitted, the load on the network NW can be further reduced and efficient data transfer can be performed.

  In the MPEG standard or the like, encoding is performed in block units (for example, 4 × 4 pixel units, 8 × 8 pixel units, etc.). For this reason, the coordinate position (Xα, Yβ) of the moving region where the separation line is provided may not exist on the boundary line of the block unit. In this case, an adjustment unit (not shown) is provided, and the coordinate position (Xα, Yβ) where the separation line is provided is adjusted so that the separation line exists on the boundary line of the block unit.

  The adjustment direction of the coordinate position is uniquely determined according to the movement direction of the movement vector detected by the movement vector detection unit 21 or the movement direction of the movement region B detected by the movement region detection unit 22. For example, as shown in FIG. 8, when the movement direction of the movement vector detected by the movement vector detection unit 21 or the movement direction of the movement region B detected by the movement region detection unit 22 is lower right, a separation line The coordinate position (Xα, Yβ) to be provided is adjusted uniquely in the → direction shown in FIGS. 17 and 20 so that the separation line exists on the boundary line of the block unit to be encoded.

<Operation / Effect of Thin Client System of this Embodiment>
As described above, in the thin client system of the present embodiment, the server apparatus 100 compares the encoding target frame 301 to be encoded shown in FIG. 8 with the encoded reference frame 302, and encodes the encoding target frame. An image area that exists in both 301 and the reference frame 302 and that moves on the screen is detected as a movement area B. Then, the detected movement area B is copied to the position after movement on the reference frame 302, and a reference frame 306 after motion compensation shown in FIG. 9A is generated. Then, a rectangular image area that is different between the generated reference frame 306 after copying shown in FIG. 9A and the encoding target frame 301 shown in FIG. 8B is detected as an update area. Thereby, for example, the update areas 1 to 4 shown in FIG. 13 can be detected. Furthermore, the server apparatus 100 according to the present embodiment further divides the detected update areas 1 and 2 based on the relationship between the coordinate position of the moving area B and the detected coordinate positions of the update areas 1 to 4. To do. Thereby, update area | region 1 ', 2', and 3-6 shown in FIG. 22 are detectable.

  As a result, the server apparatus 100 can reduce the area detected as the update area and reduce the amount of transfer data output to the thin client terminals 200-1 to 200-n. For this reason, it is possible to reduce the load on the network NW and perform efficient data transfer.

  In the above embodiment, the separation line is provided at the coordinate position where the region variation amount (ΔY) in the Y direction of the update region is the largest, and the update region is further divided through the separation line. However, it is also possible to provide a separation line at the coordinate position where the region variation amount (ΔX) in the X direction of the update region is the largest, and further divide the update region via the separation line.

  In the above embodiment, as shown in FIG. 12, a search process separation line is provided at the start point (nX1, nY1) and end point (nX2, nY2) of the movement area B after movement, and the screen is displayed by the search process separation line. The update area extraction unit 251 performs the update area detection for each update area detection area.

  However, a search process separation line is provided only at the start point (nX1, nY1) of the movement area B after the movement, and the screen is divided into two by the search process separation line. It is also possible to detect the update area. Also in this case, since the update area detection unit 25 further performs update area division processing, the total area of the areas detected as update areas can be further reduced, and the amount of code can be reduced accordingly. .

  Also, search processing separation lines are provided at the start point (X1, Y1) and end point (X2, Y2) of the movement area B before movement, and at the start point (nX1, nY1) and end point (nX2, nY2) of the movement area B after movement. Then, the screen is divided into five by the search processing separation line, and the update region extraction unit 251 can detect the update region for each update region detection region. Also in this case, since the update area detection unit 25 further performs update area division processing, the total area of the areas detected as update areas can be further reduced, and the amount of code can be reduced accordingly. .

(Second Embodiment)
Next, a second embodiment will be described.

  The server device 100 according to the second embodiment performs the division determination based on the movement direction of the movement vector detected by the movement vector detection unit 21 or the movement direction of the movement region B detected by the movement region detection unit 22. Identify the update area to be performed, perform division determination based on the relationship between the coordinate position of the identified update area and the coordinate position of the movement area B detected by the movement area detection unit 22, and move area B before and after movement In the case of an update area including the coordinate position, the update area is further divided. Thereby, the update area | region which performs a division | segmentation determination can be reduced. As a result, it is possible to further reduce the processing load performed by the server apparatus 100. Hereinafter, the server device 100 of the present embodiment will be described with reference to the accompanying drawings. Since the configuration of the server device 100 is configured in the same manner as in the first embodiment, only the processing operation different from that in the first embodiment will be described.

<Example of processing operation of update area detection unit 25>
First, an example of the processing operation of the update region detection unit 25 will be described with reference to FIG.

  The update region extraction unit 251 calculates, for each update region detection region, a difference between pixel values at the same position in the reference frame after motion compensation and the encoding target frame, the calculated difference, and a predetermined threshold value. , Compare. Then, the update region extraction unit 251 extracts an image region including a pixel value in which a difference larger than a predetermined threshold is detected as an update region (Step S81).

  Based on the movement direction of the movement vector detected by the movement vector detection unit 21, the division unit 252 identifies an update region in which division determination is performed in the next step S82 (step S′811).

  For example, as shown in FIG. 24 (a), when the window is moved to the lower right, the movement direction of the movement vector is lower right. Specify the update area on the left.

  Also, as shown in FIG. 24B, when the window is moved to the lower left, the movement direction of the movement vector is the lower left, so the update areas for performing the division determination are the update areas 1 and 2 (upper and right). Update area).

  Also, as shown in FIG. 24 (c), when the window is moved to the upper left, the moving vector moves in the upper left direction, so the update areas for performing the division determination are the update areas 1 and 2 (lower and right sides). Update area).

  Also, as shown in FIG. 24 (d), when the window is moved to the upper right, the movement direction of the movement vector is to the upper right. Therefore, the update areas for performing the division determination are the update areas 1 and 2 (lower and left sides). Update area).

  As described above, the dividing unit 252 uniquely identifies the update region in which the division determination is performed in step S82 according to the moving direction of the movement vector detected by the movement vector detecting unit 21.

  Next, the dividing unit 252 determines whether or not there is an update area including the coordinate position of the movement area B before and after the movement in the update area specified in the step S′811 (Step S82). If there is an update area that includes the coordinate position of the previous and subsequent moving area B (step S82 / Yes), a separation line is provided at the coordinate position where the area variation amount (ΔY) in the Y direction of the update area is the largest, and the separation line The update area is further divided through (step S83).

<Operation / Effect of Thin Client System of this Embodiment>
As described above, in the thin client system according to the present embodiment, the server device 100 identifies the update area for performing the division determination based on the movement direction of the movement vector detected by the movement vector detection unit 21, and the identified update. Division determination is performed based on the relationship between the coordinate position of the area and the coordinate position of the movement area B detected by the movement area detection unit 22, and in the case of an update area including the coordinate position of the movement area B before and after the movement, The update area is further divided. Thereby, the update area | region which performs a division | segmentation determination can be reduced. As a result, it is possible to further reduce the processing load performed by the server apparatus 100.

  In the processing operation described above, the update region for performing the division determination is specified based on the movement direction of the movement vector detected by the movement vector detection unit 21. However, based on the movement direction of the movement area B before and after the movement detected by the movement area detection unit 22, it is also possible to specify the update area where the division determination is performed.

  The above-described embodiment is a preferred embodiment of the present invention, and the scope of the present invention is not limited to the above-described embodiment alone, and various modifications are made without departing from the gist of the present invention. Implementation is possible.

  For example, in the above-described embodiment, the movement vector detection unit 21 compares the encoding target frame with the reference frame, detects one main movement vector, and performs the above-described series of processing operations. However, it is also possible to construct such that a plurality of movement vectors are detected and the above-described series of processing operations are performed for each of the detected plurality of movement vectors. For example, when two movement vectors are detected, it is possible to construct so as to perform the above-described series of processing operations for each of the two movement vectors.

  In the above embodiment, the update area 1 is divided as (Xa, Ya) <(nX2, nY1) ≦ (Xb, Yb) and (Xa, Ya) ≦ (X1, Y1) <(Xb, Yb) is applied, and (Xa, Ya) <(nX1, nY2) ≦ (Xb, Yb) and (Xa, Ya) ≦ (X1, nY1) <(Xb, Yb) ) Was applied. However, as long as it is possible to determine whether or not the update area includes the coordinate position of the movement area B before and after the movement, the division condition is not limited to the above, and various division conditions can be applied. is there.

  Further, the control operation in each device constituting the server device 100 in the present embodiment described above can be executed using hardware, software, or a combined configuration of both.

  In the case of executing processing using software, it is possible to install and execute a program in which a processing sequence is recorded in a memory in a computer incorporated in dedicated hardware. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

  For example, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the program can be stored (recorded) temporarily or permanently in a removable recording medium. Such a removable recording medium can be provided as so-called package software. Examples of the removable recording medium include a floppy (registered trademark) disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

  The program is installed on the computer from the removable recording medium as described above. In addition, it is wirelessly transferred from the download site to the computer. In addition, it is transferred to a computer via a network by wire.

  In addition, the server device 100 in the present embodiment is not only executed in time series according to the processing operation described in the above embodiment, but also the processing capability of the device that executes the processing, or in parallel as necessary It can also be constructed to run individually.

  The present invention is applicable to a data transfer service.

100 server device (information processing device)
200-1 to 200 thin client terminal 10 storage unit 11 encoding target frame storage unit 12 reference frame storage unit 20 control unit 21 movement vector detection unit 22 movement region detection unit 23 motion compensation unit 24 update region detection region setting unit 241 movement region Presence / absence determination unit 242 Division unit 25 Update region detection unit 251 Update region extraction unit 252 Division unit 26 Region encoding unit 30 Communication unit

Claims (10)

The encoding target frame to be encoded is compared with the encoded reference frame, and the position on the screen is moved in an image area that exists in both the encoding target frame and the reference frame. A moving area detecting means for detecting a moving image area as a moving area;
Motion compensation means for copying the image of the movement area detected by the movement area detection means to the coordinate position after movement on the reference frame;
Update area detecting means for detecting an image area different between the reference frame after copying and the encoding target frame as an update area;
The update area detecting means includes
An information processing apparatus, wherein the update area is further divided based on a relationship between a coordinate position of the movement area detected by the movement area detection means and a coordinate position of the update area. The update area detecting means includes
The update area to be divided is identified based on the movement direction of the movement area detected by the movement area detection means, and the coordinate position of the identified update area and the movement area detected by the movement area detection means are determined. The information processing apparatus according to claim 1, wherein the specified update area is further divided based on a relationship with a coordinate position. The screen is divided based on the coordinate position of the moving area detected by the moving area detecting means, and each divided area is updated as a processing unit when the update area detecting means detects the update area. Having an update area detection area setting means for setting in the area detection area;
The update area detecting means includes
For each update area detection area set by the update area detection area setting means, an image area that is different between the copied reference frame and the encoding target frame is detected as an update area, and the detected update area 3. The information processing apparatus according to claim 1, wherein the update area is further divided based on a relationship between a coordinate position and a coordinate position of the movement area detected by the movement area detection unit. The update area detecting means includes
2. The update area is further divided when the coordinate position of the moving area is included in a rectangular area constituted by the coordinate positions of the start point and end point of the update area. 4. The information processing apparatus according to any one of items 1 to 3. The update area detecting means includes
In the update region, a separation line for dividing the update region is provided at the coordinate position where the region variation amount of the image region that is different between the reference frame after copying and the encoding target frame is the largest, The information processing apparatus according to any one of claims 1 to 4, wherein the update area is further divided through a provided separation line. The update area detection area setting means includes:
4. A separation line for dividing the screen is provided at the coordinate position of the movement area detected by the movement area detection means, and the screen is divided through the provided separation line. The information processing apparatus according to any one of 5.   The information processing apparatus according to claim 5, further comprising an adjusting unit that adjusts the coordinate position where the separation line is provided. The adjusting means includes
The information processing apparatus according to claim 7, wherein an adjustment direction for adjusting the coordinate position is determined based on a movement direction of the movement area detected by the movement area detection unit. A control method performed by an information processing apparatus,
The encoding target frame to be encoded is compared with the encoded reference frame, and the position on the screen is moved in an image area that exists in both the encoding target frame and the reference frame. A moving area detecting step of detecting a moving image area as a moving area;
A motion compensation step of copying the image of the movement region detected in the movement region detection step to the coordinate position after movement on the reference frame;
An update area detecting step of detecting an image area that is different between the reference frame after copying and the encoding target frame as an update area, and
The update region detection step includes
A control method characterized by further dividing the update area based on the relationship between the coordinate position of the movement area detected in the movement area detection step and the coordinate position of the update area. A program to be executed by an information processing apparatus,
The encoding target frame to be encoded is compared with the encoded reference frame, and the position on the screen is moved in an image area that exists in both the encoding target frame and the reference frame. Moving area detection processing for detecting a moving image area as a moving area;
A motion compensation process for copying the image of the moving area detected in the moving area detection process to the coordinate position after movement on the reference frame;
An update area detection process for detecting, as an update area, an image area that is different between the reference frame after copying and the encoding target frame,
The update area detection process includes:
A program characterized in that the update area is further divided based on the relationship between the coordinate position of the movement area detected by the movement area detection process and the coordinate position of the update area.

Images