JP5340772B2 - Image communication system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image communication system which does not require a camera to correct distortion, has favorable image quality and can obtained effects of reducing a communication data quantity. <P>SOLUTION: A camera 3 includes: an image pickup section 33 for generating a picked-up image by using a super-wide angle lens 31; and a region segmenting section 35 for segmenting a region of interest from the picked-up image and segmenting an expanded region wider than a background region including the region of interest. The expanded region is transmitted to a terminal device 5 at an update time interval larger than that of the region of interest. The terminal device 5 has an expanded region storage section 59 for storing the received expanded region, an image compositing section 57 for segmenting the background region so as to include the region of interest from the expanded region and compositing the region of interest to the background region, and a distortion correcting section 61 for correcting distortion generated due to image capturing using the super-wide angle lens 31. Every time the expanded region is received, the expanded region of the expanded region storage section 59 is updated. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to an image communication system including an imaging device including a wide-angle lens and an image receiving device that receives an image from the imaging device.

  2. Description of the Related Art Conventionally, there has been proposed an image communication system that artificially provides a pan / tilt / zoom function using an ultra-wide-angle lens or a fish-eye lens. In this type of system, the camera includes an ultra-wide angle lens or the like, and generates an image with a large angle of view. For example, an image having an angle of view close to 180 ° is generated. However, the captured image is distorted. In accordance with user designation or the like, a part of the region is cut out from the captured image, distortion is corrected, and the corrected region image is displayed.

  In the case of a system that actually includes a pan / tilt mechanism, the pan / tilt mechanism rotates the camera so as to face a desired shooting direction. In the system including the wide-angle lens described above, a region corresponding to the shooting direction is cut out from the captured image. Thereby, although the camera does not actually rotate, it seems to the user that the camera is rotating and photographing a desired direction. Furthermore, a zoom effect can be obtained by enlarging or reducing the image by image processing. Thus, a pseudo pan / tilt zoom function can be provided.

  An image communication system using an ultra-wide-angle lens or a fisheye lens is disclosed in, for example, Patent Document 1. In this prior art, a camera is provided in a server and a terminal device is provided in a client.

  Patent Document 1 discloses two types of configurations. In the first configuration, distortion of the captured image is corrected on the camera side, and a part of the region is cut out. Then, the corrected region image is encoded, and the encoded data is transmitted via the network. The terminal device receives the encoded data and decodes it to obtain image data. Then, an image is displayed on a monitor or the like.

  In the second configuration of Patent Document 1, clipping and distortion correction are performed on the terminal device (client) side. That is, the camera resizes (enlarges) and encodes the captured image, and transmits the encoded data via the network. The terminal device receives the encoded data and decodes it to obtain image data. Then, the terminal device corrects the distortion of the image, cuts out a necessary area, and displays it.

As described above, distortion correction is necessary in a system including an ultra-wide-angle lens and a fish-eye lens. The distortion correction processing is disclosed in Patent Document 2, for example.
JP 2006-148767 A Japanese Patent No. 3051173

  However, the conventional image communication system requires a large amount of calculation for distortion correction. For this reason, there is a problem that it is difficult to realize the configuration for performing clipping and distortion correction on the camera side from a viewpoint of power consumption and cost with a small camera or the like.

  Further, in the configuration in which clipping and distortion correction are performed on the terminal device side, the captured image is resized and encoded. Therefore, there is a problem that the image quality is deteriorated by resizing and encoding.

  In addition, a camera may be installed in a place where the background does not change, and the cutout region may be determined in accordance with the movement of a specific person or the like. In such an application, the image of the cutout area includes a background area that does not change around the person. For this reason, there is a problem that the amount of communication data is large and the network bandwidth is wasted.

  The present invention has been made to solve the conventional problems, and an object thereof is to provide an image communication system that does not require distortion correction on the camera side, has good image quality, and can reduce the amount of communication data. There is.

The image communication system of the present invention is an image communication system including an imaging device and an image receiving device that receives an image from the imaging device, wherein the imaging device generates a captured image using a wide-angle lens. parts and said to cut out the region of interest is a predetermined level or more motion is detected area from the captured image, more widely set than the background area including the region of interest with the position and size of the user is designated And an area cutout unit that cuts out an extended area including the background area , transmits the attention area to the image receiving device at a first predetermined update interval, and transmits the extended area to the first predetermined area. At a second predetermined update interval that is greater than the update interval , the image reception device includes: an extension region storage unit that stores the received extension region; and a center and a front of the attention region. An image synthesizing unit that cuts out the background region from the extended region so that the center of the background region coincides, and combines the cut-out background region with the received attention region, and photographing using the wide-angle lens And a distortion correction unit that corrects the distortion generated by the above, and updates the extension area of the extension area storage unit each time the extension area is received.

  With this configuration, the imaging apparatus cuts out the image of the attention area and the image of the extension area, and provides them to the image reception apparatus. The extended area is an area wider than the background area including the attention area. The extension area is provided with a larger update interval than the attention area, and is stored in the image receiving apparatus. The image receiving apparatus cuts out the background area from the stored extension area, and synthesizes the background area and the attention area. Since the extension area is wider than the background area, the background area can be cut out from the extension area even if the attention area moves. Therefore, the extension area can be repeatedly used for a plurality of attention areas that are sequentially provided. According to the present invention, the clipping process is performed on the imaging device side, and the distortion correction is performed on the image receiving device. Therefore, it is not necessary to perform a large amount of distortion correction processing in the imaging apparatus. In addition, resizing for sending the entire captured image is unnecessary, and degradation of image quality due to resizing can be avoided. In addition, since the extension area can be used repeatedly and the update interval of the extension area may be larger than the attention area, the amount of communication data is reduced. In this way, it is possible to provide an image communication system that does not require distortion correction on the camera side, has good image quality, and can reduce the amount of communication data.

Another aspect of the present invention is an imaging device, which is an imaging unit that generates a captured image using a wide-angle lens, and an area in which a motion of a predetermined level or more is detected from the captured image. and cut out the region, is the target set wider than the background area including the region with the position and size of the user is specified, and a region cutting unit for cutting out the extended region including the background region, the attention Transmitting a region at a first predetermined update interval, transmitting the extended region at a second predetermined update interval that is greater than the predetermined update interval, and combining the extended region with the region of interest And a communication unit that transmits the data to be extracted from the background area. This configuration also provides the above-described advantages of the present invention.

In the imaging apparatus according to the aspect of the invention, the area cutout unit may shift the center of the extension area from the center of the attention area in the movement direction of the attention area by a shift amount calculated from the movement speed of the attention area. Determine the cutting position.

  With this configuration, even when the attention area moves, the background area is more reliably included in the extension area. Therefore, the background area can be cut out with higher accuracy. In addition, the interval for providing the extended area can be increased.

  With this configuration, even when the attention area moves, the background area is more reliably included in the extension area. Therefore, the background area can be cut out with higher accuracy. In addition, the interval for providing the extended area can be increased.

In the imaging apparatus of the present invention, the second predetermined update interval is increased as the moving speed of the region of interest decreases, and the second predetermined update interval is decreased as the moving speed of the region of interest increases.

  With this configuration, when the moving speed of the attention area is fast, the update interval is shortened. Therefore, even when the attention area moves quickly, the background area can be more reliably cut out from the expanded area.

In the imaging apparatus according to the aspect of the invention, the region cutout unit may increase the second predetermined update interval as the amount of movement of the extension region decreases, and increase the second predetermined update interval as the amount of motion of the extension region increases. Reduce the update interval.

  With this configuration, the update interval of the extension region is changed according to the amount of movement of the extension region. Thereby, when a change occurs in the background portion, the extended region can be updated quickly.

  The imaging apparatus of the present invention further includes an encoding unit that encodes the image of the attention area and the image of the extension area, and assigns a code amount to the attention area preferentially over the extension area.

  With this configuration, the overall code amount can be reduced while maintaining the high image quality of the region of interest, and therefore the communication data amount can be further reduced.

Another aspect of the present invention is an image receiving apparatus, in which a position and a size are designated by an attention area and a user, which is an area in which a movement of a predetermined level or more of a captured image obtained using a wide-angle lens is detected. And a communication unit that is set wider than the background region including the region of interest and receives the extension region including the background region at different update intervals, an extension region storage unit that stores the extension region, and the extension An image composition unit that cuts out the background area from the extension area so that the center of the attention area matches the center of the background area from the area, and combines the cut-out background area and the received attention area A distortion correction unit that corrects distortion caused by photographing using the wide-angle lens, and each time the extension region is received, the extension region of the extension region storage unit Update. This configuration also provides the above-described advantages of the present invention.

  With this configuration, the attention area is located in the center of the background area, so that the visibility of the attention area can be improved.

  The image receiving apparatus of the present invention requests the update of the extension area when the background area cannot be cut out from the extension area.

  With this configuration, when the background area cannot be cut out from the extended area due to the movement of the attention area, the imaging apparatus is required to provide the extended area. Therefore, a new extension area can be received without waiting for reception of the next extension area, and a correct background image can be cut out.

Another aspect of the present invention is an image transmission method, in which an attention area that is an area in which a motion of a predetermined level or more is detected from a captured image, and the attention area are cut out and the position and size are determined by a user. Is specified and is set wider than the background area including the attention area, and an extended area including the background area is cut out, and the attention area is transmitted to the image reception device at a first predetermined update interval, The extended area is transmitted at a second predetermined update interval that is larger than the first predetermined update interval as the data of the cutout source of the background area to be combined with the attention area. This configuration also provides the above-described advantages of the present invention.

Another aspect of the present invention is an image receiving method, wherein a region of interest, which is a region in which a motion of a captured image at a predetermined level or more is detected, is received at a first predetermined update interval, and the position and size are determined by a user. Receiving a second predetermined update interval that is specified and is set wider than a background region that includes the region of interest, and includes the background region at a second predetermined update interval that is greater than the first predetermined update interval; An image obtained by cutting out the background area from the received extension area so that the center of the attention area matches the center of the background area, and synthesizing the extracted background area and the received attention area. Correct distortion. This configuration also provides the above-described advantages of the present invention.

  As described above, the present invention has an effect of providing a configuration for communicating and combining the attention area and the extension area, so that it is not necessary to perform distortion correction on the camera side, the image quality is good, and the amount of communication data can be reduced. An image communication system can be provided.

  Hereinafter, an image communication system according to an embodiment of the present invention will be described with reference to the drawings.

  An image communication system according to an embodiment of the present invention is shown in FIG. First, the outline of the present embodiment will be described with reference to FIG. In FIG. 1, the image communication system 1 has a configuration in which a camera 3 and a terminal device 5 are communicably connected via a network 7. The camera 3 and the terminal device 5 correspond to the imaging device and the image receiving device of the present invention, respectively. The network 7 is an IP network, for example. Since the camera 3 has a function of communicating images via the network 7, it is also referred to as a network camera.

  The camera 3 is provided with a super wide angle lens or a fisheye lens. The super-wide-angle lens or fish-eye lens corresponds to the wide-angle lens of the present invention, and has a very large angle of view (field of view), but is also a lens with a large image distortion due to a large angle of view. In the following description, it is assumed that the camera 3 includes a super wide angle lens. The camera 3 uses a super-wide-angle lens to generate a captured image 11 having a field angle close to 180 ° and corresponding to a hemisphere. The camera 3 cuts out a part of the captured image 11 to generate a region image 13, encodes the region image 13, and transmits it to the terminal device 5 via the network 7. In the terminal device 5, the region image 13 is decoded, distortion is corrected, and a corrected image 15 is obtained. The corrected image 15 is displayed on a monitor, for example.

  The image communication system 1 realizes a pan / tilt / zoom function. In a normal pan / tilt zoom mechanism, the camera is mechanically rotated by the pan / tilt mechanism. A zoom function is realized by the zoom lens. On the other hand, in the configuration of FIG. 1, the region image 13 is cut out from the captured image 11. Thereby, the same effect as the camera 3 rotating in the direction corresponding to the area image 13 is obtained. Further, it is possible to zoom in by cutting out the area image 13 smaller and displaying the corrected image in an enlarged manner, and conversely, zooming out display can be performed by cutting out the area image 13 larger and displaying the corrected image in a reduced size. The same effect as a zoom lens can be obtained. In this way, a pan / tilt zoom function can be realized without a mechanical pan / tilt mechanism and zoom lens. Such a pan / tilt zoom function is called software pan / tilt zoom (software PTZ).

  Next, with reference to FIG. 2, a cutout area in the image communication system 1 will be described. As shown in FIG. 2, in the present embodiment, two spaces, a captured image space and a corrected image space, are defined. The captured image space is a distorted image space obtained using an ultra-wide-angle lens. On the other hand, the corrected image space is an image space in which distortion is corrected.

  As shown in FIG. 2, in the present embodiment, three areas such as an attention area 21, a background area 23, and an extension area 25 are used.

  The attention area 21 is an area in which a subject to be noticed is shown. The subject is a person, for example. The attention area 21 may be determined by an instruction from the user, or may be determined based on a detection result by image recognition such as motion detection. In the present embodiment, the latter image recognition result is used. Motion detection is performed by a known technique. Then, an area in which a motion of a predetermined level or more is detected is set as the attention area 21. In the following description, the image of the attention area 21 is simply referred to as the attention area 21 (the same applies to the background area 23 and the expansion area 25 described below).

  The background area 23 is an area including the attention area 21. The background area 23 is an area corresponding to an image displayed on the terminal device 5.

  The extended area 25 is set wider than the background area 23. The extended area 25 is set wider than the background area 23 by a predetermined amount. The predetermined amount may be, for example, an area ratio or an area difference, or may be a length ratio (vertical, horizontal, etc.) or a difference. For example, if the area of the background region 23 is X, the area of the extended region 25 is set to αX. α is a coefficient of 1 or more, for example “2”.

  The camera 3 cuts out the attention area 21 and the extension area 25 among the above three areas. The attention area 21 and the extension area 25 are encoded and transmitted to the terminal device 5. In the terminal device 5, the background area 23 is cut out from the extended area 25. Then, the attention area 21 and the background area 23 are combined, the distortion is corrected, and displayed on the monitor.

  Within the scope of the present invention, distortion may be corrected after combining, and distortion correction may be performed before combining. That is, distortion may be corrected for each of the attention area 21 and the background area 23, and then the attention area 21 and the background area 23 may be combined. Further, the distortion of the extended area 25 may be corrected before the background area 23 is cut out.

  FIG. 3 shows the update interval of the attention area 21 and the extension area 25. The update interval is an image provision interval from the camera 3 to the terminal device 5. As illustrated, in the present embodiment, the update intervals of the attention area 21 and the extension area 25 are set differently.

  The attention area 21 is generated and transmitted every frame. For example, if the number of frames per second is 30, the update interval of the attention area 21 is 1/30 seconds. On the other hand, the update interval of the extension region 25 is set wider than the update interval of the attention region 21 and an interval corresponding to several frames.

  The terminal device 5 functions as follows in response to the difference in the update interval between the attention area 21 and the extension area 25 described above. When the attention area 21 and the extension area 25 are received simultaneously, the background area 23 is cut out from the extension area 25 and the background area 23 is combined with the attention area 21. The extended area 25 is stored in the terminal device 5. When only the attention area 21 is received, the background area 23 is cut out from the extended area 25 received in the past. Then, the attention area 21 and the background area 23 are synthesized.

  In the above, the extended area 25 can be used repeatedly. This repeated use is realized by setting the extended area 25 wider than the background area 23. Since the extension area 25 is wide, the background area 23 can be cut out from the extension area 25 even if the attention area 21 moves.

  According to the image communication system 1 described above, the clipping process is performed on the camera 3 side, and the distortion correction is performed on the terminal device 5. Therefore, it is not necessary to perform a large amount of distortion correction processing with the camera 3. In addition, resizing for sending the entire captured image is unnecessary, and degradation of image quality due to resizing can be avoided. In addition, since the extension area 25 can be used repeatedly and the update interval of the extension area 25 may be larger than the attention area 21, the amount of communication data is reduced. In this way, it is possible to provide the image communication system 1 that does not require distortion correction on the camera side, has good image quality, and can reduce the amount of communication data.

  The outline of the image communication system 1 according to the present embodiment has been described above. Next, the configuration of the image communication system 1 will be described in more detail.

  FIG. 4 shows the configuration of the camera 3. The camera 3 includes an ultra-wide-angle lens 31, an imaging unit 33, a region extraction unit 35, a video encoding unit 37, a network transmission / reception unit 39, and an image recognition unit 41. The region extraction unit 35 is a region of interest extraction. Part 43 and an extended region cutout part 45.

  The imaging unit 33 includes a solid-state imaging device such as a CCD or a CMOS, and generates an electrical image signal from an optical image obtained by the super wide-angle lens 31. The area cutout unit 35 cuts out the attention area 21 and the extended area 25 from the captured image 11 generated by the imaging unit 33. The attention area 21 and the expansion area 25 are cut out by the attention area cutting section 43 and the expansion area cutting section 45, respectively.

  In the present embodiment, the background area 23 is instructed from the terminal device 5 to the camera 3 as will be described in detail later. In addition, the image recognition unit 41 detects the motion of the captured image 11. The area cutout unit 35 cuts out the attention area 21 and the extended area 25 using the instruction of the background area 23 from the terminal device 5 and the motion detection result.

  That is, the area cutout unit 35 identifies a moving area in the background area 23 based on the motion detection result. This moving area is cut out as the attention area 21. The area cutout unit 35 cuts out the extended area 25 wider than the background area 23 so as to include the background area 23.

  The area cutout unit 35 performs the cutout process at a frequency according to the update interval of the attention area 21 and the extended area 25. The update interval of the attention area 21 is equal to the frame interval of the imaging unit 33. Therefore, the attention area 21 is cut out from the captured image 11 of each frame. The update interval of the extension area 25 corresponds to a predetermined number of frames. Accordingly, the expansion area 25 is cut out from the captured image 11 in a predetermined number.

  The video encoding unit 37 encodes the attention area 21 and the extended area 25 cut out by the area cutting unit 35.

  The network transmission / reception unit 39 is configured to transmit / receive data to / from the terminal device 5 via the network 7 and corresponds to a communication unit. The network transmission / reception unit 39 receives various instructions from the terminal device 5. Further, the network transmission / reception unit 39 transmits the attention area 21 and the extended area 25 encoded by the video encoding unit 37 to the terminal device 5.

  FIG. 5 shows the configuration of the terminal device 5. The network transmission / reception unit 51 is a communication unit similarly to the network transmission / reception unit 39 of the camera 3, and transmits / receives data to / from the terminal device 5 via the network 7. The network transmission / reception unit 51 transmits a user instruction and other control-related instructions to the camera 3. The network transmission / reception unit 51 receives the attention area 21 and the extension area 25 from the camera 3.

  The video decoding unit 53 decodes the attention area 21 and the extended area 25 received by the network transmission / reception unit 51.

  The background area control unit 55 performs various processes related to the background area 23. The background area control unit 55 receives an instruction for the background area 23 from the user instruction input unit 65, and supplies an instruction for the background area 23 to the camera 3 via the network transmission / reception unit 51.

  When the extended area 25 is received by the network transmission / reception unit 51, the background area control unit 55 controls the image composition unit 57 to perform processing for updating the extended area 25. The image composition unit 57 updates the extended area 25 stored in the extended area storage unit 59 in accordance with an instruction from the background area control unit 55. The extended area storage unit 59 is configured by a storage device of the terminal device 5.

  Further, the background area control unit 55 controls the image composition unit 57 to perform a process of cutting out the background area 23 from the extended area 25 and a process of synthesizing the background area 23 and the attention area 21. In this control, the background area control unit 55 instructs the cutout position of the background area 23 from the extended area 25. In the present embodiment, the cut-out position of the background area 23 is set so that the center of the background area 23 coincides with the attention area 21. Therefore, the cutout position of the background region 23 changes according to the movement of the attention region 21.

  The distortion correcting unit 61 corrects the distortion of the combined image of the attention area 21 and the background area 23 generated by the image combining unit 57. Here, distortion caused by using the super-wide-angle lens 31 is corrected.

  The display unit 63 is a monitor and displays an image after distortion correction. The user instruction input unit 65 is realized by a keyboard, a pointing device, or the like, and inputs a user instruction. In the present embodiment, the position and size of the background area 23 are designated using the monitor display.

  The configuration of each part of the image communication system 1 has been described above. Next, the overall operation of the image communication system 1 will be described.

  It is assumed that the position and size of the background area 23 are input to the user instruction input unit 65. The instruction for the background area 23 is supplied to the background area control unit 55, and the background area control unit 55 transmits the instruction for the background area 23 to the camera 3 via the network transmission / reception unit 51. The instruction of the background area 23 is received by the network transmission / reception unit 39 of the camera 3 and supplied to the area extraction unit 35.

  In the camera 3, the imaging unit 33 generates the captured image 11 using the super-wide-angle lens 31 and supplies the captured image 11 to the region extraction unit 35 and the image recognition unit 41. The image recognition unit 41 performs a motion detection process on the captured image 11 and supplies the detection result to the region cutout unit 35.

  In the region cutout unit 35, the region of interest cutout unit 43 cuts out the region of interest 21 from the captured image 11, and the extended region cutout unit 45 cuts out the extended region 25 from the captured image 11. The attention area 21 is a moving area in the background area 23. Once the attention area 21 is specified, the attention area 21 may be tracked based on the motion detection result in subsequent frames. Further, as described above, the extended area 25 is cut out so as to include the background area 23 specified by the user. The attention area 21 is cut out from each frame, and the expansion area 25 is cut out from one frame in a predetermined number.

  The attention area 21 and the extended area 25 are supplied from the area cutout section 35 to the video encoding section 37 and encoded by the video encoding section 37. The encoded data is supplied to the network transmission / reception unit 39 and transmitted to the terminal device 5 via the network 7.

  In the terminal device 5, the network transmission / reception unit 51 receives encoded data from the network 7. As described above, the attention area 21 is cut out from all frames, and the extended area 25 is cut out from one frame in a predetermined number. Therefore, the data of a plurality of attention areas 21 is received for the data of one extension area 25.

  The attention area 21 and the extension area 25 are decoded by the video decoding section 53 and supplied to the background area control section 55.

  The background area control unit 55 supplies the data of the extension area 25 to the image composition unit 57 and instructs to update the extension area 25. In response to this instruction, the image composition unit 57 rewrites the extended area 25 stored in the extended area storage unit 59.

  The background region control unit 55 supplies the attention region 21 obtained sequentially to the image composition unit 57. Further, the background area control unit 55 causes the image composition unit 57 to cut out the background area 23 from the extended area 25 so that the background area 23 includes the attention area 21. For each region of interest 21, a background region 23 is cut out from the same extended region 25. Then, the image composition unit 57 synthesizes each extracted background area 23 with the corresponding attention area 21.

  Here, the attention area 21 is a moving area specified by motion detection, and therefore the position of the attention area 21 changes for each frame. Therefore, as the attention area 21 moves, the cutout position of the background area 23 needs to change for each frame. This process is realized as follows.

  From the camera 3 to the terminal device 5, the coordinate information of the attention area 21 (on the captured image space) and the coordinate information of the extension area 25 (on the captured image space) are transmitted together with the image signals of those areas. The coordinate information is, for example, coordinate data of the four vertices of the region, and specifies the position and range of the region (other data that can specify the region position may be used instead of the coordinate information). The coordinate information is acquired by the image composition unit 57 via the network transmission / reception unit 51 and the video decoding unit 53. The background area control unit 55 determines the cutout position of the background area 23 using the coordinate information so that the attention area 21 is included in the background area 23.

  In particular, in the present embodiment, it is preferable to cut out the background area 23 so that the attention area 21 is located at the center of the background area 23 after distortion correction, and the visibility of the attention area 21 can be improved. Therefore, in consideration of distortion correction, the cutout position is determined so that the center of the attention area 21 and the position of the background area 23 coincide.

  The background area control unit 55 supplies the cutout position of the background area 23 together with the image of the attention area 21 to the image composition unit 57. The image composition unit 57 cuts out the background region 23 from the extended region 25 of the extended region storage unit 59 in accordance with an instruction from the background region control unit 55.

  The image composition unit 57 composes the background area 23 cut out as described above with the attention area 21. The composite image is supplied to the distortion correction unit 61 and distortion is corrected. The corrected composite image is supplied to the display unit 63 and displayed.

  With the above operation, the terminal device 5 repeatedly uses the same extended area 25 while changing the cutout position of the background area 23 until a new extended area 25 is received. Since the extension area 25 is wider than the background area 23, the extension area 25 can be used even if the attention area 21 moves. When a new extension area 25 is received, the extension area 25 of the extension area storage unit 59 is updated.

  Next, the setting of the cutting position of the extended area 25 in the area cutting part 35 of the camera 3 will be described in more detail. The cut-out position of the extension area 25 may be set so that the attention area 21 is located at the center of the extension area 25. However, more preferably, the cutout position of the expansion region 25 is adjusted based on the moving direction and moving speed of the attention region 21.

  The moving direction and moving speed of the attention area 21 are obtained from the motion detection result of the image recognition unit 41. The moving speed may be an average moving amount of blocks constituting the attention area 21 (here, the block is preferably a macro block which is a basic unit of encoding). The area cutout unit 35 shifts the center of the cutout position of the extended area 25 from the center of the attention area 21 in the moving direction of the attention area 21. The area cutout unit 35 increases the shift amount of the cutout position as the moving speed of the attention area 21 increases. For example, the movement amount of the attention area 21 in the update interval of the extension area 25 is obtained from the movement speed. If the amount of movement is “L”, the center of the cutout position is shifted by “L” in the same direction.

  By setting the attention area 21 in this way, even when the attention area 21 moves, the background area 23 is more reliably included in the extension area 25. Therefore, the background region 23 can be cut out with higher accuracy. In addition, the update interval of the extension area 25 can be increased.

  Next, the setting of the update interval of the extended area 25 will be further described. The update interval of the extension area 25 corresponds to a predetermined number of frames. This update interval may be fixed. However, more preferably, the update interval is changed according to the moving speed of the attention area 21.

  The moving speed of the attention area 21 is obtained from the motion detection result of the image recognition unit 41 as described above. The area cutout unit 35 increases the update interval as the moving speed decreases, and decreases the update interval as the moving speed increases. That is, the update frequency decreases as the moving speed decreases, and the update frequency increases as the moving speed increases.

  Such a setting provides the following advantages. When the moving speed of the attention area 21 is low, the attention area 21 is unlikely to go out of the expansion area 25 even if time passes. This means that the extended area 25 can be used for a longer time. Therefore, the update interval is set large, and the communication data amount is reduced. On the other hand, when the moving speed of the attention area 21 is high, the attention area 21 moves out of the expansion area 25 in a short time. Therefore, the update interval is set to be small so that the attention area 21 is more reliably included in the expansion area 25. Thus, the background area 23 can be more reliably cut out from the extended area 25 while suppressing the amount of communication data.

  Further, the update interval of the extension area 25 may be further set according to the amount of movement of the extension area 25. The amount of movement of the extended area 25 is also obtained from the image recognition result of the image recognition unit 41. The greater the amount of motion, the smaller the update interval (increase the update frequency). For example, the amount of motion of the expansion area 25 is compared with the threshold value Z. If the amount of motion exceeds the threshold value Z, the update interval is shortened to ½. This doubles the update frequency. By such processing, when the background portion changes, the extended area 25 can be updated quickly.

  Next, detailed operation of the image communication system 1 of the present embodiment will be described with reference to a flowchart.

  6 and 7 show the processing of the background area control unit 55 of the terminal device 5. As already described, the background area control unit 55 generally performs the following three processes. (1) An instruction about the background area 23 is obtained from the user instruction input unit 65 and sent to the terminal device 5. (2) When the extended area 25 is received, the image composition unit 57 is instructed to update the extended area 25. (3) Determine the cut-out position of the background region 23 for each region of interest 21 and instruct the image composition unit 57. These processes are realized by FIG. 6 and FIG.

  The background area control unit 55 determines whether there is an event (S1), and if there is no event, repeats the determination in step S1. If there is an event, it is determined whether or not the event is an end event (S3). If it is an end event, the process ends.

  If the event is not an end event, the background region control unit 55 determines whether or not the generated event is a background region instruction event from the user instruction input unit 65 (S5). The background area instruction event occurs when an instruction for the background area 23 is input to the user instruction input unit 65 by the user.

  Here, in the present embodiment, the software PTZ is realized as already described. The user instruction input unit 65 is configured to be able to input PTZ information (pan direction, tilt direction, and zoom magnification). This PTZ information corresponds to the range of the image to be displayed and is input as an instruction for the background area 23. The user instruction input unit 65 may be configured by an arbitrary user interface capable of inputting arbitrary information indicating the pan direction, the tilt direction, and the zoom magnification.

  When the PTZ information is input to the user instruction input unit 65, the user instruction input unit 65 supplies the PTZ information to the background area control unit 55, and step S5 becomes YES. Then, the background area control unit 55 determines a corrected image space based on the PTZ information (S7). Conceptually, the corrected image space is a plane (two-dimensional space) perpendicular to the imaging direction specified by the pan direction and the tilt direction, and is separated by a distance corresponding to the zoom magnification. Then, the background area control unit 55 acquires coordinates for determining the position of the background area 23 in the corrected image space (S9). These coordinates are, for example, the coordinates of the four vertices in the rectangular background region 23. Since the PTZ information and the background area 23 have a one-to-one correspondence, the coordinates of the background area 23 (in the corrected image space) are obtained from the PTZ information.

  Then, the background area control unit 55 transmits a coordinate information transmission event to the network transmission / reception unit 51 (S11), and returns to step S1. The coordinate information transmission event includes the coordinate information (on the corrected image space) of the background area 23 obtained in step S9. The network transmission / reception unit 51 sends the coordinate information of the background region 23 to the network 7 in response to the coordinate information transmission event from the background region control unit 55.

  On the other hand, in the case of NO in step S5, the background area control unit 55 determines whether or not the generated event is a processing event from the video decoding unit 53 (S13). If step S13 is NO, the process returns to step S1. When the attention area 21 or the extended area 25 is received by the network transmission / reception unit 51, decoded by the video decoding unit 53, and supplied to the background region control unit 55, step S13 becomes YES. Then, the background area control unit 55 determines whether or not the image of the extension area 25 has been decoded (S15). If step S15 is YES, the background region control unit 55 transmits an extended region update event to the image composition unit 57 (S17), and returns to step S1. The extended area update event includes an image of the extended area 25 and coordinate information (on the imaging space) of the extended area 25. The coordinate information is, for example, the coordinates of four points in the extension area 25, and specifies the position and range of the extension area 25.

  When step S15 is NO, the image of the attention area 21 is decoded by the video decoding unit 53. The background area control unit 55 acquires the image of the attention area 21 and the coordinate information (on the captured image space) of the attention area 21 from the video decoding section 53. Here, the coordinate information is the coordinates of the four points of the attention area 21, and the position and range of the attention area 21 are specified. Then, the background area control unit 55 determines a corrected image space from the coordinate information (on the captured image space) of the attention area 21 (S19). Here, it is difficult to uniquely determine the corrected image space from the coordinates on the captured image space (since it is not one-to-one correspondence). For example, the corrected image space is calculated under the condition that the distance between the corrected image space and the center of the captured image space is kept constant. Apart from the corrected image space determined in step S7 described above, a corrected image space is newly determined from the region of interest 21.

  Further, the background area control unit 55 determines a position where the attention area 21 is to be synthesized on the captured image space (S21). Further, the background area control unit 55 determines the background area 23 on the captured image space coordinates based on the coordinate information of the attention area 21 (S23). Here, the coordinates of the background area 23 are determined so that the attention area 21 is located at the center of the background area 23. This coordinate represents the cutout position of the background area 23 from the extended area 25. Apart from the coordinates of the background area 23 according to the user instruction described above, the background area 23 is determined according to the attention area 21. Then, the background area control unit 55 transmits an image composition event to the image composition unit 57 (S25), and returns to step S1. The image synthesis event is determined in step S23, the image of the attention area 21, the corrected image space determined in step S19, the coordinates (on the captured image space) where the attention area 21 determined in step S21 is to be combined. And the cutout coordinates (on the captured image space) of the background area 23.

  Next, FIG. 8 shows processing of the image composition unit 57 of the terminal device 5. As already described, the image composition unit 57 generally performs the following two processes. (1) When a new extension area 25 is received, the extension area 25 held in the extension area storage unit 59 is updated. (2) When the attention area 21 is received, the background area 23 is cut out from the extended area 25 and the background area 23 and the attention area 21 are combined. These processes are realized by FIG.

  The image composition unit 57 determines whether there is an event (S31), and if there is no event, repeats the determination in step S31. If there is an event, it is determined whether or not the event is an end event (S33). If it is an end event, the process ends.

  If the event is not an end event, the image composition unit 57 determines whether the generated event is an extended region update event (S35). If the background area control unit 55 transmits an extended area update event to the image composition unit 57 in step S17 of FIG. 7, step S35 is YES. The extended area update event includes an image of the extended area 25 and coordinate information (on the captured image space). The image composition unit 57 rewrites the image and coordinate information of the extension area 25 stored in the extension area storage unit 59 (S37), and returns to step S31.

  On the other hand, when step S35 is NO, the image composition unit 57 determines whether or not the generated event is an image composition event (S39). If step S39 is NO, the process returns to step S31. When the background area control unit 55 transmits an image composition event to the image composition unit 57 in step S25 of FIG. 7, step S39 becomes YES. The image synthesis event includes an image of the attention area 21, a corrected image space, a position where the attention area 21 is to be synthesized, and designation of the cut-out coordinates (on the captured image space) of the background area 23. The image composition unit 57 cuts out the background area 23 from the extension area 25 stored in the extension area storage section 59 at the specified coordinates (cutout position) (S41), and the specified coordinates on the background area 23 are displayed. To synthesize the attention area 21 (S43) and return to step S31. The composite image is supplied from the image composition unit 57 to the distortion correction unit 61, subjected to distortion correction, and displayed on the display unit 63.

  Next, FIG. 9 and FIG. 10 show processing of the area cutout unit 35 of the camera 3. The region cutout unit 35 generally performs the following four processes. (1) The extended area 25 is determined from the background area 23 designated by the user. (2) The attention area 21 is determined from the background area 23 and the motion information. (3) The extended area 25 is cut out. (4) The attention area 21 is cut out. These processes are realized by FIG. 9 and FIG.

  The area cutout unit 35 determines whether or not there is an event (S51), and if there is no event, repeats the determination in step S51. If there is an event, it is determined whether or not the event is an end event (S53). If it is an end event, the process ends.

  If the event is not an end event, the area cutout unit 35 determines whether or not the generated event is a coordinate information transmission event from the terminal device 5 (S55). When the coordinate information transmission event is transmitted by the background area control unit 55 in step S11 of FIG. 6, step S55 becomes YES. The coordinate information transmission event includes coordinate information of the background area 23 in the corrected image space. The area cutout unit 35 converts the coordinates of the background area 23 in the corrected image space into the coordinates of the captured image space (S57). Further, the region cutout unit 35 acquires motion information of the captured image space from the image recognition unit 41 (S59). Then, the background area control unit 55 determines the coordinate information of the extended area 25 using the motion information (S61). Here, the extended area 25 includes the background area 23 and is set wider than the background area 23. Then, the center of the extension area 25 is shifted with respect to the center of the attention area 21. The shift direction and shift amount are determined according to the moving direction and moving speed of the attention area 21. The displacement direction is matched to the movement direction. The shift amount is set in accordance with the estimated movement amount of the attention area 21 during the period in which the same extension area 25 is used. This estimated movement amount is obtained from the update interval of the extension area 25 and the movement speed of the attention area 21. The area cutout unit 35 holds the coordinate information of the extended area 25 of the captured image space determined in step S61 (S63), and returns to step S61. The information in the extension area 25 is used when sending the area in the subsequent stage.

  Furthermore, the coordinate information of the extension area 25 is repeatedly updated with an update interval of the extension area 25. In the update process, the coordinate information (on the captured image space) of the extension region 25 is determined again from the latest coordinate information (on the captured image space) of the attention area 21. Similar to the above-described processing, based on the motion information of the attention area 21, the position of the extension area 25 is determined so that the centers of the attention area 21 and the extension area 25 are shifted. Information on the updated extended area 25 is also held in the area cutout section 35.

  On the other hand, when step S55 is NO, the area cutout unit 35 determines whether or not the generated event is an attention area setting event (S65). In the attention area setting event, the area cutout section 35 acquires motion information from the image recognition section 41 (S67), and sets an area with a large amount of motion as the attention area 21 (S69). Then, the area cutout unit 35 holds the coordinate information of the attention area 21 determined in step S69 for use when sending out the area (S71).

  The coordinate information of the attention area 21 is repeatedly updated at intervals of updating the attention area 21, that is, for each frame. Here, the attention area 21 set in step S69 is tracked based on the motion information. Then, at the timing of each frame, the latest coordinates of the attention area 21 are determined. Information on the updated attention area 21 is also held in the area cutout section 35.

  On the other hand, when step S65 is NO, the area cutout unit 35 determines whether or not the generated event is an area sending event (S73). If step S73 is NO, the process returns to step S51. If step S73 is YES, the area cutout unit 35 determines whether or not the transmission timing of the extended area 25 has arrived (S75). If step S75 is YES, the area cutout unit 35 uses the information on the extension area 25 held in step S63 to cut out the extension area 25 from the captured image 11 (S77), and displays the extension area 25 as a video. It transmits to the encoding part 37 (S79), and returns to step S51.

  If step S75 is NO, the area cutout unit 35 determines whether or not the sending timing of the attention area 21 has arrived (S81). If step S81 is NO, the process returns to step S51. If step S81 is YES, the area cutout unit 35 uses the information on the attention area 21 held in step S71 to cut out the attention area 21 from the captured image 11 (S83), and displays the attention area 21 as a video. It transmits to the encoding part 37 (S85), and returns to step S51.

  In FIG. 3, in a frame in which both the attention area 21 and the extension area 25 are to be updated, the update timing of the attention area 21 and the extension area 25 arrives at the same time, and the processing in steps S77 and S79 and the processing in steps S83 and S85 are performed. Processing is performed continuously. Further, in the frame where only the attention area 21 is to be updated, only the processes of steps S83 and S85 are performed.

  The operations of the background region control unit 55, the image composition unit 57, and the region cutout unit 35 have been described in detail above with reference to FIGS. According to the above processing flow, when the user inputs the PTZ information to the terminal device 5 as the information of the background region 23, the coordinate information of the background region 23 is transferred from the terminal device 5 to the camera 3 by the processing flow of FIG. Sent. Then, the extended region 25 and the region of interest 21 are determined and cut out from the captured image 11 by the processing flows of FIGS. 9 and 10. The expanded area 25 and the attention area 21 cut out are transmitted from the camera 3 to the terminal device 5. 7 and 8, when a new extension area 25 is received, the extension area 25 of the extension area storage unit 59 is rewritten. When the attention area 21 is received, the background area 23 is cut out from the extended area 25, and the cut out background area 23 is combined with the attention area 21. In this way, the operation of the image communication system 1 is realized by the processing flows of FIGS.

  Next, a modification of the present embodiment will be described. As described above, in the present embodiment, the extension area 25 is set wider than the background area 23, and the background area 23 is repeatedly used from the extension area 25. However, since the moving speed of the attention area 21 is high, the background area 23 may not be cut out from the extended area 25. In such a case, there arises a problem that the correct background area 23 cannot be cut out until the next extension area 25 is updated.

  Therefore, in this modification, when the background area 23 cannot be cut out from the extended area 25 in the terminal device 5, the terminal apparatus 5 requests the camera 3 to update the extended area 25. As a result, the correct background area 23 can be cut out without waiting for the update of the next extension area 25.

  FIG. 11 shows a processing flow for realizing the above update request. FIG. 11 is a processing flow of the background area control unit 55 of the terminal device 5, and steps S91 to S95 are added to FIG. In step S91, the background area control unit 55 refers to the coordinate information (on the captured image space) of the background area 23 determined in step S23, and determines whether or not the background area 23 is in the extended area 25. If the background area 23 is included in the extended area 25, an image composition event is transmitted to the image composition unit 57 as in the above-described embodiment.

  If the background area 23 is not included in the extended area 25, the correct background area 23 cannot be cut out. Therefore, the background area 23 is recalculated (S93). The background area control unit 55 converts the background area 23 on the captured image space coordinates determined in step S23 into coordinate information in the corrected image space. Then, the background area control unit 55 transmits a coordinate information transmission event to the network transmission / reception unit 51 (S95). This process is the same as step S11 in FIG. The coordinate information of the background area 23 after recalculation is transmitted to the camera 3. Then, in the camera 3, the processes of FIGS. 9 and 10 are performed, and the new extended area 25 is sent back to the terminal device 5.

  Next, another modification of the present embodiment will be described. In this modification, the processing of the video encoding unit 37 is changed, and a code amount is preferentially assigned to the attention area 21 rather than the extension area 25. As a result, the overall code amount can be reduced while maintaining the high image quality of the attention area 21, and therefore the communication data amount can be further reduced.

  FIG. 12 is a processing flow of the video encoding unit 37 and shows processing for realizing the above-described code amount allocation control.

  The video encoding unit 37 determines whether there is an event (S101), and if there is no event, repeats the determination in step S101. If there is an event, it is determined whether or not the event is an end event (S103). If it is an end event, the process ends.

  If the event is not an end event, the video encoding unit 37 determines whether the generated event is a video encoding event (S105). If step S105 is YES, the video encoding unit 37 determines whether or not the image to be encoded is the extended region 25 (S107). If step S107 is YES, the video encoding unit 37 suppresses the generated code amount (S109). Here, for example, the quantization step is changed to a value larger by a predetermined width. Alternatively, the code amount may be reduced by another method.

  On the other hand, if step S107 is NO, the video encoding unit 37 determines whether or not the image to be encoded is the attention area 21 (S111). If step S111 is NO, the process returns to step S101. If step S111 is YES, the video encoding unit 37 increases the generated code amount (S113). Here, for example, the quantization step is changed to a value smaller by a predetermined width. Alternatively, the code amount may be increased by another method.

  The image communication system 1 according to the embodiment of the present invention has been described above. According to the present embodiment, the imaging device (camera 3) cuts out the image of the attention area 21 and the image of the extension area 25 and provides them to the image reception apparatus (terminal device 5). The extended area 25 is an area wider than the background area 23 including the attention area 21. The extension area 25 is provided with a larger update interval than the attention area 21 and is stored in the image receiving apparatus. The image receiving apparatus cuts out the background area 23 from the stored extension area 25 and combines the background area 23 and the attention area 21. Since the extension area 25 is wider than the background area 23, the background area 23 can be cut out from the extension area 25 even if the attention area 21 moves. Therefore, the extended area 25 can be repeatedly used for a plurality of attention areas 21 that are sequentially provided.

  Therefore, according to the present embodiment, since the clipping process is performed on the imaging apparatus side and the distortion correction is performed on the image receiving apparatus, it is not necessary to perform a large amount of distortion correction processing on the imaging apparatus. In addition, resizing for sending the entire captured image 11 is not necessary, and deterioration in image quality due to resizing can be avoided. In addition, since the extension area 25 can be used repeatedly and the update interval of the extension area 25 may be larger than the attention area 21, the amount of communication data is reduced. In this way, it is possible to provide an image communication system that does not require distortion correction on the camera side, has good image quality, and can reduce the amount of communication data.

  In the present embodiment, the cut-out position of the expansion area 25 is determined according to the moving direction of the attention area 21. Further, the cut-out position of the expansion area 25 may be determined according to the moving speed of the attention area 21. The moving direction and moving speed may be obtained from the image recognition result of the captured image 11. With such a configuration, even when the attention area 21 moves, the background area 23 is more reliably included in the extension area 25. Therefore, the background region 23 can be cut out with higher accuracy. In addition, the interval for providing the extended region 25 can be increased.

  In the present embodiment, the update interval of the extension area 25 is changed according to the moving speed of the attention area 21. The moving speed may be obtained from the image recognition result of the captured image 11. With this configuration, when the moving speed of the attention area 21 is fast, the update interval is shortened. Therefore, even when the attention area 21 moves quickly, the background area 23 can be more reliably cut out from the expansion area 25.

  In the present embodiment, the update interval of the extension region 25 is changed according to the amount of movement of the extension region 25. The amount of motion may be obtained from the image recognition result of the captured image 11. Thereby, when a change occurs in the background portion 23, the extended area 25 can be updated quickly.

  In the present embodiment, when the image of the attention area 21 and the image of the extension area 25 are encoded, the code amount is preferentially assigned to the attention area 21 rather than the extension area 25. As a result, the overall code amount can be reduced while maintaining the high image quality of the attention area 21, and therefore the communication data amount can be further reduced.

  In the present embodiment, the background area 23 is cut out from the extended area 25 so that the attention area 21 is located at the center of the background area 23. Thereby, since the attention area 21 is located in the center of the background area 23, the visibility of the attention area 21 can be improved.

  In the present embodiment, when the background area 23 cannot be cut out from the extended area 25, the imaging apparatus is requested to update the extended area 25. Thus, when the background area 23 cannot be cut out from the extension area 25 due to a sudden movement of the attention area 21, a new extension area 25 can be received without waiting for the reception of the next extension area 25, and the correct background The image 23 can be cut out.

  The preferred embodiments of the present invention have been described above. However, the present invention is not limited to the above-described embodiments, and it goes without saying that those skilled in the art can modify the above-described embodiments within the scope of the present invention.

  As described above, the image communication system according to the present invention does not need to perform distortion correction on the camera side, has an effect that the image quality is good, and the amount of communication data can be reduced. Useful as.

The figure which shows the image communication system in embodiment of this invention The figure explaining an attention area, a background area, and an expansion area The figure explaining the update space | interval of an attention area and an expansion area Block diagram showing the configuration of the camera of the image communication system The block diagram which shows the structure of the terminal device of an image communication system Flow chart showing operation of background area control unit of terminal device Flow chart showing operation of background area control unit of terminal device Flow chart showing operation of image composition unit of terminal device Flow chart showing the operation of the camera area cutout Flow chart showing the operation of the camera area cutout Flow chart showing operation of background area control unit in modification Flow chart showing operation of video encoding unit in modified example

DESCRIPTION OF SYMBOLS 1 Image communication system 3 Camera 5 Terminal device 7 Network 11 Captured image 21 Attention area 23 Background area 25 Expansion area 31 Super wide-angle lens 33 Imaging part 35 Area extraction part 37 Video encoding part 39, 51 Network transmission / reception part 41 Image recognition part 43 attention area cutout section 45 extended area cutout section 53 video decoding section 55 background area control section 57 image composition section 59 extended area storage section 61 distortion correction section 63 display section 65 user instruction input section

Claims (10)

An image communication system comprising an imaging device and an image receiving device that receives an image from the imaging device,
The imaging device
An imaging unit that generates a captured image using a wide-angle lens;
Wherein to cut out the region of interest is a predetermined level or more motion is detected area from the captured image is the still than the background region including the region of interest set wider with the position and size of the user is specified, and An area cutout section for cutting out the extended area including the background area ,
Transmitting the region of interest to the image receiving device at a first predetermined update interval, and transmitting the extension region to the image receiving device at a second predetermined update interval that is greater than the first predetermined update interval ;
The image receiving device includes:
An extended area storage unit for storing the received extended area;
An image composition unit that cuts out the background area from the extension area so that the center of the attention area matches the center of the background area, and combines the extracted background area and the received attention area ;
A distortion correction unit that corrects distortion caused by photographing using the wide-angle lens,
An image communication system, wherein the extension area of the extension area storage unit is updated each time the extension area is received. An imaging unit that generates a captured image using a wide-angle lens;
Wherein to cut out the region of interest is a predetermined level or more motion is detected area from the captured image is the still than the background region including the region of interest set wider with the position and size of the user is specified, and An area cutout section for cutting out an extended area including the background area ;
The attention area is transmitted at a first predetermined update interval, the extension area is transmitted at a second predetermined update interval that is larger than the predetermined update interval, and the extension area is combined with the attention area. An image pickup apparatus comprising: a communication unit that transmits the data as the original data of the background area to be extracted. The region cutout unit determines a cutout position by shifting the center of the extended region from the center of the region of interest in the movement direction of the region of interest by a shift amount calculated from the moving speed of the region of interest. The imaging apparatus according to claim 2. The area cutout section increases the second predetermined update interval as the moving speed of the attention area decreases, and decreases the second predetermined update interval as the moving speed of the attention area increases. The imaging device according to claim 2. The area cutout section increases the second predetermined update interval as the amount of movement of the extension region decreases, and decreases the second predetermined update interval as the amount of movement of the extension region increases. The imaging device according to claim 2.   The imaging according to claim 2, further comprising an encoding unit that encodes the image of the attention area and the image of the extension area, and assigning a code amount to the attention area preferentially over the extension area. apparatus. An attention area that is an area in which a motion of a predetermined level or more of a captured image obtained using a wide-angle lens is detected, and a position and size are designated by the user, and are set wider than a background area that includes the attention area. And a communication unit that receives the extension area including the background area at different update intervals;
An extended area storage unit for storing the extended area;
Image composition that cuts out the background area from the extension area so that the center of the attention area matches the center of the background area from the extension area, and combines the cut-out background area and the received attention area and parts,
A distortion correction unit that corrects distortion caused by photographing using the wide-angle lens,
An image receiving apparatus that updates the extension area of the extension area storage unit each time the extension area is received.   The image receiving apparatus according to claim 7, wherein when the background area cannot be cut out from the extension area, an update of the extension area is requested. An attention area that is an area in which a motion of a predetermined level or more is detected from the captured image, and the attention area are cut out, a position and a size are specified by the user, and set wider than a background area including the attention area. And an extended area including the background area is cut out,
The region of interest is transmitted to the image receiving device at a first predetermined update interval, and the first predetermined update is performed using the extended region as data of a cutout source of the background region to be combined with the region of interest. An image transmission method for transmitting at a second predetermined update interval larger than the interval . An attention area that is an area in which a movement of a captured image at a predetermined level or more is detected is received at a first predetermined update interval, and a position and a size are designated by the user, and further than a background area including the attention area Receiving an extended region that is widely set and includes the background region at a second predetermined update interval that is greater than the first predetermined update interval;
The background area is cut out from the received extension area so that the center of the attention area matches the center of the background area, and the cut-out background area is combined with the attention area .
An image receiving method for correcting distortion of a synthesized image.