JP2020022065A - Distribution device, camera device, distribution system, distribution method, and distribution program - Google Patents

Abstract

To provide a distribution device that transmits a picked-up image picked up by an imaging apparatus to a receiving device to display the image, and reduce inconvenience that a received image has a low resolution in the receiving device during a communication delay.SOLUTION: An acquisition unit acquires a picked-up image picked up by an imaging apparatus, and a creation unit creates a composite image with an image of a partial area, of the entire area of the acquired picked-up image, having high image quality, the partial area being the partial area displayed on a receiving device, an image of an extended area obtained by extending the partial area in a transverse direction or a vertical direction having high image quality or middle image quality, and an image in an area other than the partial area and extended area having low image quality. A transmission unit transmits the created composite image to the receiving device. Thus, when display of an image of the extended area is designated on the receiving device, a high-quality image of the extended area can be displayed, which reduces inconvenience that an image with a low resolution is displayed even when a communication delay occurs.SELECTED DRAWING: Figure 11

Description

  The present invention relates to a distribution device, a camera device, a distribution system, a distribution method, and a distribution program.

  2. Description of the Related Art In recent years, an omnidirectional camera that can capture an omnidirectional sphere within a range of a solid angle of 360 ° around an imaging position by one imaging has been put into practical use. In addition, in the omnidirectional image which is an image having a solid angle of 360 ° captured by the omnidirectional camera, the observer who observes the omnidirectional image can select an arbitrary viewpoint. . Also, by transmitting this celestial sphere image to the receiving device via a network, a distribution system capable of confirming the current situation of a physically distant location from an arbitrary viewpoint in the receiving device, Has been put to practical use.

  Patent Document 1 (Japanese Patent No. 3963261) describes a system in which a wide-angle camera including a wide-angle lens including a fish-eye lens and a pan-tilt camera having a pan and tilt function are combined. According to Patent Literature 1, a moving object is detected from an image captured by a wide-angle camera, and the imaging direction of the pan-tilt camera is directed to the detected moving object, thereby enabling tracking of the moving object.

  However, conventionally, if a communication delay occurs at the time of transmission, the received image received by the receiving device has a low resolution, and there is a possibility that the operation of the user may be stressed.

  The present invention has been made in view of the above-described problems, and has as its object to provide a distribution device, a camera device, a distribution system, a distribution method, and a distribution program that reduce the inconvenience of receiving a low-resolution image when communication is delayed. .

  In order to solve the above-described problem and achieve the object, the present invention provides an acquisition unit that acquires a captured image captured by an imaging device, and a part of the entire captured image that is displayed by a reception device. The image quality of the image of the partial area, which is a partial area, is enhanced, and the image of the extended area obtained by extending the partial area in the left-right direction or the vertical direction is enhanced or medium-quality, and the image of the area other than the partial area and the extended area is obtained. And a transmitting unit that transmits the generated composite image to the receiving device.

  ADVANTAGE OF THE INVENTION According to this invention, there exists an effect that the inconvenience which becomes a low resolution received image at the time of communication delay can be reduced.

FIG. 1 is a diagram illustrating a system configuration of a distribution system according to an embodiment. FIG. 2 is a diagram showing the configuration of the appearance of the spherical camera. FIG. 3 is a block diagram of the spherical camera. FIG. 4 is a diagram illustrating an example of an image captured by a spherical camera. FIG. 5 is a diagram for explaining a method of generating an omnidirectional image by the video processing unit of the omnidirectional camera. FIG. 6 is a block diagram of the distribution device. FIG. 7 is a functional block diagram of the distribution device. FIG. 8 is a diagram illustrating a hardware configuration of the receiving device. FIG. 9 is a functional block diagram of the receiving device. FIG. 10 is a flowchart illustrating the flow of the distribution operation of the distribution device. FIG. 11 is a diagram for explaining an area where the image quality is improved in the omnidirectional image. FIG. 12 is a flowchart showing the flow of the processing of the receiving device. FIG. 13 is a diagram for explaining the designated area. FIG. 14 is another diagram for explaining the designated area. FIG. 15 is a diagram for describing a first modification of the embodiment. FIG. 16 is a diagram for describing a second modification of the embodiment.

  Hereinafter, a distribution system according to an embodiment to which a distribution device, a camera device, a distribution system, a distribution method, and a distribution program are applied will be described.

(System configuration)
FIG. 1 is a diagram illustrating a system configuration of a distribution system according to an embodiment. As shown in FIG. 1, the distribution system 1 includes a distribution device 10 and a receiving device 40. The distribution device 10 can be connected to the receiving device 40 via the network 2. For example, the receiving device 40 is capable of wireless communication, communicates with an AP (access point) 3 connected to the network 2 using wireless communication, and is connected to the network 2.

  The omnidirectional camera 20 which is an example of a photographing device is connected to the distribution device 10. Note that, instead of the spherical camera 20, a camera device having a normal angle of view may be provided.

  The spherical camera 20 obtains a captured image having an angle of view of 4π [sr] (steradian) by combining, for example, two fish-eye lenses each having an angle of view of 180 ° or more and an optical system including a mirror or a prism. It is possible. Hereinafter, an image captured by the spherical camera 20 and having a view angle of 4π [sr] (steradian) is referred to as a spherical image, and the center of the solid angle of 4π [sr] is referred to as a view angle center. . Further, the omnidirectional camera 20 can output an omnidirectional image captured as a moving image. The spherical image output from the spherical camera 20 is transmitted to the distribution device 10. The spherical camera 20 is installed while being held by, for example, a tripod 21.

  As the receiving device 40, for example, a smartphone or a tablet-type computer device can be used. The receiving device 40 has a touch panel 42 in which a display device and an input device that outputs a control signal according to an operation position are integrally formed. For example, a screen based on a received signal received from the AP 3 is displayed on the touch panel 42. Let it. Touch panel 42 outputs a control signal according to an operation on a displayed screen.

  In the distribution system 1 according to such an embodiment, the distribution device 10 transmits the omnidirectional image output from the omnidirectional camera 20 to the receiving device 40 via the network 2. The receiving device 40 displays, on the touch panel 42, the image of the designated region (a part of the whole image) among the omnidirectional images received from the distribution device 10.

  Here, a case is considered where, in the receiving device 40, an arbitrary position on the touch panel 42 is touched with the user's finger 41 or the like while the image of the designated area is displayed on the touch panel 42. It is assumed that after the touch operation of the touch panel 42 with the finger 41 or the like, the position of the finger 41 is moved while the contact state of the finger 41 with the touch panel 42 is maintained. Upon detecting the movement of the finger 41 while keeping the state of contact with the touch panel 42, the receiving device 40 moves the position of the designated area in the omnidirectional image according to the moving direction and the moving distance. Here, it is assumed that the moving direction of the finger 41 on the touch panel 42 matches the moving direction of the designated area in the omnidirectional image. Thus, the user can specify and view a desired area of the omnidirectional image on the screen displayed on the touch panel 42.

  The act of bringing the finger 41 into contact with the touch panel 42 is referred to as “tap”. The act of moving the finger 41 on the touch panel 42 while keeping the finger 41 in contact with the touch panel 42 is referred to as “swipe”.

  Next, it is assumed that the user stops the movement of the finger 41 performing the swipe for a predetermined time (for example, one second) while keeping the finger 41 in contact with the touch panel 42. In this case, the receiving device 40 acquires the position (coordinates) of the omnidirectional image corresponding to the position of the finger 41 in the designated area displayed on the touch panel 42, and transmits position information indicating the acquired position to the distribution device 10. I do.

  When receiving the position information transmitted from the receiving device 40, the distribution device 10 transmits an omnidirectional image centered on the position indicated by the received position information to the receiving device 40. As will be described later, the distribution device 10 transmits a designated area located at the center of the omnidirectional image transmitted to the reception device 40 as a high-quality image. In addition, the distribution device 10 transmits the left and right regions of the designated region as medium-quality images. In addition, the distribution device 10 transmits low-quality images in the designated region and the upper and lower regions of the left and right regions of the designated region (= the designated region and the left and right regions of the designated region).

  The swipe is more likely to be operated left and right than the up and down operation. For this reason, by transmitting a medium-quality image as an image of the left and right regions of the designated region from the distribution device 10 to the receiving device 40 in advance, the medium-quality image can be displayed according to the left and right swipe. be able to. In this example, a description will be given assuming that a medium-quality image is transmitted as an image of the left and right areas of the designated area. However, a high-quality image may be transmitted as an image of the left and right areas of the designated area.

  The high image quality is, for example, image quality such as HD (High Definition) image quality (1280 pixels × * 720 pixels), full HD image quality (1920 pixels × 1080 pixels), or 4K image quality (3840 pixels × 1920 pixels). The medium image quality is a standard image quality such as “960 pixels × 720 pixels”. The low image quality is an image quality lower than a standard image quality such as “480 pixels × 360 pixels”. This definition of image quality is an example. For this reason, the high image quality, the medium image quality, and the low image quality may be relatively determined so that the total number of pixels or the like is “high image quality> medium image quality> low image quality” (may be determined by comparison).

(External configuration of spherical camera)
FIG. 2 is a diagram illustrating a configuration of an external appearance of the spherical camera 20. 2A shows a top view of the omnidirectional camera 20 and FIG. 2B shows a front view of the omnidirectional camera 20. In FIG. 2A, the lenses 210a and 210b collect the light on the front side and the rear side of the spherical camera 20 at an angle of view of 2π [sr] or more, respectively. In FIG. 2B, a shutter button 211, a power lamp 212, and a mode display unit 213 are provided on the front side of the spherical camera 20 in addition to the lens 210a. The mode display unit 213 indicates the currently selected shooting mode (still image shooting mode or moving image shooting mode) of the spherical camera 20.

  FIG. 3 is a block diagram of the spherical camera 20. 3, the omnidirectional camera 20 includes a photographing unit 220, a video processing unit 223, and a photographing control unit 224 as a structure of a photographing system. The control system includes a CPU (Central Processing Unit) 230, a ROM (Read Only Memory) 231, a RAM (Random Access Memory) 232, an operation I / F (interface) 233, a switch 234, a data An I / F 235, a wireless I / F 236, and a bus 237 connecting these components are included. Further, to the bus 237, a video processing unit 223 and a photographing control unit 224 are also connected.

  In the configuration of the control system, the CPU 230 operates using the RAM 232 as a work memory according to a program stored in the ROM 231 in advance, and controls the overall operation of the omnidirectional camera 20. The switch 234 includes one or more operators such as the shutter button 211 described above, and outputs a control signal according to the operation. The control signal output from the switch 234 is sent to the bus 237 via the operation I / F 233 and passed to the CPU 230.

  The data I / F 235 is an interface corresponding to data communication such as USB (Universal Serial Bus), and performs data communication with an external device under the control of the CPU 230. A connector for performing connection by the data I / F 235 is provided, for example, on the bottom surface side of the spherical camera 20. The wireless I / F 236 performs wireless communication under the control of the CPU 230.

  In the configuration of the photographing system, the photographing unit 220 includes a front photographing element 221a for photographing the front side of the spherical camera 20 and a rear photographing element 221b for photographing the rear side. The front photographing element 221a and the rear photographing element 221b are each configured using, for example, a CCD (Charge Coupled Device), and convert light incident from the lenses 210a and 210b into electric signals and output them as photographing signals.

  The imaging control unit 224 controls the imaging process such as the imaging timing of the front imaging element 221a and the rear imaging element 221b under the control of the CPU 230. The video processing unit 223 performs predetermined processing such as noise removal and level correction on each signal output from the front imaging element 221a and the rear imaging element 221b, and then performs A / D conversion to obtain a captured image using digital data. Convert to Further, the video processing unit 223 combines the captured images obtained by the front imaging element 221a and the rear imaging element 221b to generate a two-dimensional image (referred to as a spherical image) in an XY coordinate system. The spherical image generated by the video processing unit 223 can be output from the data I / F 235 to the outside, for example.

(Example of video to be shot)
FIG. 4 is a diagram illustrating an example of an image captured by the spherical camera 20. 4A shows an example of an image captured by the front imaging element 221a of the spherical camera 20, and FIG. 4B shows an example of an image captured by the rear imaging element 221b. 4A and 4B, the image is upside down. This simplifies the optical system, reflects light rays incident from the lenses 210a and 210b disposed on the front and rear of the housing of the spherical camera 20 up and down on the front and back of one mirror, and the optical axis of the reflected light. This is because the front photographing element 221a and the rear photographing element 221b are arranged above. The images in FIGS. 4A and 4B are combined into one image by the image processing unit 223 to form a spherical image by the equirectangular projection illustrated in FIG. 4C.

(How to generate a spherical image)
FIG. 5 is a diagram for explaining how the image processing unit 223 of the spherical camera 20 generates a spherical image. In FIG. 5, the upper part shows the correspondence between the horizontal angle θ and the vertical angle φ with respect to the pixel position of each image obtained from the front imaging element 221a and the rear imaging element 221b. On the upper left and right sides, the sign of the angle φ is reversed, corresponding to the upside down of one image.

  The image processing unit 223 converts each pixel included in each image shown in the upper part into each image shown in the middle part based on a look-up table that converts the angle θ and the angle φ into orthogonal coordinates. Next, the video processing unit 223 combines the two videos shown in the middle, and generates one video whose angle θ is 0 ° to 360 ° shown in the lower. Then, this is decomposed into triangular polygons, and is made three-dimensional data in association with the angles θ and φ. This is output from the video processing unit 223 as a spherical image.

(Hardware configuration of distribution device)
FIG. 6 is a block diagram of the distribution device 10. 6, the distribution device 10 includes a CPU 1000, a ROM 1001, a RAM 1002, a video output I / F 1003, and an operation unit I / F 1009, and these units are connected to a bus 1011. The distribution device 10 further includes a communication I / F 1004, a data I / F 1005, video input I / Fs 1006 and 1007, and an audio I / F 1008, and these units are connected to the bus 1011. As described above, the distribution device 10 can be considered to include a computer including the CPU 1000, the ROM 1001, and the RAM 1002 connected by the bus 1011.

  The CPU 1000 operates using the RAM 1002 as a work memory in accordance with a program stored in the ROM 1001 in advance, and controls the overall operation of the distribution device 10. The video output I / F 1003 is an interface for connecting a display to the distribution device 10. The video output I / F 1003 converts, for example, an image (video) supplied from the CPU 1000 via the bus 1011 into a format compatible with the display, and outputs the converted image.

  The operation unit I / F 1009 is connected to the operation input unit 1010. The operation input unit 1010 includes various operators for manually operating the distribution device 10, and outputs a control signal according to the operation. The control signal output from the operation input unit 1010 is sent to the bus 1011 via the operation unit I / F 1009 and passed to the CPU 1000.

  The video output I / F 1003 is an interface for connecting a display to the distribution device 10. The video output I / F 1003 converts, for example, an image (video) supplied from the CPU 1000 via the bus 1011 into a format compatible with the display, and outputs the converted image.

  The communication I / F 1004 controls communication with the network 2 under the control of the CPU 1000. The data I / F 1005 is an interface corresponding to data communication such as USB, and can perform data communication with an external device under the control of the CPU 1000. For example, the data I / F 235 of the spherical camera 20 and the data I / F 1005 are connected, and the operation of the spherical camera 20 can be controlled from the distribution device 10 via the data I / F 1005.

  The video input I / Fs 1006 and 1007 each receive an input of an external image (video) and output the input image to the bus 1011. For example, the spherical image output from the spherical camera 20 described above is input to the video input I / F 1006 or 1007 and passed to the CPU 1000. Note that the communication I / F 1004 and the data I / F 1005 can be used as the video input I / Fs 1006 and 1007.

  The audio I / F 1008 converts a digital audio signal supplied via the bus 1011 into an analog audio signal by D / A conversion, further performs processing such as amplification, and outputs the signal, and supplies the signal to, for example, a speaker. . The audio I / F 1008 converts an analog audio signal input from a microphone into a digital audio signal by A / D conversion, and outputs the digital audio signal to the bus 1011.

(Function of distribution device)
FIG. 7 is a functional block diagram of the distribution device 10. 7, the distribution device 10 includes a control unit 100, a communication unit 101, a generation unit 102, and an acquisition unit 103. The control unit 100, the communication unit 101, the generation unit 102, and the acquisition unit 103 are configured by a distribution program operating on the CPU 1000. However, the invention is not limited thereto, and some or all of the control unit 100, the communication unit 101, the generation unit 102, and the acquisition unit 103 may be configured by hardware circuits that operate in cooperation with each other.

  The control unit 100 controls the entire operation of the distribution device 10 including the communication unit 101, the generation unit 102, and the acquisition unit 103. The communication unit 101 controls the communication I / F 1004 to perform communication via the network 2. The generation unit 102 forms an omnidirectional image formed of high, medium, and low image quality images as described above in response to a user swipe.

  The acquisition unit 103 acquires an omnidirectional image captured by the omnidirectional camera 20. The information acquired by the acquiring unit 103 is transmitted to the receiving device 40 via the communication unit 101 under the control of the control unit 100.

  The distribution program of the distribution apparatus 10 for realizing each of the functions shown in FIG. 7 is a file in an installable format or an executable format in a CD (Compact Disk), a flexible disk (FD), a DVD (Digital It is provided by being recorded on a computer-readable recording medium such as a Versatile Disk. However, the present invention is not limited to this, and the distribution program may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. Further, the distribution program may be provided or distributed via a network such as the Internet.

  This distribution program has a module configuration including a control unit 100, a communication unit 101, a generation unit 102, and an acquisition unit 103. As actual hardware, the CPU 1000 reads out the distribution program from a storage medium such as the ROM 1001 and executes the distribution program, whereby the above-described units are loaded onto a main storage device such as the RAM 1002, and the control unit 100, the communication unit 101, and the The unit 102 and the acquisition unit 103 are generated on the main storage device.

  In the example of the embodiment, description will be made assuming that the distribution device 10 and the spherical camera 20 are provided as separate devices, but the distribution device 10 is provided in the spherical camera 20. Thus, the spherical camera 20 and the distribution device 10 may be provided as one device. In this case, the configuration shown in FIG. 1 is a configuration in which the omnidirectional camera 20 communicates with the receiving device 40 via the network 2.

(Hardware configuration of receiving device)
FIG. 8 is a diagram illustrating a hardware configuration of the receiving device 40. 8, the receiving device 40 includes a CPU 4000, a ROM 4001, a RAM 4002, an audio I / F 4003, a storage 4004, a communication I / F 4005, a video output I / F 4006, and an input I / F 4007. , These parts are connected by a bus 4010. Thus, the receiving device 40 can be considered to be provided with a computer including the CPU 4000, the ROM 4001, and the RAM 4002 connected by the bus 4010.

  The storage 4004 is an example of a storage unit, and is a non-volatile storage medium such as a non-volatile memory or a hard disk drive, and can store a program for operating the CPU 4000 and various data. The CPU 4000 operates using the RAM 4002 as a work memory according to a program stored in the ROM 4001 and the storage 4004 in advance, and controls the overall operation of the receiving device 40.

  The video output I / F 4006 is connected to the display 4008a, and converts an image (video) or a display control signal supplied from the CPU 4000 via the bus 4010 into a format that can be supported by the display 4008a and outputs it. For example, the video output I / F 4006 includes a frame memory that stores an image corresponding to each pixel of the display 4008a, and transfers an image stored in the frame memory in pixel units to the display 4008a. Note that the display 4008a includes a display device such as an LCD (Liquid Crystal Display) and a drive circuit for driving the display device.

  The input I / F 4007 is connected to an input device 4008b that outputs a control signal corresponding to the position, converts the control signal output from the input device 4008b into a format that can be supported by the CPU 4000, and sends the control signal to the CPU 4000 via the bus 4010. hand over. The display 4008a and the input device 4008b are integrally formed to constitute the touch panel 42 shown in FIG.

  The communication I / F 4005 has a function of transmission and reception by wireless communication, performs communication with the AP 3 under the control of the CPU 4000, and performs communication via the network 2. The audio I / F 4003 converts the digital audio signal supplied from the bus 4010 into an analog audio signal by D / A conversion, further performs processing such as amplification, and outputs the analog audio signal to the speaker SP4020.

(Receiver function)
FIG. 9 is a functional block diagram of the receiving device 40. In FIG. 9, the receiving device 40 includes a control unit 400, a communication unit 401, a UI unit 402, a display control unit 403, and a position designation unit 406. The control unit 400, the communication unit 401, the UI unit 402, the display control unit 403, and the position designation unit 406 are configured by a reception program that operates on the CPU 4000. However, the present invention is not limited thereto, and some or all of the control unit 400, the communication unit 401, the UI unit 402, the display control unit 403, and the position specification unit 406 may be configured by hardware circuits operating in cooperation with each other.

  The control unit 400 controls the overall operation of the receiving device 40 including the communication unit 401, the UI unit 402, the display control unit 403, and the position designation unit 406. The communication unit 401 controls the communication I / F 4005 to perform communication via the network 2.

  The UI unit 402 receives an input operation on the input device 4008b on the touch panel 42. For example, the UI unit 402 detects contact of the finger 41 with the input device 4008b, and detects a position where the finger 41 is contacted. In addition, the UI unit 402 determines a tap operation or a swipe operation on the input device 4008b based on the detection result of the contact of the finger 41 with the input device 4008b.

  The display control unit 403 includes a drawing unit 404 and a combining unit 405, and controls display of a still image and a moving image on the display 4008a. The drawing unit 404 draws a still image or a moving image in the frame memory. By transferring the image drawn in the frame memory to the display 4008a, a still image or a moving image is displayed on the display 4008a.

  The synthesis unit 405 overwrites, for example, the image drawn in the frame memory by the drawing unit 404 with the other image when the synthesis of another image is specified for the image displayed on the display 4008a. Thus, the other image is synthesized.

  The position designating unit 406 is a position on the touch panel 42 corresponding to the position input by the input device 4008b with respect to the designated area, which is displayed on the display 4008a and is a part of the whole celestial sphere image. Is specified. The position information indicating the position specified by the position specifying unit 406 is transmitted to the distribution device 10 by the communication unit 401.

  A receiving program for realizing each function of the receiving device 40 is provided by being recorded in a computer-readable recording medium such as a CD, a flexible disk, and a DVD in a file in an installable format or an executable format. Is done. However, the present invention is not limited thereto, and the receiving program may be provided by being stored on a computer connected to a network such as the Internet, and being downloaded via the network. Further, the receiving program may be provided or distributed via a network such as the Internet.

  This receiving program has a module configuration including a control unit 400, a communication unit 401, a UI unit 402, a display control unit 403, and a position designation unit 406. As actual hardware, the CPU 4000 reads out the reception program from a storage medium such as the ROM 4001 and the storage 4004 and executes the reception program, whereby the above-described units are loaded on the main storage device such as the RAM 4002, and the control unit 400 and the communication unit 401, a UI unit 402, a display control unit 403, and a position designation unit 406 are generated on a main storage device.

(Distribution operation)
Next, an image distribution operation by the distribution system 1 according to the first embodiment will be described. FIG. 10 is a flowchart illustrating the flow of the distribution operation of the distribution device 10. In the flowchart of FIG. 10, first, the acquiring unit 101 of the distribution device 10 acquires an omnidirectional image from the omnidirectional camera 20 via the communication unit 103 (step S1). Next, the generation unit 102 of the distribution device, for example, as shown in FIG. 11, of the entire region of the acquired omnidirectional image, is the region displayed first on the receiving device 40, for example, the central region HR ( An example of the partial area) is a high-quality image. Further, left and right regions (an example of an extended region) adjacent to each other in the horizontal direction at the same height as the central region HR are taken as medium-quality images. As a result, the band-like region (region B in FIG. 11) in the horizontal direction at the center of the omnidirectional image shown in FIG. 11 becomes a high-quality image or a medium-quality image. Further, the generation unit 102 outputs a high-quality image in the center area, and The upper and lower regions (regions A and C in FIG. 11) of the left and right regions adjacent to the central region which is a medium quality image are defined as low quality images.

  When an omnidirectional image (an example of a composite image) in which images of image quality of each resolution are combined is generated as described above, the communication unit 101 transmits the omnidirectional image to the receiving device 40 via the communication I / F 1004. The image is transmitted (Step S2).

  In this example, the above-mentioned left and right areas are medium-quality images, but these left and right areas may be high-quality images. In this case, the band-shaped region (region B) in the horizontal direction at the center of the omnidirectional image shown in FIG. 11 is a high-quality image.

  Next, the control unit 100 of the distribution device 10 determines whether or not there is a distribution end instruction in step S3 (step S3). When the distribution end is instructed (step S3: Yes), the processing of the flowchart in FIG. On the other hand, when the distribution end instruction is not detected (step S3: No), the process returns to step S1 of the flowchart in FIG. 10 and the acquisition unit 103 continues to transmit the whole sky image captured by the spherical camera 20. The operation of acquiring a sphere image, generating a composite image in which the above-mentioned band-like region in the center in the horizontal direction (the region B in FIG. 11) has high image quality and medium image quality, and transmitting it to the receiving device 40 is repeatedly executed. As a result, the moving image (still image) of the composite image described above is distributed to the receiving device 40 (live distribution) until an instruction to end the distribution is detected.

(Processing of receiving device)
FIG. 12 is a flowchart showing the flow of the processing of the receiving device 40. In addition, the omnidirectional image as a moving image or a still image captured by the omnidirectional camera 20 is received by the receiving device 40 via the distribution device 10 and is displayed on the touch panel 42, and thus, in FIG. The processing of the flowchart is started.

  In step S120, in the receiving device 40, the display control unit 403 receives the omnidirectional image transmitted from the distribution device 10 and stores the omnidirectional image in the storage 4004, and corresponds to the default display area in the omnidirectional image. Then, an image of the central region of the spherical image is acquired and displayed on the touch panel 42. As described with reference to FIG. 11, the image in the central region HR of the omnidirectional image is a high-quality image. Therefore, a high-quality image is displayed on the touch panel 42.

  In step S121, the UI unit 402 determines whether there is a touch operation on the input device 4008b with the finger 41 or the like. When the UI unit 402 does not detect a contact operation (step S121: No), the process returns to step S121. On the other hand, if the UI unit 402 detects a contact operation (step S121: Yes), the UI unit 402 recommends the process to step S122.

  In step S122, the display control unit 403 acquires, from the UI unit 402, the position on the touch panel 42 where the contact is detected (the contact position), and, of the spherical image stored in the storage 4004, the acquired contact The partial image of the designated area corresponding to the position is read. Then, the drawing unit 404 displays a partial image of the designated area corresponding to the contact position on the touch panel 42, whereby the designated area designated by the swipe operation is selected from the spherical images stored in the storage 4004. Can be displayed on the touch panel 42.

  Here, the swipe operation of the user is often a left-right movement operation rather than a vertical movement operation of a finger or the like. Correspondingly, in the case of the distribution system according to the embodiment, as described with reference to FIG. It is an image. Therefore, when the designated area is designated by the left and right swipe operations, a high-quality or medium-quality image can be displayed with high probability. For this reason, even if a communication delay occurs between the distribution device 10 and the receiving device 40, it is possible to reduce the inconvenience that the image displayed on the receiving device 40 side is a low-resolution image.

  Next, in step S123, the display control unit 40 determines whether a new omnidirectional image has been received from the distribution device 10. When a new omnidirectional image is received (Step S123: Yes), the control section 400 updates the omnidirectional image stored in the storage 4004 with the received omnidirectional image. After this update, the process returns to step S120.

(Specified operation of specified area)
Next, referring to FIGS. 13 and 14, an operation of designating the designated area of the omnidirectional image will be described. FIG. 13 is a diagram illustrating an example of the designated area. FIG. 14A shows the relationship among the X axis, the Y axis, and the Z axis. The three-dimensional sphere CS is a spherical surface on which the above-mentioned omnidirectional image is virtually attached. FIG. 14B shows an example of an arrangement of virtual cameras CAM for specifying a designated area. The virtual camera CAM is arranged inside the three-dimensional sphere CS, and a predetermined area T captured by the virtual camera CAM is a designated area. FIG. 14C shows the field of view of the virtual camera CAM, and the angle between both ends of the diagonal line of the predetermined area T is the viewing angle α. The distance f is a distance between the virtual camera CAM and the center point CP of the predetermined area T.

  FIG. 14 is a diagram illustrating the relationship between the position of the virtual camera CAM and the origin of the three-dimensional sphere CS. In FIG. 14, the radius of the solid sphere CS is normalized to “1”. In this manner, the viewing angle α is obtained by viewing the solid sphere CS from a position receded by the distance d from the origin. When the position of the distance d is inside the solid sphere CS, it is assumed that the position where the solid sphere CS intersects at the angle of view ω from the origin and the position where the solid sphere CS intersects at the viewing angle α. On the other hand, when the position of the distance d is outside the three-dimensional sphere CS, the viewing angle α circumscribes the three-dimensional sphere CS. When the distance d is 0, the angle of view ω from the origin and the viewing angle α match. It is considered that the omnidirectional camera 20 is arranged at a distance d = 0 with respect to the three-dimensional sphere CS, and the position of the omnidirectional camera 20 at this time is set as the center of the angle of view.

  Note that by appropriately controlling the angle of view ω, the viewing angle α, and the distance d, the image in the designated area can be enlarged and reduced. The enlargement / reduction magnification is determined by the angle of view ω, the viewing angle α, and the distance d. As the image is reduced, the outline of the displayed image becomes circular.

(Effects of Embodiment)
As is clear from the above description, the swipe operation is more likely to be operated left and right than the up and down operation. For this reason, in the distribution system of the embodiment, among the celestial sphere images, the image quality of the central horizontal band-shaped area is set to high image quality and medium image quality (all central horizontal band-shaped areas are High-quality images may be used). Further, the images in the other areas are low-quality images. This makes it possible to significantly reduce the data amount between the distribution device 10 and the receiving device 40, compared to transmitting and receiving a full celestial sphere image in which images of all regions are of high quality. In addition, when the left and right swipe operations are performed, a medium-quality or high-quality image can be displayed regardless of the communication state between the distribution device 10 and the reception device 40. Therefore, even if a communication delay occurs between the distribution device 10 and the receiving device 40, it is possible to reduce the inconvenience that the image displayed on the receiving device 40 becomes a low-resolution image.

(First Modification)
In the description of the above-described embodiment, an example in which the center horizontal band-shaped area of the omnidirectional image is made to have high image quality, the center vertical band-shaped area of the omnidirectional image is heightened. The image quality may be improved. FIG. 15A shows an example in which a spherical camera 20 is provided on a shovel car 30 having a power shovel for excavation. In the case of the shovel car 30, it is important to check the up and down movement of the power shovel during work.

  For this reason, in the case of this first modification, as shown in FIG. 15B, the omnidirectional camera 20 generates an omnidirectional image centered on the power shovel, and the distribution device 10 controls the omnidirectional image. In the spherical image, the image quality is improved in the central band-like area in the vertical direction in the center (or the medium image quality is improved), and the other areas are reduced in image quality and transmitted to the receiving device 40. Accordingly, when the user performs a vertical swipe operation, a high-quality image at a position corresponding to the swipe operation can be displayed, and the same effect as in the above-described embodiment can be obtained. it can.

(Second Modification)
Also, the CPU 1000 of the distribution device 10 performs the image recognition process to specify the position of the power shovel 31 illustrated in FIG. 16A and to receive a celestial sphere image in which the area including the specified position has high image quality. 40. Thereby, as shown in FIG. 16B, for example, an image of the entire periphery of the power shovel 31 which is important for the operation of the excavator 30 is transmitted to the receiver 40 as a high-quality image of the periphery of the power shovel 31, and the important part is transmitted. In addition to displaying images with high image quality, effects similar to those of the above-described embodiment can be obtained.

  Although the above-described embodiment is a preferred example of the present invention, the present invention is not limited to this, and can be implemented by various modifications without departing from the gist of the present invention. It is.

  For example, in the description of the above-described embodiment, the image quality in the center left-right direction or the center vertical direction of the omnidirectional image is set to be high. The image quality in both directions may be improved. Further, the image quality in other directions, such as the diagonal direction, in addition to the horizontal direction or the vertical direction, may be improved. In any case, the same effect as described above can be obtained.

Reference Signs List 1 delivery system 2 network 10 delivery device 20 omnidirectional camera 40 receiving device 42 touch panel 100 control unit 101 communication unit 102 generation unit 103 acquisition unit 1000 CPU
400 control unit 402 UI unit 403 display control unit

Japanese Patent No. 3963261

Claims (7)

An acquisition unit configured to acquire a captured image captured by the imaging device;
An image of an extended area obtained by extending the partial area, which is a partial area displayed by the receiving device, in the entire area of the acquired captured image and extending the partial area in the horizontal direction or the vertical direction. A generating unit that increases the image quality of medium or high image quality, and generates a composite image in which the image of the area other than the partial area and the extended area is reduced in quality.
A transmitting unit that transmits the generated composite image to the receiving device. The distribution device according to claim 1, wherein the acquisition unit acquires, as the captured image, a celestial sphere image captured by the imaging device at an angle of view of 360 degrees. An imaging unit that captures a captured image;
Of the entire area of the captured image, the image of the partial area that is a partial area displayed on the receiving device is improved in image quality, and the extended area obtained by extending the partial area in the horizontal direction or the vertical direction A generation unit that increases the image quality or the medium image quality, and generates a composite image in which the image of the area other than the partial area and the extension area is reduced in quality,
A transmitting unit that transmits the generated composite image to the receiving device. The camera device according to claim 3, wherein the imaging unit captures a 360 ° spherical image of the celestial sphere. An imaging device that captures a captured image;
An acquisition unit configured to acquire a captured image captured by the imaging device;
An image of an extended area obtained by extending the partial area, which is a partial area displayed by the receiving device, in the entire area of the acquired captured image and extending the partial area in the horizontal direction or the vertical direction. A generating unit that increases the image quality of medium or high image quality, and generates a composite image in which the image of the area other than the partial area and the extended area is reduced in quality.
A transmitting unit that transmits the generated composite image to the receiving device,
A receiving unit that receives the composite image,
A storage unit for storing the received composite image,
A display control unit that reads, from the storage unit, an image of the partial area corresponding to the position specified by the operation unit from among the composite images stored in the storage unit and displays the image on a display unit. A distribution system comprising: a device; An acquiring step of acquiring an image captured by the imaging device,
The generation unit, of the entire region of the acquired captured image, while improving the quality of the image of the partial region that is a partial region displayed on the receiving device, and extending the partial region in the horizontal direction or the vertical direction A generation step of increasing the quality of the image of the extended area or increasing the quality of the medium, and generating a composite image in which the quality of the image of the area other than the partial area and the extended area is reduced,
A transmitting step of transmitting a generated composite image to the receiving device by a transmitting unit. Computer
An acquisition unit configured to acquire a captured image captured by the imaging device;
An image of an extended area obtained by extending the partial area, which is a partial area displayed by the receiving device, in the entire area of the acquired captured image and extending the partial area in the horizontal direction or the vertical direction. A generating unit that increases the image quality of medium or high image quality, and generates a composite image in which the image of the area other than the partial area and the extended area is reduced in quality.
A distribution program that causes the generated composite image to function as a transmission unit that transmits the composite image to the reception device.