JPH11205772A - Omnidirectionally picked-up image sending system and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To simultaneously view videos in different directions through the use of a single camera as if pural users freely operated respectively assigned cameras. SOLUTION: An omnidirectional image pickup part 117 picks up the images of the surrounding scenes of 360 deg. to obtain an omnidirectional image, an information storing part 114 stores the omnidirectional image, an image compressing part 118 compresses the omnidirectional image and a transmission/reception part 116 transmits the omnidirectional image to a transmission/reception part 154. The part 154 receives the omnidirectional image, an information storing part 152 stores the omnidirectional image, an image extension part 151 extends the omnidirectional image, an image conversion part 153 converts a fan-shaped image on the omnidirectional image into a rectangular image and a display part 156 displays a rectangular image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup device using an omnidirectional image pickup device having a 360-degree observation field of view.
The present invention relates to an omnidirectional captured image transmission system and an omnidirectional captured image transmission method provided to a remote observer.

[0002]

2. Description of the Related Art In recent years, with the remarkable development of computer technology, multimedia technology for converting and processing information such as video and audio into digital information has been remarkably developed. In addition, communication technology for transmitting and receiving such information via a network such as the Internet has come to be used.

[0003] Utilizing these technologies, in sightseeing spots, etc.
2. Description of the Related Art An image providing system that converts an image captured by a camera installed outdoors with a good view into a digital signal and provides the digitalized image to a user via the Internet has been put to practical use. With this video providing system,
For tourists and those who are planning to revitalize the area, by transmitting images, it will be used to advertise sightseeing spots and introduce areas,
In addition, there is a merit that users can enjoy the scenery while staying without going to those places.

In such an image providing system, a camera is installed at a high place such as the roof of a building. The camera receives designation of the camera's azimuth and angle of view from a server computer that controls the operation of the camera, captures an image of the surrounding landscape at the specified azimuth and angle of view, converts the captured image into a digital signal, The signalized video is transmitted to the server computer. The server computer is connected to the Internet, receives a video transmission request, a camera direction and an angle of view from a client computer connected to the Internet operated by a user, and receives the transmission request. When an azimuth and an angle of view are specified to the camera, and a digitalized video is received from the camera, the digitalized video is transmitted to the client computer that transmitted the transmission request. The client computer operated by the user is connected to the Internet, receives a video transmission request from the user, specifies the direction and angle of view of the camera, requests the server computer to transmit a video via the Internet, The azimuth and angle of view are transmitted. Further, when receiving the digitalized video from the server computer, the client computer reproduces and displays the received digitalized video.

[0005]

However, in such a conventional image providing system, although the user can operate the azimuth and the angle of view of the camera, only one user can operate the camera. However, there is a problem that the user cannot operate the camera at the same time and simultaneously view images in different directions.

[0006] The present invention solves such a problem, and as if a plurality of users are freely operating the cameras assigned to each of them, a single camera is used to simultaneously operate separate cameras. It is an object of the present invention to provide an omnidirectional captured image transmission system and an omnidirectional captured image transmission method capable of viewing images in different directions.

[0007]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides an omnidirectional image received and transmitted from a transmitting apparatus for transmitting an omnidirectional image obtained by capturing the surrounding scenery. A receiving device for displaying a partial image of the image,
Transmission request transmitting means for transmitting a transmission request for an omnidirectional image to the transmitting device according to a user operation; omnidirectional image receiving means for receiving an omnidirectional image from the transmitting device to which the transmission request has been transmitted; Fan-shaped area receiving means for receiving designation of a fan-shaped area corresponding to a predetermined viewing angle on an image; first rectangular image converting means for cutting out an image corresponding to the fan-shaped area from an omnidirectional image and converting the image into a rectangular image; And first display means for displaying the rectangular image.

Further, the present invention provides a transmitting apparatus for transmitting an omnidirectional image obtained by capturing the surrounding scenery, a receiving apparatus for receiving an omnidirectional image and displaying a part of the received omnidirectional image. An omnidirectional captured image transmission system, comprising: a transmitting mirror, and a reflecting mirror formed in a convex shape, and is installed to face the reflecting mirror, and captures an image of a surrounding landscape through the reflecting mirror. An omnidirectional image pickup means comprising an image pickup unit for obtaining an omnidirectional image, an omnidirectional image storage means for storing the taken omnidirectional image, and a transmission request for an omnidirectional image transmitted from the receiving device. A transmission request receiving unit that reads out the omnidirectional image from the omnidirectional image storage unit when receiving the transmission request, and transmits the read omnidirectional image to the receiving device. Is a user operation Therefore, transmission request transmitting means for transmitting a transmission request for an omnidirectional image to the transmitting device, omnidirectional image receiving means for receiving an omnidirectional image from the transmitting device to which the transmission request has been transmitted, and A fan-shaped area receiving means for receiving designation of a fan-shaped area corresponding to the predetermined viewing angle, a first rectangular image converting means for cutting out an image corresponding to the fan-shaped area from an omnidirectional image, and converting the image into a rectangular image, And a first display unit for displaying a rectangular image.

[0009] Here, the imaging unit is configured to project an omnidirectional image by obtaining an omnidirectional image by reflecting the surrounding scenery and the periphery of the reflecting mirror via the reflecting mirror, and to digitally convert the captured omnidirectional image. A digital image conversion unit that converts the image into an image and obtains a digital omnidirectional image;
A pixel value replacement unit that replaces the image with a value, and a compression code that obtains a compression-coded digital omnidirectional image by compression-coding a digital omnidirectional image in which image parts other than the image part in which the scenery is displayed are replaced with 0 values. The omnidirectional image receiving means receives a compression-encoded digital omnidirectional image from the transmitting device to which the transmission request has been transmitted, and decompresses the received compression-encoded digital omnidirectional image. May be configured.

[0010] Here, the image pickup unit reflects the surrounding scenery and the peripheral portion of the reflector via the reflector, and obtains an omnidirectional image. A digital image conversion unit that converts the image into an image and obtains a digital omnidirectional image, and a landscape extraction unit that extracts an image portion in which a landscape is displayed from the obtained digital omnidirectional image, the omnidirectional image receiving unit Receives the image portion in which the landscape is projected from the transmitting device that has transmitted the transmission request,
The image portion showing the scenery may be arranged again at the original position of the digital omnidirectional image.

Here, the omnidirectional imaging means further comprises:
The maximum diameter of the reflective surface is a diameter, a predetermined number of bars parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflective surface, so as to surround these bars. ,
A grid on a cylindrical peripheral surface composed of a predetermined number of rings provided at equal intervals in the axial direction is provided around the reflection surface and the imaging unit, and the first rectangular image conversion unit includes:
A configuration may be made such that a rectangular image is converted for each area formed by the grid displayed on the image corresponding to the received fan-shaped area.

In this case, the omnidirectional imaging means has a maximum diameter of the reflecting surface as a diameter, and a predetermined diameter parallel to the axis at equal intervals on a cylindrical peripheral surface having the axis of the reflecting surface as an axis. A number of rods and a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods, and the upper part of the cylindrical shape and the upper part of the reflection surface, And a grid on a cylindrical peripheral surface that supports the reflection surface, wherein the first rectangular image conversion means is provided for each area formed by the grid shown in an image corresponding to the received fan-shaped area. Alternatively, it may be configured to convert to a rectangular image.

[0013] Further, according to the present invention, there is provided a transmitting apparatus for capturing an image of a surrounding landscape, converting the obtained omnidirectional image into a rectangular image, and transmitting a part of the rectangular image to the receiving apparatus in accordance with a transmission request of the receiving apparatus. An omnidirectional device, comprising: a reflecting mirror formed in a convex shape; and an imaging unit installed opposite to the reflecting mirror and configured to capture an image of a surrounding landscape via the reflecting mirror and obtain an omnidirectional image. Imaging means; second rectangular image converting means for converting the obtained omnidirectional image into a rectangular image; rectangular image storing means for storing the rectangular image; and a predetermined viewing angle transmitted from the receiving device A rectangular image area receiving means for receiving a transmission request of the small rectangular image including the designation of the area of the small rectangular image to be transmitted, and, when the transmission request is received, from the rectangular image storage means to the area of the received small rectangular image. Cut out the corresponding image and Characterized in that it comprises a small rectangular image transmitting means for transmitting the image cut out to the receiving device.

Here, the omnidirectional imaging means further comprises:
The maximum diameter of the reflective surface is a diameter, a predetermined number of bars parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflective surface, so as to surround these bars. ,
A grid on a cylindrical peripheral surface composed of a predetermined number of rings provided at equal intervals in the axial direction is provided around the reflection surface and the imaging unit, and the second rectangular image conversion unit includes:
A configuration may be made such that a rectangular image is converted for each area formed by the grid displayed in the omnidirectional image.

Here, the omnidirectional imaging means has a maximum diameter of the reflecting surface as a diameter, and a predetermined diameter parallel to the axis at equal intervals on a cylindrical peripheral surface having the axis of the reflecting surface as an axis. A number of rods and a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods, and the upper part of the cylindrical shape and the upper part of the reflection surface, And a grid on a cylindrical peripheral surface that supports the reflection surface, wherein the second rectangular image conversion unit converts the rectangular image into a rectangular image for each region formed by the grid displayed in the omnidirectional image. May be configured.

Here, the reflecting mirror may be formed of a hyperboloid consisting of a single leaf of a bilobal hyperboloid.
Further, the present invention, a transmitting device that captures the surrounding landscape, converts the obtained omnidirectional image into a rectangular image, and transmits a part of the rectangular image to the receiving device according to a transmission request of the receiving device, An omnidirectional captured image transmission system including a receiving device that receives a partial image of a rectangular image and displays the received rectangular image, wherein the transmitting device includes a reflecting mirror formed in a convex shape, An omnidirectional imaging means, which is provided opposite to the reflecting mirror and includes an imaging unit that captures the surrounding landscape via the reflecting mirror and obtains an omnidirectional image, and converts the obtained omnidirectional image into a rectangular image. Second rectangular image converting means for converting, rectangular image storing means for storing the rectangular image, and a rectangular image area for receiving a small rectangular image transmission request including designation of a small rectangular image area transmitted from the receiving device Receiving means for receiving the transmission request In this case, there is provided a small rectangular image transmitting unit that cuts out an image corresponding to the area of the received small rectangular image from the rectangular image storage unit and transmits the cut out image to the receiving device. A rectangular image area transmitting means for transmitting a small rectangular image transmission request including designation of a small rectangular image area corresponding to the viewing angle, and an image corresponding to the small rectangular image area from the transmitting device having transmitted the transmission request. And a second display means for displaying the received image.

Here, the reflecting mirror may be formed of a hyperboloid composed of one leaf of a two-lobed hyperboloid.
Here, the omnidirectional imaging means may further include a maximum diameter of the reflection surface as a diameter, and a predetermined number parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflection surface. Rods, and a grid on a cylindrical peripheral surface composed of a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods. The second rectangular image conversion means may be provided so as to convert the image into a rectangular image for each area formed by the grid displayed in the omnidirectional image.

Here, the omnidirectional imaging means may be arranged so that the maximum diameter of the reflecting surface is a diameter, and the omnidirectional imaging means is equally spaced and parallel to the axis of the cylindrical surface around the axis of the reflecting surface. A number of rods and a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods, and the upper part of the cylindrical shape and the upper part of the reflection surface, And a grid on a cylindrical peripheral surface that supports the reflection surface, wherein the second rectangular image conversion unit converts the rectangular image into a rectangular image for each region formed by the grid displayed in the omnidirectional image. May be configured.

Further, the present invention relates to a receiving apparatus for receiving an omnidirectional image and displaying a part of the received omnidirectional image from a transmitting apparatus for transmitting an omnidirectional image obtained by capturing the surrounding scenery. A receiving method used, comprising: a transmitting request transmitting step of transmitting a transmitting request of an omnidirectional image to the transmitting device according to a user operation; and an omnidirectional image receiving receiving an omnidirectional image from the transmitting device to which the transmitting request has been transmitted. Step, a fan-shaped area receiving step of receiving a designation of a fan-shaped area corresponding to a predetermined viewing angle on the received omnidirectional image,
An image corresponding to the fan-shaped region is cut out from the omnidirectional image, and a first rectangular image converting step of converting the image into a rectangular image and a first displaying step of displaying the rectangular image are provided.

Further, according to the present invention, the surrounding landscape is imaged, the obtained omnidirectional image is converted into a rectangular image, and the rectangular image is transmitted to the receiving device in accordance with a transmission request of the receiving device, and is formed into a convex shape. Omnidirectional imaging means including a reflector and an imaging unit which is installed opposite to the reflector and captures the surrounding landscape via the reflector to obtain an omnidirectional image, and a rectangular image holding means for holding a rectangular image A second rectangular image conversion step of converting the captured omnidirectional image into a rectangular image, and a rectangle storing the rectangular image in the rectangular image holding means. An image storing step, a rectangular image area receiving step of receiving a transmission request of a small rectangular image including an area designation of a small rectangular image corresponding to a predetermined viewing angle transmitted from the receiving device, and a case where the transmission request is received. , From serial rectangular image holding means, cutting out the image corresponding to the area of the small rectangular image thus received, characterized in that it comprises a small rectangular image transmission step of transmitting the image cut out to the receiving device.

Further, the present invention is a computer-readable recording medium on which a program is recorded, characterized by including a program for causing a computer to execute the above method.

[0022]

DESCRIPTION OF THE PREFERRED EMBODIMENTS [Table of Contents] Configuration of omnidirectional captured image transmission system 1.1 Omnidirectional captured image transmission device 110 1.1.1 Omnidirectional imaging unit 117 1.1.2 A / D conversion unit 113 1.1.3 Information storage unit 114 1.4 Image processing unit 115 1.1.5 Image compression unit 118 1.1.6 Display unit 120 1.1.7 Input unit 119 1.1.8 Transmission / reception unit 116 1.2 Omnidirectional captured image reception device 150 1.2.1 Display unit 156 1.2.2 Input unit 155 1.2.3 Information storage unit 152 1.2.4 Image conversion unit 153 1.2.5 Image expansion unit 151 1.2.6 Transmission / reception unit 154 2. Operation of omnidirectional captured image transmission system 2.1 Image capturing operation of omnidirectional captured image transmitting device 110 2.2 Video receiving and display operation of omnidirectional captured image receiving device 150 2.3 Video receiving process 2.4 All around Display processing 2.5 Partial image display processing 2.6 Partial image generation processing 2.7 Upward movement processing 2.8 Leftward movement processing 2.9 Rightward movement processing 2.10 Downward movement processing 2.11 Enlarged display processing 2.12 Reduction 2. Display processing Conclusion [Text] The configuration and operation of an omnidirectional captured image transmission system as one embodiment according to the present invention will be described. 1. Configuration of omnidirectional captured image transmission system FIG. 1 shows a block diagram of an omnidirectional captured image transmission system as one embodiment according to the present invention.

As shown in FIG. 1, the omnidirectional captured image transmission system 100 includes an omnidirectional captured image transmitting device 110 and a plurality of omnidirectional captured image receiving devices 150. Next, the omnidirectional captured image transmitting device 110 and the omnidirectional captured image receiving device 150 will be described. 1.1 Omnidirectional Imaging Image Transmitting Device 110 Omnidirectional Imaging Image Transmitting Device 110
7, A / D conversion unit 113, information storage unit 114, image processing unit 115, image compression unit 118, input unit 119, display unit 1.
20 and a transmission / reception unit 116. 1.1.1 Omnidirectional Imaging Unit 117 The omnidirectional imaging unit 117 includes, as shown in FIG. 2, a vertically downward light-collecting unit 202 formed by a support 200 arranged above a vertically upward imaging unit 203. It is configured to be supported, and can capture a 360-degree landscape around the light collector 202 in real time. The support 200 is made of a material such as glass, and has a spherical surface centered on the focal point of the light collector 202 so that incident light is orthogonal to the glass surface in order to minimize refraction on the glass surface.

The configuration of the omnidirectional imaging section 117 and the principle of light collection are known, and are described in “Development of an Omnidirectional Vision Sensor for Mobile Robot” (Automation Technology Vol. 29, No. 6, 1997). Since it has been described in detail, only the characteristic portions will be described below. (1) Example of Video Captured by Omnidirectional Imaging Unit 117 An example of a video obtained by installing the omnidirectional imaging unit 117 on the roof of a tall building and capturing an image of the surroundings using the omnidirectional imaging unit 117 is shown. 3 is shown. FIG. 3 is a schematic drawing for easy understanding of the characteristics of the obtained video.
Attention is required because the image is not an image itself obtained by actually imaging using the omnidirectional imaging unit 117.

In FIG. 3, an image 303 includes a circle 301
, An area 304 surrounded by the circumference 301 and the circumference 302, and an area 306 surrounded by the circumference 302
Can be divided into In the area 305, a portion outside the light collecting section 202 is imaged by the imaging section 203, and
4, an image such as a landscape around the omnidirectional imaging unit 201 is captured, and the image capturing unit 203 itself is captured in an area 306.

In the area 304, a plurality of buildings are imaged.
The lower part of the building is imaged in a direction close to 2. (2) Configuration of Light Converging Unit 202 The light converging unit 202 is constituted by a part of a hyperboloid.
The mirror surface is formed on the convex surface of the convex leading portion of the hyperboloid obtained by cutting the hyperboloid with a plane perpendicular to the axis of the hyperboloid.

The light converging section 202 uses a two-lobed hyperboloid as a hyperboloid. As shown in FIG. 4, the bilobal hyperboloid is a curved surface obtained by rotating a hyperbola about an axis (Z axis), and includes hyperboloids 401 and 402. For the light condensing unit 202, a hyperboloid 401 in a region of Z> 0 among two hyperboloids is used. Note that the axis (Z axis) is referred to as the central axis of the hyperbola or simply the axis.

As shown in FIG. 4, the origin O441, the X axis 4
In the three-dimensional coordinate system O-XYZ 445 including the Y axis 443, the Y axis 443, and the Z axis 444, a two-lobed hyperboloid can be expressed by Expression 1.

[0029]

(Equation 1)

Here, a and b are constants defining the shape of the hyperboloid, c is a constant defining the position of the focal point,

[0031]

(Equation 2)

Is as follows. This hyperboloid has two focal points 411
(0, 0, c) and a focal point 412 (0, 0, -c). (3) Configuration of Imaging Unit 203 The imaging unit 203 includes a CCD or the like, captures an image of the light collecting unit 202 at predetermined time intervals measured by the imaging unit 203, converts the captured video into an electric signal, and will be described later. A / D converter 1
13 is output. (4) Positional Relationship Between Light Condensing Unit 202 and Image Pickup Unit 203 The positional relationship between the light collecting unit 202 and the image pickup unit 203 is shown in FIG. As shown in FIG. 5, the origin O521, the X axis 52
2, in a three-dimensional coordinate system O-XYZ 525 composed of a Y-axis 523 and a Z-axis 524, the light-collecting unit 202 has a central axis as a Z-axis and two focal points 502 (0, 0, c) and a focal point 506.
Of the two-lobed hyperboloid having (0, 0, -c), the hyperboloid 501 is located in a region of Z> 0 and has a vertically downward convex shape.

The image pickup unit 203 picks up an image vertically upward with the center axis of the lens coincident with the Z axis, and sets the center of the lens as the focal point 506 (0, 0, -c). 1.1.2 A / D Conversion Unit 113 The A / D conversion unit 113 converts a video converted into an electric signal output from the imaging unit 203 into a digital signal, and outputs the digital signal to the information storage unit 114. 1.1.3 Information Storage Unit 114 The information storage unit 114 stores the video 6 converted into a digital signal output from the A / D conversion unit 113, as shown in FIG.
03, a processed video 621 output from the image processing unit 115 described later, a compressed video 622 output from the image compression unit 118 described later, and the input unit 11 described later
The coordinates 631 of the imaging center position, the radius 632 of the inner circumference, and the radius 633 of the outer circumference output from 9 are stored.

FIG. 7 schematically shows a video 603 converted into a digital signal stored in the information storage unit 114. The image 603 converted into a digital signal has a width of 750
And 490 pixels vertically. Video 603
Can be divided into an area 605 outside the circumference 601, an area 604 surrounded by the circumference 601 and the circumference 602, and an area 606 surrounded by the circumference 602. Here, pixels located on the circumference 601 and pixels located on the circumference 602 are included in the region 604. The center point and the radius of the circumference 601 and the circumference 602 are determined by coordinates 631 of an imaging center position described later, an inner circumference radius 632 described later, and an outer circumference radius 633 described later.

In an area 605, a portion outside the light collecting section 202 is imaged by the imaging section 203 and converted into a digital signal. In an area 604, an image such as a landscape around the omnidirectional imaging section 201 is captured. The image signal is digitalized, and in the area 606, the image pickup unit 203 itself is imaged and converted into a digital signal. This figure shows the information storage unit 114
603 converted to a digital signal stored in
It should be noted that this is not a digital signal of the actual video imaged by the omnidirectional imaging unit 117. 1.1.4 Image Processing Unit 115 The image processing unit 115 converts the image 6 converted into a digital signal.
03 is output from the A / D conversion unit 113 to the information storage unit 114 and stored in the information storage unit 114, the video 603 converted into a digital signal stored in the information storage unit 114 is read out, and the area 605 is read. And the pixels included in the area 606 are rewritten to the 0 value, and the area 605 and the area 6 are rewritten.
The processed image 621 in which the pixels included in 06 have been rewritten to 0 values is output to the information storage unit 114.

Specifically, the center coordinate of the image 603 is set to (X
₀, Y₀) And the radius of the inner circle is R _TwoAnd half of the outer circle
The diameter is R₁Then, the image processing unit 115 calculates Expression 3 and Expression 4
Is set to 0 for coordinates (x, y) that satisfy
rewrite.

[0037]

(Equation 3)

[0038]

(Equation 4)

1.1.5 Image Compression Unit 118 The image compression unit 118 stores information when the video 621 processed by the image processing unit 115 is output to the information storage unit 114 and stored in the information storage unit 114. The processed video 621 is read from the unit 114, the read processed video 621 is compressed, and the compressed video 622 is output to the information storage unit 114.

The image compression section 118 uses a run-length encoding method for encoding a continuous length of pixels of the same color as an image compression method that does not degrade image quality. 1.1.6 Display Unit 120 The display unit 120 displays the image 603 converted into a digital signal.
Is output from the A / D conversion unit 113 to the information storage unit 114 and is stored in the information storage unit 114. The video 603 converted into a digital signal stored in the information storage unit 114 is read and displayed. 1.1.7 Input Unit 119 While viewing the video displayed on the display unit 120, the operation manager of the omnidirectional captured image transmission system 100 determines the imaging center position of the video and the area where the imaging unit 203 itself is captured. 60
6 and the radius of an area 604 where a video such as a surrounding landscape is captured.

The assembling accuracy of the omnidirectional imaging unit 117 differs depending on the actual product.
Since there is a case where the axes 3 do not exactly match each other, the above-mentioned designation is made by the operation manager. In FIG. 7, the upper left point of the image 603 is the origin O611, and the X axis 61 is in the left direction.
1. A two-dimensional coordinate system O-XY with the Y axis 612 in the downward direction is set.

The input section 119 is operated by the operation manager of the omnidirectional captured image transmission system 100 to obtain the coordinates (X, Y) of the image capturing center position in the two-dimensional coordinate system O-XY shown in FIG.
631, the input of the radius 632 of the inner circumference circle for determining the area 606 and the input of the radius 633 of the outer circumference circle for determining the area 604 are received, and the received coordinates 631 of the imaging center position, the radius 632 of the inner circumference circle, and the radius of the outer circumference circle are received. 633 is the information storage unit 1
14 is output. 1.1.8 Transmission / Reception Unit 116 The transmission / reception unit 116 is an omnidirectional captured image receiving device 1 described later.
The communication unit 160 is connected to 50 transmission / reception units 154.

The transmitting / receiving section 116 receives the “video transmission request” and the receiving apparatus identification number for identifying the omnidirectional captured image receiving apparatus 150 from the transmitting / receiving section 154. Transceiver 116
When the “video transmission request” and the receiving device identification number are received from the transmission / reception unit 154, the compressed video 622, the coordinates 631 of the imaging center position, the inner circumference radius 632, and the outer circumference are received from the information storage unit 114. Radius 633 of the compressed image 622 read out, the coordinates 63 of the imaging center position
1. The radius 632 of the inner circumference and the radius 633 of the outer circumference are transmitted to the transmitting / receiving unit 154 of the omnidirectional captured image receiving device 150 identified by the received receiving device identification number. 1.2 Omnidirectional Captured Image Receiving Device 150 The omnidirectional captured image receiving device 150 includes an image decompression unit 151,
Information storage unit 152, image conversion unit 153, transmission / reception unit 15
4, an input unit 155 and a display unit 156. 1.2.1 Display Unit 156 The display unit 156 displays a display screen 701 shown in FIG.

The display screen 701 includes a video display unit 702,
Image receiving button 703, all-around image button 704, partial image button 705, up button 706, left button 7
07, a right button 708, a down button 709, an enlarge button 710, and a reduce button 711. The video display unit 702 displays the video imaged by the omnidirectional imaging unit 117 or a part of the video image.

The video receiving button 703 is operated by the user when requesting the omnidirectional captured image transmitting apparatus 110 to transmit a new video. All-around video button 704
Is operated by the user when the user wants to display the entire video imaged by the omnidirectional imaging unit 117 on the video display unit 702. When the all-around video button 704 is operated by the user, the entire captured video is displayed on the video display unit 702 of the display screen 701, as shown in FIG.

The partial image button 705 is
When the user wants to display an image of a part of the image captured by the omnidirectional imaging unit 117 in 2, the operation is performed by the user.
As shown in FIG. 10, in a two-dimensional coordinate system O-XY composed of an origin O760, an X axis 761, and a Y axis 762,
It is assumed that an image 776 having a point 763 at the center of the image exists. When the partial image button 705 is operated by the user, the sector area 7 which is a part of the captured image is
The video corresponding to 67 is displayed on the display screen 7 as shown in FIG.
01 is displayed on the image display unit 702.

The up button 706 is operated by the image display unit 702.
If you want to display the upper part of the part of the image displayed in
Operated by the user. When an image corresponding to the fan-shaped area 767 in FIG. 10 is displayed on the video display unit 702 of the display screen 701 and the user operates the up button 706, the point 763 is adjacent to the fan-shaped area 767.
Is displayed on the video display unit 702 of the display screen 701.

The left moving button 707 is used for the image display 702
If you want to display the left side of the part of the image displayed in,
Operated by the user. When an image corresponding to the fan-shaped area 767 in FIG. 10 is displayed on the image display unit 702 of the display screen 701 and the user operates the left shift button 707, the point 763 is adjacent to the fan-shaped area 767.
Sector 77 located on the left side with respect to a circle centered on
0 is displayed on the image display unit 70 of the display screen 701.
2 is displayed.

A rightward movement button 708 is displayed on the image display unit 702.
If you want to display the right side of a part of the video displayed in,
Operated by the user. When an image corresponding to the fan-shaped area 767 in FIG. 10 is displayed on the image display unit 702 of the display screen 701, when the user operates the right movement button 708, the point 763 is adjacent to the fan-shaped area 767.
Sector 76 located on the right side of the circumference around
9 is displayed on the image display unit 70 of the display screen 701.
2 is displayed.

The down button 709 is operated by the video display unit 702.
If you want to display the lower part of the image displayed in
Operated by the user. When an image corresponding to the fan-shaped area 767 in FIG. 10 is displayed on the image display unit 702 of the display screen 701 and the user operates the down button 709, the point 763 is adjacent to the fan-shaped area 767.
Are displayed on the video display unit 702 of the display screen 701.

An enlarge button 710 is used to enlarge and display a part of the image displayed on the image display unit 702.
Operated by the user. The reduction button 711 is operated by the user when the user wants to reduce and display a part of the image displayed on the image display unit 702. Display 156
Receives the “full-circle display” from the input unit 155, reads the expanded video 732 stored in the information storage unit 114, and displays the entirety of the read expanded video 732 on the video display unit 702.

The display unit 156 includes an image conversion unit 153.
When receiving the video, the received video is displayed on the video display unit 702. 1.2.2 Input Unit 155 The input unit 155 is provided by the user for receiving the video image receiving button 703, all-round video button 704, and partial video button 7 shown in FIG.
05, up button 706, left button 707, right button 708, down button 709, enlarge button 71
Operation of 0 and reduction button 711 is accepted.

Upon receiving the operation of the video receiving button 703 by the user, the input unit 155 outputs a “video transmission request” to the transmitting / receiving unit 154. When the input unit 155 receives the operation of the full-circle image button 704 by the user, the input unit 155 displays “all-circle display” meaning a request to display the entire image.
Output to 56. When the input unit 155 receives the operation of the partial video button 705 by the user, the input unit 155 converts the “partial display” meaning a request to display a part of the video into the image conversion unit 153.
Output to

When the image corresponding to the sector area 767 shown in FIG. 10 is displayed on the image display unit 702, and the coordinate 741 of the upper left point of the partial image in the information storage unit 152 indicates the coordinate of the point 771, the input unit 155 Move up button 7 by user
When the operation of step 06 is received, “upper display” meaning a request to display the upper part of the displayed part of the video is output to the image converter 153. Next, the input unit 155 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, calculates the coordinates of the point 772 using the read coordinates 741, and substitutes the calculated coordinates of the point 772 into the partial The information is written in the information storage unit 152 as the coordinates 741 of the upper left point of the video.

When the image corresponding to the sector area 767 shown in FIG. 10 is displayed on the image display unit 702, and the coordinate 741 of the upper left point of the partial image in the information storage unit 152 indicates the coordinate of the point 771, the input unit 155 Move left button 7 by user
When the operation of step 07 is received, “left display” meaning a request to display the left side of the displayed partial image is output to the image conversion unit 153. Next, the input unit 155 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, calculates the coordinates of the point 774 using the read coordinates 741, and substitutes the calculated coordinates of the point 774 into the partial The information is written in the information storage unit 152 as the coordinates 741 of the upper left point of the video.

When the image corresponding to the sector area 767 shown in FIG. 10 is displayed on the image display unit 702, and the coordinate 741 of the upper left point of the partial image in the information storage unit 152 indicates the coordinate of the point 771, the input unit 155 Move right button 7 by user
When the operation of step 08 is accepted, “right display” meaning a request to display the right side of the displayed partial image is output to the image conversion unit 153. Next, the input unit 155 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, calculates the coordinates of the point 775 using the read coordinates 741, and substitutes the calculated coordinates of the point 775 into the partial The information is written in the information storage unit 152 as the coordinates 741 of the upper left point of the video.

When the image corresponding to the sector area 767 shown in FIG. 10 is displayed on the image display unit 702, and the coordinate 741 of the upper left point of the partial image in the information storage unit 152 indicates the coordinate of the point 771, the input unit 155 Move down button 7 by user
When the operation of step 09 is received, “lower display” meaning a request to display the lower part of the displayed image is output to the image converter 153. Next, the input unit 155 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, calculates the coordinates of the point 773 using the read coordinates 741, and substitutes the calculated coordinates of the point 773 into the partial The information is written in the information storage unit 152 as the coordinates 741 of the upper left point of the video.

The input unit 155 receives a request for an enlarged display of a part of the displayed image when the user accepts an operation of the enlarge button 710 while a part of the image is displayed on the image display unit 702. Is output to the image conversion unit 153. When a user operates the reduction button 711 while a part of an image is displayed on the image display unit 702, the input unit 155 performs “reduction” which means a request for reduction display of the displayed part of the image. "Display" is output to the image conversion unit 153. 1.2.
3 Information storage unit 152 The information storage unit 152 stores the compressed video 731 output from the transmission / reception unit 154 and the video 73 decompressed by the image decompression unit 151, as shown in FIG.
2, and the coordinates 75 of the imaging center position in the two-dimensional coordinate system O-XY shown in FIG.
1. The radius 752 of the inner circumference and the radius 753 of the outer circumference are stored.

The information storage unit 152 stores in advance
The coordinates 741 of the upper left point of the predetermined partial image in the two-dimensional coordinate system O-XY shown in FIG. 7 are stored. In addition, the information storage unit 152 receives the coordinates of the upper left point of the partial video from the input unit 155, and stores the coordinates of the upper left point of the received partial video as the coordinates 741 of the upper left point of the partial video. 1.2.4 Image Conversion Unit 153 (1) Principle of Image Conversion The image captured by the omnidirectional imaging unit 117 is distorted as shown in FIG.
Here, the principle of the conversion of the distorted video will be described.

The image distortion shown in FIG. 3 is considered by being decomposed into radial distortion passing through the center point 307 and circumferential distortion centering on the center point 307. (Radial distortion) As shown in FIG.
1. A three-dimensional coordinate system O-XYZ composed of a Z-axis 805 and an XY plane 872 perpendicular to the paper surface has a parameter a8.
A hyperboloid 801 having a focal point 803 specified by 81, b882, c883, 884;
4 and an image plane 807 separated from the focal point 804 by a focal distance f808 of the camera on the Z axis 805.

The incident light beam 811 at an elevation angle of 0 degree with respect to the Z axis 805 is reflected at a point 851 on the hyperboloid 801, passes through a point 841 on the image plane 807, and proceeds toward the focal point 804. . The incident light ray 812 at an elevation angle Δd degrees with respect to the Z axis 805 is reflected at a point 852 on the hyperboloid 801, passes through a point 842 on the image plane 807, and
Proceed toward.

The incident light beam 813 at an elevation angle 2Δd degrees with respect to the Z-axis 805 is reflected at a point 853 on the hyperboloid 801, passes through a point 843 on the image plane 807, and becomes a focal point 804.
Proceed toward. Similarly, the incident light ray 81 at an elevation angle of 3Δd degrees, 4Δd degrees, and 5Δd degrees with respect to the Z axis 805.
4, 815 and 816 respectively reflect at points 854, 855 and 856 on the hyperboloid 801 and pass through points 844, 845 and 846 on the image plane 807 and proceed toward the focal point 804.

In this way, the points 841, 842, 8 on the image plane 807 are generated by the light rays having different elevation angles by Δd degrees.
43, 844, 845, 846 are drawn. Points 841, 842, 843, 844, 845, 8 thus drawn
Since the lines 46 are not arranged at equal intervals, a radial distortion passing through the center point 307 appears as shown in FIG. In order to correct this distortion, these points 841, 842,
It can be seen that 843, 844, 845, and 846 should be redrawn at regular intervals.

Next, these points 841, 842, 84
A method of calculating the distances of 3, 844, 845, and 846 from the Z axis 805 will be described. As shown in FIG. 13, the origin O958, the Z-axis 953, and the XY plane 9
The focus 9 specified by the parameters a968, b969, and c970 in the three-dimensional coordinate system O-XYZ
A hyperboloid 951 with a 59 and a focal point 960 at the center of the lens
, And an image plane 956 separated from the focal point 960 by a focal distance f967 of the camera on the Z axis 953.

A light ray 963 emitted from the point P (X, Y, Z) 962 enters the Z axis 953 at an elevation angle α degrees, is reflected at a point 981 on the hyperboloid 951, and has a distance R from the Z axis 953.
Plane 95 passing through point p (x, y) 964 on image plane 956
At an angle γ relative to the focal point 960.

The distance R is calculated as follows. Equation 5 is obtained from the above-mentioned paper "Development of an Omnidirectional Vision Sensor for a Mobile Robot".

[0067]

(Equation 5)

From Expression 5, Expression 6 is obtained.

[0069]

(Equation 6)

[0070]

(Equation 7)

Here, if Expression 7 is used, Expression 6 becomes Expression 8.

[0072]

(Equation 8)

From Expression 8, Expression 9 is obtained.

[0074]

(Equation 9)

[0075]

(Equation 10)

[0076]

[Equation 11]

Here, assuming Expressions 10 and 11, Expression 9
Is as shown in Expression 12.

[0078]

(Equation 12)

From Expression 12, Expression 13 is obtained.

[0080]

(Equation 13)

From Expression 13, Expression 14 is obtained.

[0082]

[Equation 14]

Therefore, Expression 15 is obtained.

[0084]

(Equation 15)

From the above-mentioned article “Development of Omnidirectional Vision Sensor for Mobile Robot”, Expression 16 is obtained.

[0086]

(Equation 16)

From equation (16), equation (17) is obtained.

[0088]

[Equation 17]

Thus, by using the equations (15) and (17), Z
The distance R of the point p (x, y) 964 drawn on the image plane 956 from the Z axis 953 can be calculated from the incident light ray 963 at an elevation angle α degrees with respect to the axis 953. In the present embodiment, the range from the elevation angle of 0 degrees to the elevation angle of 70 degrees is evenly set to 200 degrees.
And 201 rays are incident on the divided elevation angle, 201 points are obtained on the image plane 956, and the distance R from the Z axis 805 of the obtained 201 points is calculated as follows. , Equation 1
5 and Equation 17 are used. FIG. 14 illustrates the three-dimensional coordinate system O-XYZ shown in FIG. 13 viewed from the Z-axis 953 direction.

As shown in FIG. 14, the origin O1001, X
In a three-dimensional coordinate system O-XYZ composed of an axis 1003, a Y axis 1002, and a Z axis (not shown) perpendicular to the paper surface, there is a hyperboloid 1004 whose axis coincides with the Z axis. Ray 101 emitted from point P (X, Y, Z) 1011
4 is incident on the X axis 1003 at an angle θ degrees, is reflected at a point on the hyperboloid 1004, and has a point p (x, y) 1 on the image plane.
012 at an angle θ with respect to the X axis 1003.

Similarly, a light ray (not shown) incident at an angle of 2θ degrees with respect to the X axis 1003 is reflected at a point on the hyperboloid 1004, and is reflected at a point on the image plane at an angle of 2θ with respect to the X axis 1003. Reach in degrees. As described above, since a plurality of light beams incident at equal intervals on the X axis 1003 draw points at equal intervals on the image plane, distortion on the circumference around the center point 307 shown in FIG. It is thought that there is no. (2) Determination of Position of Pixel for Image Conversion The image conversion unit 153 determines the position of a pixel for image conversion of the expanded video 732 stored in the information storage unit 152 as follows.

The image conversion section 153 reads the coordinates 751, the inner circumference radius 752, and the outer circumference radius 753 of the imaging center position stored in the information storage section 152. The image conversion unit 153 sets the read coordinates 751 of the imaging center position as the coordinates of the center point 1103 in FIG. 15, and uses the read inner radius 752 and outer radius 753 as the distances 1104 and 753 in FIG. 15, respectively. The distance is 1105.

The image conversion section 153 draws a circumference 1102 with the distance 1104 as the radius and the center point 1103 as a center point, and draws a circumference 1101 with the distance 1105 as the radius and the center point 1103 as the center point. As shown in FIG. 15, the image conversion unit 153 converts the circumference 1101 and the circumference 1102 on the decompressed video 732 stored in the information storage unit 152.
Is calculated using Equations 15 and 17,
The center point 11 is defined as the radius of the distance R from the Z axis 805 to the number of points.
Draw 201 circumferences centered on 03.

The image conversion unit 153 draws one line segment 1106 from the center point 1103, and draws the drawn line segment 1106.
And the 201 intersections drawn above are drawn. Next, the drawn 201 circles are divided into the drawn 20 circles.
Based on one intersection, the arc is divided into 720 equal-length arcs, and points at both ends of the arc are obtained. Thus, 2
On each of the 01 circumferences, 720 points are obtained.

The 720 points on each of the 201 circles thus obtained are set as the positions of the pixels for image conversion. (3) Calculation of Luminance of Pixel for Image Conversion The image conversion unit 153 determines the luminance of the pixel at the position where the image conversion is performed as follows.

As shown in FIG. 16, a pixel 1202 is arranged on the right side of a pixel 1201, a pixel 1203 is arranged below the pixel 1201, and a pixel 1202 is arranged below the pixel 1202.
4, and four pixels 1201, 1202, 120
3 and 1204 are luminances Sa, Sb, Sc, and Sd, respectively.
Have Assuming that a point separated from the pixel center 1211 which is the center position of the pixel 1201 by α in the left lateral direction and β in the downward direction is a point 1221 for image conversion, the luminance Ss at the point 1221 for image conversion is given by Expression 18. Can be

[0097]

(Equation 18)

(4) Relationship between the position of the pixel to be image-converted and the position of the pixel after conversion The pixels to be subjected to image conversion included in the fan-shaped area 1111 shown in FIG. 15 are extracted and shown as a fan-shaped area 1302 in FIG. The image conversion unit 153 is a pixel to be subjected to image conversion included in the sector area 1302, and includes arcs 1321, 1322, 1323, 1324, 1325, centered on the center point 1301.
Those existing on 1326 have lines 1341, 1342, 1343, 1344, and 134 on a rectangular area 1310, assuming that the distance between adjacent pixels to be converted is equal.
5, 1346, and lines 1331, 1332, 133
3, 1334, 1335, and 1336, lines 1351 and 1352 on the rectangular area 1310 are set at equal distances from adjacent pixels for image conversion.
1353, 1354, 1355, and 1356.

In this way, the image conversion unit 153 converts the pixels on the sector area 1111 shown in FIG.
Place on top of 10. At this time, the image conversion unit 153 calculates the luminance of each pixel using Expression 18. (5) Output of Video Upon receiving the “partial display” from the input unit 155, the image conversion unit 153 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, and uses the read coordinates 741 to perform image conversion. The fan-shaped area to be converted is determined, and the rectangular area 1
The luminance of the pixel included in 310 is calculated as described above and output to display unit 156.

Upon receiving “up display” from the input unit 155, the image conversion unit 153 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, and uses the read coordinates 741 to perform a sector conversion for the image. The area is determined, and the brightness of the pixels included in the rectangular area 1310 adjacent to the rectangular area in the upper direction of the video is calculated as described above and output to the display unit 156.

Upon receiving “left display” from the input unit 155, the image conversion unit 153 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, and uses the read coordinates 741 to perform sector conversion for image conversion. The area is determined, and the brightness of the pixels included in the rectangular area 1310 adjacent to the rectangular area adjacent to the left side of the image is calculated and output to the display unit 156 as described above.

Upon receiving the "right display" from the input unit 155, the image conversion unit 153 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, and uses the read coordinates 741 to convert the sector into a sector shape. The region is determined, and the luminance of the pixels included in the rectangular region 1310 adjacent to the right side of the video in the rectangular direction is calculated as described above, and output to the display unit 156.

Upon receiving “down display” from the input unit 155, the image conversion unit 153 reads the coordinates 741 of the upper left point of the partial video from the information storage unit 152, and uses the read coordinates 741 to perform a sector conversion for image conversion. The area is determined, and the luminance of the pixels included in the rectangular area adjacent to the rectangular area 1310 in the lower direction of the video is calculated as described above and output to the display unit 156.

Upon receiving the “enlarged display” from the input unit 155, the image conversion unit 153 enlarges the rectangular area 1310 and outputs it to the display unit 156. Image conversion unit 153
Receives “reduced display” from the input unit 155, reduces the rectangular area 1310 and outputs the reduced area to the display unit 156. 1.2.5 Image Decompression Unit 151 The image decompression unit 151 outputs the video 731 compressed in the information storage unit 152 from the transmission / reception unit 154,
When the compressed video 731 is stored in the information storage unit 152, the compressed video 731 is read from the information storage unit 152, and the read compressed video 731 is decompressed.
32 to the information storage unit 152. 1.2.6 Transmission / Reception Unit 154 The transmission / reception unit 154 is connected to the transmission / reception unit 116 of the omnidirectional captured image transmission device 110 via the communication line 160.

The transmitting / receiving section 154 has a receiving device identification number for identifying the omnidirectional captured image receiving device 150. When receiving the “video transmission request” from the input unit 155, the transmission / reception unit 154 transmits the “video transmission request” and the reception device identification number of the omnidirectional captured image reception device 150 to the transmission / reception unit 116.

The transmission / reception unit 154 converts the pixels contained in the areas 605 and 606 from the transmission / reception unit 116 to 0 values and compresses the compressed image, the coordinates of the imaging center position, the radius of the inner circumference, and the radius of the outer circumference. Upon reception, the received compressed video, the coordinates of the imaging center position, the radius of the inner circumference, and the radius of the outer circumference are output to the information storage unit 152. 2. Operation of Omnidirectional Image Transmission System Here, the operation of the omnidirectional image transmission system shown in FIG. 1 will be described with reference to the flowcharts in FIGS. 2.1 Operation of Image Capture of Omnidirectional Image Transmission Apparatus 110 The operation of image capture of the omnidirectional image transmission apparatus 110 will be described with reference to the flowchart shown in FIG.

The omnidirectional imaging section 117 captures the surrounding scenery and outputs the captured image to the A / D conversion section 113 (step S1401). The A / D conversion section 113 outputs the image from the omnidirectional imaging section 117. The converted video is converted into a digital signal and output to the information storage unit 114 (step S1402). The information storage unit 114 stores the digitalized video output from the A / D conversion unit 113 (step S140).
3), the image processing unit 115 reads the digital signalized video stored in the information storage unit 114, and
The pixels included in the image data 05 and 606 are rewritten to the 0 value, and the rewritten video is output to the information storage unit 114 (step S1404).
Is compressed, and the compressed video is output to the information storage unit 114 (step S140).
5). The omnidirectional imaging unit 117 measures whether a predetermined time has elapsed, and if the predetermined time has elapsed (step S
1406), returning to step S1401, the omnidirectional imaging section 117 repeats imaging, and if the predetermined time has not elapsed, the omnidirectional imaging section 117 waits for the lapse of the predetermined time (step S1407). 2.2 Video receiving and displaying operation of omnidirectional captured image receiving device 150 The video receiving and displaying operation of the omnidirectional captured image receiving device 150 will be described with reference to the flowcharts shown in FIGS.

When input unit 155 receives image reception button 703 (step S1501), image reception processing shown in the flowchart of FIG. 21 is executed (step S15).
11), the control returns to step S1501. Input unit 15
5 receives the all-round video button 704 (step S
1502), an all-round display process shown in the flowchart of FIG. 22 is executed (step S1512), and step S15 is performed.
Control returns to 01. When the input unit 155 is the partial video button 70
When 5 is received (step S1503), the partial video display processing shown in the flowchart of FIG. 23 is executed (step S1513), and the control returns to step S1501.
When the input unit 155 receives the up button 706 (step S1504), the up process shown in the flowchart of FIG. 25 is executed (step S1514), and the control returns to step S1501. When input unit 155 receives left movement button 707 (step S1505), FIG.
Is executed (step S1515), and the control returns to step S1501. When the input unit 155 receives the rightward movement button 708 (step S1506), the rightward movement processing shown in the flowchart of FIG. 27 is executed (step S1516), and step S1516 is executed.
Control returns to 1501. The input unit 155 is the down button 7
When 09 is received (step S1507), the downward movement process shown in the flowchart of FIG. 28 is executed (step S1517), and the control returns to step S1501. When the input unit 155 receives the enlargement button 710 (step S1508), the enlargement display processing shown in the flowchart of FIG. 29 is executed (step S1518), and step S1 is performed.
Control returns to 501. The input unit 155 is a reduction button 711
Upon receiving (step S1509), a reduced display process shown in the flowchart of FIG. 30 is executed (step S1509).
1519), control returns to step S1501. 2.3 Video Reception Processing The video reception processing shown in step S1511 of the flowchart in FIG. 19 will be described with reference to the flowchart shown in FIG.

The input unit 155 outputs a “video transmission request” to the transmission / reception unit 154 (step S1701), and the transmission / reception unit 154 transmits the output “video transmission request” to the transmission / reception unit 11.
6 (step S1702), and the transmission / reception unit 116
Reads the compressed video, the coordinates of the imaging center position, the radius of the inner circumference circle, and the radius of the outer circumference circle from the information storage unit 114, and transmits them to the transmission / reception unit 154 (step S170).
3), the transmission / reception unit 154 receives the compressed video, the coordinates of the imaging center position, the radius of the inner circle, and the radius of the outer circle, and receives the received compressed video and the coordinates of the imaging center position. And the radius of the inner circle and the radius of the outer circle are stored in the information storage unit 15.
2, the information storage unit 152 stores the output coordinates of the imaging center position, the radius of the inner circumference circle, and the radius of the outer circumference circle (step S1404), and the image expansion unit 151 stores the information in the information storage unit 152. Expands the compressed video stored in
The expanded video is output to the information storage unit 152, and the information storage unit 152 stores the expanded video (step S1).
705). 2.4 Full circumference display processing The full circumference display processing shown in step S1512 of the flowchart of FIG. 19 will be described with reference to the flowchart shown in FIG.

The input section 155 displays “all-around display” on the display section 1.
56 (step S1801), and the display unit 156
Reads the entire video from the information storage unit 152 and displays the entire read video on the video display unit 702 (step S1802). 2.5 Partial Video Display Processing The partial video display processing shown in step S1513 of the flowchart of FIG. 19 will be described with reference to the flowchart shown in FIG.

The partial image generation processing shown in the flowchart of FIG. 24 is executed (step S1901), and the image conversion unit 153 displays the rectangular area 1310 for which the luminance has been calculated on the display unit 1.
56 (step S1902), and the display unit 156
Displays the rectangular area 1310 on the video display unit 702 (step S1903). 2.6 Partial Video Generation Processing The partial video generation processing shown in step S1901 of the flowchart of FIG. 23 will be described with reference to the flowchart shown in FIG.

The image conversion unit 153 uses the radius calculated by the equation 17 in the sector area 1302 to calculate the center point 1301.
A predetermined number of arcs centered on are drawn (step S200
1) The image conversion unit 153 draws a predetermined number of line segments starting from the center point 1301 in the sector area 1302 (step S2002), and the image conversion unit 153 generates the predetermined number of arcs and the predetermined number of line segments. The intersection is determined as the image conversion position (Step S2003), and the image conversion unit 153 calculates the luminance at the image conversion position using Expression 18 (Step S200).
4), the image conversion unit 153 expands the obtained luminance on a line segment on the rectangular area 1310 (Step S200)
5). 2.7 Upward Movement Processing Upward movement processing shown in step S1514 of the flowchart of FIG. 19 will be described with reference to the flowchart shown in FIG.

The input unit 155 outputs “upper display”, which means a request to display the upper part of the displayed part of the image, to the image converter 153, and outputs the coordinates 741 of the upper left point of the partial image from the information storage unit 152. And the read coordinates 741
Is used to calculate the coordinates of the point 772, and the calculated point 77
2 is written in the information storage unit 152 as the coordinates 741 of the upper left point of the partial image,
Is moved upward (step S2101), the partial video generation processing shown in the flowchart of FIG. 24 is executed (step S2102), and the image conversion unit 153 outputs the rectangular area 1310 for which the luminance has been calculated to the display unit 156 (step S2101). S
1203), the display unit 156 displays the rectangular area 1310 on the video display unit 702 (step S2104). 2.8 Left Movement Processing The left movement processing shown in step S1515 of the flowchart of FIG. 19 will be described with reference to the flowchart shown in FIG.

The input unit 155 outputs “left display” meaning a request to display the left side of the displayed partial image to the image conversion unit 153, and outputs the coordinates 741 of the upper left point of the partial image from the information storage unit 152. And the read coordinates 741
Is used to calculate the coordinates of the point 774, and the calculated point 77
4 is written in the information storage unit 152 as the coordinates 741 of the upper left point of the partial image, so that the sector area 1302
Is moved to the left (step S2201), the partial video generation processing shown in the flowchart of FIG. 24 is executed (step S2202), and the image conversion unit 153 outputs the rectangular area 1310 whose luminance has been calculated to the display unit 156 (step S2201). S
2203), the display unit 156 displays the rectangular area 1310 on the video display unit 702 (step S2204). 2.9 Right Movement Processing The right movement processing shown in step S1516 of the flowchart in FIG. 20 will be described with reference to the flowchart shown in FIG.

The input unit 155 outputs “right display” meaning a request to display the right side of the displayed partial image to the image conversion unit 153, and outputs the coordinates 741 of the upper left point of the partial image from the information storage unit 152. And the read coordinates 741
Is used to calculate the coordinates of the point 775, and the calculated point 77
5 is written into the information storage unit 152 as the coordinates 741 of the upper left point of the partial image, so that the sector area 1302
Is moved to the right (step S2301), the partial video generation processing shown in the flowchart of FIG. 24 is executed (step S2302), and the image conversion unit 153 outputs the rectangular area 1310 for which the luminance has been calculated to the display unit 156 (step S2301). S
2303), the display unit 156 displays the rectangular area 1310 on the video display unit 702 (step S2304). 2.10 Downward Movement Processing Downward movement processing shown in step S1517 of the flowchart in FIG. 20 will be described with reference to the flowchart shown in FIG.

The input unit 155 outputs “lower display”, which means a request to display the lower side of the displayed part of the image, to the image converter 153, and outputs the coordinates of the upper left point of the partial image from the information storage unit 152. 741 is read, and the read coordinates 741
Is used to calculate the coordinates of the point 773, and the calculated point 77
3 is written in the information storage unit 152 as the coordinates 741 of the upper left point of the partial image,
Is moved downward (step S2401), the partial video generation processing shown in the flowchart of FIG. 24 is executed (step S2402), and the image conversion unit 153 outputs the rectangular area 1310 for which the luminance has been calculated to the display unit 156 ( Step S
2403), the display unit 156 displays the rectangular area 1310 on the video display unit 702 (step S2404). 2.11 Enlarged Display Processing The enlarged display processing shown in step S1518 of the flowchart in FIG. 20 will be described with reference to the flowchart shown in FIG.

The input unit 155 outputs “enlarged display” indicating a request for enlarged display of a part of the displayed image to the image conversion unit 153 (step S2501). The enlarged rectangular area 1310 is output to the display unit 156 and enlarged (Step S
2502) The display unit 156 displays the enlarged rectangular area 131.
0 is displayed on the video display unit 702 (step S250).
3). 2.12 Reduced Display Processing The reduced display processing shown in step S1519 of the flowchart in FIG. 20 will be described with reference to the flowchart shown in FIG.

The input unit 155 outputs “reduced display” meaning a request for reduced display of the displayed part of the video to the image conversion unit 153 (step S2601). The reduced rectangular area 1310 is output to the display unit 156 and reduced (Step S
2602), the display unit 156 displays the reduced rectangular area 131.
0 is displayed on the video display unit 702 (step S260).
3). 3. Conclusion As described above, in the omnidirectional captured image transmitting apparatus 110, the omnidirectional imaging section 117 captures an image of the surrounding scenery, stores the captured video in the information storage section 114, and stores a plurality of omnidirectional captured image receiving apparatuses. In response to an instruction from the input unit 155 of the input / output unit 150, the transmission / reception unit 116
The image conversion unit 153 removes the distortion of the transmitted video, and the display unit 156 displays the transmitted video. As described above, an image corresponding to a desired viewing angle of a plurality of users is cut out from an image in which a 360-degree surrounding landscape is projected, and distortion of a video is removed, and the image can be provided to each user. There is an effect that images in different directions can be simultaneously viewed as if the user operates each camera.

Further, since a hyperboloid is used as the reflecting surface, a wider field of view can be ensured on the side and lower side of the image pickup unit as compared with the case where a spherical surface or a conical surface is used as the reflecting surface.
There is an effect that a high resolution can be obtained. Although the present invention has been described based on the above embodiment, it is needless to say that the present invention is not limited to the above embodiment. That is, the following cases are also included in the present invention. (1) In the above embodiment, the image processing unit 115 reads out the video 603 converted into a digital signal stored in the information storage unit 114, and rewrites the pixels included in the areas 605 and 606 to 0 values. The processed image 621 in which the pixels included in the regions 605 and 606 have been rewritten to 0 values is output to the information storage unit 114. The image processing unit 115 extracts only the pixels included in the region 604 by A video, in which only the pixels included in the area 604 are extracted, and the number of horizontal pixels and the number of vertical pixels of the video 603 are output to the information storage unit 114. The image compression unit 118 compresses the video and stores the information. The transmitting / receiving unit 116 reads out the number of horizontal pixels and the number of vertical pixels of the compressed video and the video 603 from the information storage unit 114, and transmits the read data to the transmitting / receiving unit 154. The transmission / reception unit 154 receives the transmitted image and the number of horizontal pixels and the number of vertical pixels of the image 603, and outputs the received image and the information to the information storage unit 152. The image expansion unit 151 reads the number of horizontal pixels and the number of vertical pixels of the compressed image and the image 603 from the information storage unit 152, and reads the number of horizontal pixels and the number of vertical pixels. Even if the read compressed video is expanded and reproduced on a rectangular area where the number is the number of horizontal pixels and the number of vertical pixels, respectively, the compressed video is relocated to the position of the original image and written to the information storage unit 152. Good.

Here, the image compression unit 118 compresses the video, and the image expansion unit 152 expands the compressed video. However, the image compression unit 118 does not compress the video, The decompression unit 152 may not decompress the compressed video. By doing so, the transmission / reception unit 116
Since the amount of video data transmitted to the transmission / reception unit 154 from can be reduced, the transmission time can be shortened. (2) In the above-described embodiment, the input unit 119 is operated by the operation manager of the omnidirectional captured image transmission system 100 by using the two-dimensional coordinate system O-XY shown in FIG. Y) 631, radius 632 of the inner circumference circle that determines the area 606, and radius 63 of the outer circumference circle that determines the area 604
3, the coordinates 631 of the received imaging center position, the radius 632 of the inner circumference circle, and the radius 633 of the outer circumference circle are received.
Is output to the information storage unit 114, but the input unit 119
In place of, a video recognition unit that recognizes the captured video is provided. The video recognition unit automatically recognizes the coordinates of the imaging center position of the video, the radius of the inner circumference circle and the radius of the outer circumference circle, and captures the recognized video. The coordinates of the center position, the radius of the inner circle, and the radius of the outer circle may be output to the information storage unit 114.

In this way, the assembling accuracy of the omnidirectional imaging unit 117 differs for each actual product,
When the axis of the image pickup unit 203 does not exactly coincide with the axis of the image pickup unit 203, the operation of specifying the axis of the light collection unit 202 and the axis of the image pickup unit 203 by the operation manager for each actual product can be omitted. This has the effect. (3) In the above embodiment, the image conversion unit 153 provided in the omnidirectional captured image receiving device corrects a distortion of a part of the video and outputs the corrected video to the display unit.
The omnidirectional captured image transmitting device is provided with an image conversion unit, and the image converting unit provided on the omnidirectional captured image transmitting device is configured to perform image correction using the distortion correction method described in the present embodiment. The entire distortion is corrected, and the corrected entire image is output to the information storage unit 114. The transmission / reception unit 116 reads the corrected entire image from the information storage unit 114, and transmits the read entire image to the transmission / reception unit 154. The transmission / reception unit 154 outputs the transmitted corrected whole image to the information storage unit 152, and the display unit 156 reads a part of the corrected whole image from the information storage unit 152 and displays the read out part. You may.

Further, an image conversion unit is provided on the omnidirectional captured image transmission device side, and the image conversion unit provided on the omnidirectional captured image transmission device side uses the distortion correction method described in the present embodiment. Then, the distortion of the entire captured image is corrected, the corrected entire image is output to the information storage unit 114, and the transmission / reception unit 116 transmits from the omnidirectional captured image receiving apparatus including designation of the area of the small rectangular image. The request is accepted and the information storage unit 11
4 reads out a small rectangular image composed of a small rectangular image area included in the transmission request from the corrected whole video stored in the transmission request, and transmits the small rectangular image to the transmission / reception unit 154.
54 may output the transmitted small rectangular image to the information storage unit 152, and the display unit 156 may read the small rectangular image from the information storage unit 152 and display the read small rectangular image.

With this configuration, it is not necessary to provide the image conversion unit 153 on the omnidirectional captured image receiving device 150 side.
The device scale of the omnidirectional captured image receiving device 150 can be further reduced. (4) In the above embodiment, the shape of the light converging section 202 is a hyperboloid, but may be a spherical surface or a conical surface.
In this case, the image conversion unit 153 performs an image conversion process according to the shape.

As shown in FIG. 31 (a), a predetermined number of rods 2702 parallel to the cylinder are arranged on the peripheral surface of the cylinder at equal intervals in the axial direction of the cylinder, and these rods 2702 are surrounded. As shown in FIG. 31 (b), a grid 2701 having a cylindrical peripheral surface composed of a predetermined number of rings 2703 provided at equal intervals in the axial direction of the cylindrical shape is
The omnidirectional imaging unit 117 is installed so as to go around, and the image conversion unit 153 converts the mapping 27 of the cylindrical circumferential lattice 2701 on the captured video 2722 as shown in FIG.
21 are recognized, and each small area 2723 is recognized.
The image included in the rectangular area may be subjected to image conversion processing for correcting the pixel included in the rectangular area into a rectangular area, and the obtained image of the rectangular area may be output to the display unit 156. As a method of image conversion processing for correcting the image of each small area 2723 into an image of a rectangular area, linear interpolation is performed in the circumferential direction and the radial axis direction of each small area.

It is needless to say that the cylindrical lattice 2701 is not limited to a cylindrical shape.
It may be a polygon. With this configuration, even if the shape of the light collecting unit 202 is a hyperboloid, a spherical surface, or a conical surface, distortion of an image can be corrected regardless of the shape of the light collecting unit 202 of the omnidirectional imaging unit 117. This has the effect.
In addition, there is an effect that an operation of adjusting a mismatch between the axis of the light collecting unit 202 and the axis of the imaging unit 203 is unnecessary. (5) In the above embodiment, the omnidirectional imaging unit 117 includes the imaging unit 203, the light collecting unit 202, and the support 200 made of a glass surface for fixing the light collecting unit 202 above the imaging unit 203. However, as shown in FIG. 32, the imaging unit 3213, the light collecting unit 3211, and the imaging unit 32
13 for fixing the light condensing part 3211 above the light source 13
212.

Here, the image pickup section 3213 and the condenser section 321
1 is equivalent to the image capturing unit 203 and the light collecting unit 202, respectively, and thus the description is omitted. The shape of the support 3212 is
This is equivalent to the cylindrical peripheral lattice 2701 shown in FIG. The diameter of the cylindrical surface of the support 3212 is
11, the upper part of the support 3212 is joined to the light collector 3211, and supports the light collector 3211 above the imaging part 3213.

With this configuration, as described in (4) above, it is possible to correct the distortion of the image regardless of the shape of the light-collecting unit, and it is possible to correct the axis of the light-collecting unit and the axis of the imaging unit. In addition to the effect that the operation of adjusting the mismatch is unnecessary, compared with the above embodiment, the incident light is not refracted by the support body 200 made of the glass surface as described in the above embodiment, Because it reaches the light condensing part 3211,
There is an effect that an image closer to a real image can be obtained. Further, there is an effect that the strength is more advantageous than the support made of a glass surface. (6) In the above-described embodiment, the imaging unit 203 is the focusing unit 2
02 and the surrounding area are imaged.
May image only the inside of the light collecting unit 202. In this case, since the area 305 shown in FIG. 3 is not imaged,
There is an effect that it is not necessary to replace the area 305 with a pixel value of 0. (7) In the above embodiment, the display screen 701 displays the video reception button 70 operated by the user when requesting the omnidirectional captured image transmission device 110 to transmit a new video.
3, the input unit 155 outputs a “video transmission request” to the transmission / reception unit 154 when the operation of the video reception button 703 is received by the user, and the transmission / reception unit 154 outputs “video transmission request” from the input unit 155. Upon receiving the “request”, the “video transmission request” and the receiving device identification number of the omnidirectional captured image receiving device 150 are transmitted to the transmitting / receiving unit 116.
The display screen 701 does not include the video reception button 703, and the input unit 155 does not accept the operation of the video reception button 703 by the user, and
May be configured to transmit the “video transmission request” and the receiving device identification number of the omnidirectional captured image receiving device 150 to the transmitting / receiving unit 116 at every predetermined time measured by the transmitting / receiving unit 154.

With this configuration, a new omnidirectional image is transmitted from the omnidirectional captured image transmitting device 110 to the omnidirectional captured image receiving device 150 every predetermined time measured by the transmitting / receiving section 154. In addition, there is an effect that the user does not need to perform the operation of requesting the transmission of the video. (8) The present invention is an omnidirectional captured image transmitting method and an omnidirectional captured image receiving method shown by the flowcharts in FIGS. Further, the present invention is a computer-readable recording medium including an omnidirectional captured image transmitting program and an omnidirectional captured image receiving program for causing a computer to execute the omnidirectional captured image transmitting method and the omnidirectional captured image receiving method. Further, the present invention includes a mode in which the omnidirectional captured image transmission program and the omnidirectional captured image receiving program are transmitted via a communication line.

[0129]

As described above, according to the present invention, an omnidirectional image is received from a transmitting apparatus for transmitting an omnidirectional image obtained by capturing the surrounding scenery, and a partial image of the received omnidirectional image is received. A transmission request transmitting means for transmitting a transmission request for an omnidirectional image to the transmitting device in accordance with a user operation, and an omnidirectional image for receiving an omnidirectional image from the transmitting device to which the transmission request has been transmitted. Receiving means, a fan-shaped area receiving means for receiving designation of a fan-shaped area corresponding to a predetermined viewing angle on the received omnidirectional image, and cutting out an image corresponding to the fan-shaped area from the omnidirectional image, and converting the image into a rectangular image It is characterized by comprising a first rectangular image conversion means for converting and a first display means for displaying the rectangular image.

According to this configuration, an omnidirectional image obtained by imaging a 360-degree surrounding landscape is received, converted into a rectangular image, and displayed. As described above, an image corresponding to a desired viewing angle of a plurality of users is cut out from an image in which a 360-degree surrounding landscape is projected, and distortion of a video is removed, and the image can be provided to each user. There is an effect that it is possible to receive and view images in different directions at the same time by using one camera as if the user were operating the cameras assigned to each of them freely.

Further, the present invention relates to a transmitting apparatus for transmitting an omnidirectional image obtained by capturing the surrounding scenery, a receiving apparatus for receiving an omnidirectional image and displaying a part of the received omnidirectional image. An omnidirectional captured image transmission system, comprising: a transmitting mirror, and a reflecting mirror formed in a convex shape, and is installed to face the reflecting mirror, and captures an image of a surrounding landscape through the reflecting mirror. An omnidirectional image pickup means comprising an image pickup unit for obtaining an omnidirectional image, an omnidirectional image storage means for storing the taken omnidirectional image, and a transmission request for an omnidirectional image transmitted from the receiving device. A transmission request receiving unit that reads out the omnidirectional image from the omnidirectional image storage unit when receiving the transmission request, and transmits the read omnidirectional image to the receiving device. Is a user operation Therefore, transmission request transmitting means for transmitting a transmission request for an omnidirectional image to the transmitting device, omnidirectional image receiving means for receiving an omnidirectional image from the transmitting device to which the transmission request has been transmitted, and A fan-shaped area receiving means for receiving designation of a fan-shaped area corresponding to the predetermined viewing angle, a first rectangular image converting means for cutting out an image corresponding to the fan-shaped area from an omnidirectional image, and converting the image into a rectangular image, And a first display unit for displaying a rectangular image.

As described above, a transmitting apparatus for transmitting an omnidirectional image obtained by imaging a 360-degree surrounding landscape and a receiving apparatus for receiving the transmitted omnidirectional image, converting the omnidirectional image into a rectangular image, and displaying the rectangular image. Be composed. Multiple images can be cut out from an image in which a 360-degree view of the surroundings is projected, according to the viewing angles desired by a plurality of users, and the distortion of the image can be removed and provided to each user. There is an effect that it is possible to receive and view images in different directions at the same time by using one camera as if the cameras assigned to each were operated freely.

Here, the imaging unit reflects the surrounding scenery and the surrounding area of the reflector via the reflector, and obtains an omnidirectional image, and digitally converts the captured omnidirectional image. A digital image conversion unit that converts the image into an image and obtains a digital omnidirectional image;
A pixel value replacement unit that replaces the image with a value, and a compression code that obtains a compression-coded digital omnidirectional image by compression-coding a digital omnidirectional image in which image parts other than the image part in which the scenery is displayed are replaced with 0 values. The omnidirectional image receiving means receives a compression-encoded digital omnidirectional image from the transmitting device to which the transmission request has been transmitted, and decompresses the received compression-encoded digital omnidirectional image. May be configured.

According to this configuration, the image portion other than the surrounding scenery in the omnidirectional image is replaced with a zero value, the omnidirectional image is compression-coded, and the omnidirectional captured image is transmitted from the omnidirectional captured image transmitting device. Since the data is transmitted to the device, the time required to transmit the omnidirectional image can be reduced. here,
The imaging unit reflects the surrounding scenery and the periphery of the reflector via the reflector, and converts the captured omnidirectional image into a digital image, and an omnidirectional image capturing unit that obtains an omnidirectional image.
A digital image conversion unit that obtains a digital omnidirectional image; and a landscape extraction unit that extracts, from the obtained digital omnidirectional image, an image portion in which a landscape is displayed, wherein the omnidirectional image receiving unit performs the transmission request. May be configured to receive the image portion showing the landscape from the transmitting device to which the image has been transmitted, and to relocate the image portion showing the landscape to the original position of the digital omnidirectional image.

According to this configuration, only the image portion of the surrounding landscape in the omnidirectional image is transmitted from the omnidirectional captured image transmitting device to the omnidirectional captured image receiving device, so that the time required to transmit the omnidirectional image is reduced. There is an effect that can be. here,
The omnidirectional imaging means further has a maximum diameter of the reflection surface as a diameter, and a predetermined number of rods parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflection surface. A grid on a cylindrical peripheral surface composed of a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods is provided around the reflection surface and the imaging unit. The first rectangular image conversion means may be configured to convert a rectangular image for each area formed by the grid displayed on the image corresponding to the received fan-shaped area.

According to this configuration, even if the shape of the condensing portion is a hyperboloid, a spherical surface, or a conical surface, regardless of the shape,
There is an effect that distortion of an image can be corrected. In addition, there is an effect that an operation of adjusting a mismatch between the axis of the light collecting unit and the axis of the imaging unit is unnecessary. Here, the omnidirectional imaging means has a maximum diameter of the reflecting surface as a diameter, and a predetermined number of rods parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflecting surface. And a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods, and the upper ring of the cylindrical shape and the upper portion of the reflection surface are joined. A grid on a cylindrical peripheral surface that supports the reflection surface, wherein the first rectangular image conversion unit performs a rectangular process for each area formed by the grid shown in the image corresponding to the received fan-shaped area. You may comprise so that it may convert into an image.

According to this configuration, even if the shape of the condensing portion is a hyperboloid, a spherical surface, or a conical surface, regardless of the shape,
There is an effect that distortion of an image can be corrected. Further, in addition to the effect that the operation of adjusting the mismatch between the axis of the light collecting unit and the axis of the imaging unit is not required, the incident light reaches the light collecting unit without being refracted by the support made of a glass surface. Therefore, there is an effect that an image closer to a real image can be obtained. Also,
There is an effect that strength is more advantageous than a support made of a glass surface.

According to the present invention, there is provided a transmission method for capturing an image of a surrounding landscape, converting the obtained omnidirectional image into a rectangular image, and transmitting a part of the rectangular image to the receiving device in accordance with a transmission request of the receiving device. An omnidirectional device, comprising: a reflecting mirror formed in a convex shape; and an imaging unit installed opposite to the reflecting mirror and configured to capture an image of a surrounding landscape via the reflecting mirror and obtain an omnidirectional image. Imaging means; second rectangular image converting means for converting the obtained omnidirectional image into a rectangular image; rectangular image storing means for storing the rectangular image; and a predetermined viewing angle transmitted from the receiving device A rectangular image area receiving means for receiving a transmission request of the small rectangular image including the designation of the area of the small rectangular image to be transmitted, and, when the transmission request is received, from the rectangular image storage means to the area of the received small rectangular image. Cut out the corresponding image and Characterized in that it comprises a small rectangular image transmitting means for transmitting the image cut out to the receiving device.

According to this configuration, an omnidirectional image obtained by imaging a 360-degree surrounding landscape is converted into a rectangular image and transmitted to the receiving device. As described above, an image corresponding to a desired viewing angle of a plurality of users is cut out from an image in which a 360-degree surrounding landscape is projected, and distortion of a video is removed, and the image can be provided to each user. There is an effect that it is possible to receive and view images in different directions at the same time by using one camera as if the user were operating the cameras assigned to each of them freely.

Here, the omnidirectional imaging means further comprises:
The maximum diameter of the reflective surface is a diameter, a predetermined number of bars parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflective surface, so as to surround these bars. ,
A grid on a cylindrical peripheral surface composed of a predetermined number of rings provided at equal intervals in the axial direction is provided around the reflection surface and the imaging unit, and the second rectangular image conversion unit includes:
A configuration may be made such that a rectangular image is converted for each area formed by the grid displayed in the omnidirectional image.

According to this configuration, even if the shape of the condensing portion is a hyperboloid, a spherical surface, or a conical surface, regardless of the shape,
There is an effect that distortion of an image can be corrected. In addition, there is an effect that an operation of adjusting a mismatch between the axis of the light collecting unit and the axis of the imaging unit is unnecessary. Here, the omnidirectional imaging means has a maximum diameter of the reflecting surface as a diameter, and a predetermined number of rods parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflecting surface. And a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods, and the upper ring of the cylindrical shape and the upper portion of the reflection surface are joined. A grid on a cylindrical peripheral surface that supports the reflection surface, wherein the second rectangular image conversion means converts a rectangular image for each area formed by the grid displayed in the omnidirectional image. You may comprise.

According to this configuration, even if the shape of the condensing portion is a hyperboloid, a spherical surface, or a conical surface, regardless of the shape,
There is an effect that distortion of an image can be corrected. Further, in addition to the effect that the operation of adjusting the mismatch between the axis of the light collecting unit and the axis of the imaging unit is not required, the incident light reaches the light collecting unit without being refracted by the support made of a glass surface. Therefore, there is an effect that an image closer to a real image can be obtained. Also,
There is an effect that strength is more advantageous than a support made of a glass surface.

Here, the reflecting mirror may be configured to be formed from a hyperboloid consisting of a single leaf of a bilobal hyperboloid.
According to this configuration, there is an effect that a wide field of view on the side and under the imaging unit can be secured, and high resolution can be obtained. Further, the present invention, a transmitting device that captures the surrounding landscape, converts the obtained omnidirectional image into a rectangular image, and transmits a part of the rectangular image to the receiving device according to a transmission request of the receiving device, An omnidirectional captured image transmission system including a receiving device that receives a partial image of a rectangular image and displays the received rectangular image, wherein the transmitting device includes a reflecting mirror formed in a convex shape, An omnidirectional imaging means, which is provided opposite to the reflecting mirror and includes an imaging unit that captures the surrounding landscape via the reflecting mirror and obtains an omnidirectional image, and converts the obtained omnidirectional image into a rectangular image. Second rectangular image converting means for converting, rectangular image storing means for storing the rectangular image,
A rectangular image area receiving unit that receives a transmission request for a small rectangular image including a designation of a region of the small rectangular image transmitted from the receiving device, and, if the transmission request is received, from the rectangular image storage unit, A small rectangular image transmitting unit that cuts out an image corresponding to the region of the small rectangular image and transmits the cut image to the receiving device, wherein the receiving device specifies a region of the small rectangular image corresponding to a predetermined viewing angle. A rectangular image area transmitting means for transmitting a transmission request for a small rectangular image including
From the transmitting device that has transmitted the transmission request, a small rectangular image receiving unit that receives an image corresponding to the area of the small rectangular image,
And a second display means for displaying the received image.

As described above, the omnidirectional image obtained by imaging the 360-degree landscape is converted into a rectangular image, and the rectangular image is transmitted from the transmitting device to the receiving device, and from the receiving device for displaying the transmitted rectangular image. Be composed. As described above, an image corresponding to a desired viewing angle of a plurality of users is cut out from an image in which a 360-degree surrounding landscape is projected, and distortion of a video is removed, and the image can be provided to each user. There is an effect that it is possible to receive and view images in different directions at the same time by using one camera as if the user were operating the cameras assigned to each of them freely.

Here, the reflecting mirror may be configured to be formed from a hyperboloid composed of one leaf of a bilobal hyperboloid.
According to this configuration, there is an effect that a wide field of view on the side and under the imaging unit can be secured, and high resolution can be obtained. Here, the omnidirectional imaging means may further include a maximum diameter of the reflection surface as a diameter, and a predetermined number parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflection surface. Rods, and a grid on a cylindrical peripheral surface composed of a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods. The second rectangular image conversion means may be provided so as to convert the image into a rectangular image for each area formed by the grid displayed in the omnidirectional image.

According to this configuration, even if the shape of the condensing portion is a hyperboloid, a spherical surface, or a conical surface, regardless of the shape,
There is an effect that distortion of an image can be corrected. In addition, there is an effect that an operation of adjusting a mismatch between the axis of the light collecting unit and the axis of the imaging unit is unnecessary. Here, the omnidirectional imaging means has a maximum diameter of the reflecting surface as a diameter, and a predetermined number of rods parallel to the axis at equal intervals on a cylindrical peripheral surface around the axis of the reflecting surface. And a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods, and the upper ring of the cylindrical shape and the upper portion of the reflection surface are joined. A grid on a cylindrical peripheral surface that supports the reflection surface, wherein the second rectangular image conversion means converts a rectangular image for each area formed by the grid displayed in the omnidirectional image. You may comprise.

According to this configuration, even if the shape of the condensing portion is a hyperboloid, a spherical surface, or a conical surface, regardless of its shape,
There is an effect that distortion of an image can be corrected. Further, in addition to the effect that the operation of adjusting the mismatch between the axis of the light collecting unit and the axis of the imaging unit is not required, the incident light reaches the light collecting unit without being refracted by the support made of a glass surface. Therefore, there is an effect that an image closer to a real image can be obtained. Also,
There is an effect that strength is more advantageous than a support made of a glass surface.

Also, the present invention relates to a receiving apparatus for receiving an omnidirectional image and displaying a part of the received omnidirectional image from a transmitting apparatus for transmitting an omnidirectional image obtained by capturing the surrounding scenery. A receiving method used, comprising: a transmitting request transmitting step of transmitting a transmitting request of an omnidirectional image to the transmitting device according to a user operation; and an omnidirectional image receiving receiving an omnidirectional image from the transmitting device to which the transmitting request has been transmitted. Step, a fan-shaped area receiving step of receiving a designation of a fan-shaped area corresponding to a predetermined viewing angle on the received omnidirectional image,
An image corresponding to the fan-shaped region is cut out from the omnidirectional image, and a first rectangular image converting step of converting the image into a rectangular image and a first displaying step of displaying the rectangular image are provided.

As described above, this method is used in a receiving apparatus that receives an omnidirectional image and displays the received omnidirectional image from a transmitting apparatus that transmits an omnidirectional image obtained by capturing a 360-degree landscape. Then, the omnidirectional image obtained by imaging the 360-degree surrounding landscape is received, converted into a rectangular image, and displayed. Multiple images can be cut out from an image in which a 360-degree view of the surroundings is projected, according to the viewing angles desired by a plurality of users, and the distortion of the image can be removed and provided to each user. There is an effect that it is possible to receive and view images in different directions at the same time by using one camera as if the cameras assigned to each were operated freely.

Further, according to the present invention, the surrounding landscape is imaged, the obtained omnidirectional image is converted into a rectangular image, and the rectangular image is transmitted to the receiving device in accordance with the transmission request of the receiving device. Omnidirectional imaging means including a reflector and an imaging unit which is installed opposite to the reflector and captures the surrounding landscape via the reflector to obtain an omnidirectional image, and a rectangular image holding means for holding a rectangular image A second rectangular image conversion step of converting the captured omnidirectional image into a rectangular image, and a rectangle storing the rectangular image in the rectangular image holding means. An image storing step, a rectangular image area receiving step of receiving a transmission request of a small rectangular image including an area designation of a small rectangular image corresponding to a predetermined viewing angle transmitted from the receiving device, and a case where the transmission request is received. , From serial rectangular image holding means, cutting out the image corresponding to the area of the small rectangular image thus received, characterized in that it comprises a small rectangular image transmission step of transmitting the image cut out to the receiving device.

As described above, this method is used in a transmitting device that converts an omnidirectional image obtained by capturing a 360-degree surrounding landscape into a rectangular image and transmits the rectangular image to the receiving device according to a transmission request from the receiving device. And an image corresponding to the viewing angle desired by a plurality of users is cut out from the image in which the 360-degree surrounding landscape is projected, and the distortion of the video is removed, and the image can be provided to each user. There is an effect that it is possible to receive and view images in different directions at the same time by using one camera as if the user were operating the cameras assigned to them.

Further, the present invention is a computer-readable recording medium on which the above-described receiving program or transmitting program is recorded. Obviously, the same effects as those of the device can be obtained.

[Brief description of the drawings]

FIG. 1 shows a block diagram of an omnidirectional captured image transmission system as one embodiment according to the present invention.

FIG. 2 is a perspective view of an omnidirectional imaging unit.

FIG. 3 illustrates an example of an image captured by an omnidirectional imaging unit.

FIG. 4 is a diagram showing a hyperboloid.

FIG. 5 illustrates a positional relationship between a light-collecting unit and an imaging unit of the omnidirectional imaging unit.

FIG. 6 shows information stored in an information storage unit 114.

FIG. 7 illustrates an example of an image converted into a digital signal by an omnidirectional imaging unit.

FIG. 8 shows an example of a display screen displayed on the display unit 156.

FIG. 9 shows an example of a display screen displayed by the display unit 156.

FIG. 10 is a diagram illustrating a display portion on a digital signalized video.

FIG. 11 shows information stored in an information storage unit 152.

FIG. 12 shows a trajectory of a light beam incident on the omnidirectional imaging unit.

FIG. 13 shows a trajectory of a light beam incident on the omnidirectional imaging unit.

FIG. 14 illustrates a trajectory of a light beam incident on the omnidirectional imaging unit.

FIG. 15 shows positions of pixels for image conversion on a digital signalized video.

FIG. 16 shows a method of calculating luminance of a pixel to be image-converted on a digital signalized video.

FIG. 17 shows a relationship between a fan-shaped area for image conversion on a digital signalized video and a rectangular area for image conversion.

FIG. 18 is a flowchart showing the operation of the omnidirectional captured image transmission device shown in FIG.

19 is a flowchart showing the operation of the omnidirectional captured image receiving device shown in FIG.

FIG. 20 is a flowchart showing a continuation of FIG. 19;

FIG. 21 is a flowchart illustrating an operation of a video receiving process of the omnidirectional captured image receiving device illustrated in FIG. 1;

FIG. 22 is a flowchart illustrating an operation of an omnidirectional display process of the omnidirectional captured image receiving device illustrated in FIG. 1;

23 is a flowchart showing an operation of a partial video display process of the omnidirectional captured image receiving apparatus shown in FIG.

24 is a flowchart showing an operation of a partial video generation process of the omnidirectional captured image receiving device shown in FIG.

FIG. 25 is a flowchart illustrating an operation of an upward movement process of the omnidirectional captured image receiving apparatus illustrated in FIG. 1;

FIG. 26 is a flowchart showing an operation of a leftward movement process of the omnidirectional captured image receiving device shown in FIG. 1;

FIG. 27 is a flowchart showing an operation of a rightward movement process of the omnidirectional captured image receiving apparatus shown in FIG. 1;

FIG. 28 is a flowchart illustrating an operation of a downward movement process of the omnidirectional captured image receiving device illustrated in FIG. 1;

FIG. 29 is a flowchart showing an operation of an enlarged display process of the omnidirectional captured image receiving device shown in FIG. 1;

FIG. 30 is a flowchart showing an operation of a reduced display process of the omnidirectional captured image receiving apparatus shown in FIG. 1;

FIG. 31 (a) is a perspective view showing a cylindrical peripheral lattice. (B) It is a perspective view which shows the omnidirectional imaging part and the grating | lattice of a cylindrical peripheral surface. (C) It is an example of the video imaged by the omnidirectional imaging unit shown in FIG.

FIG. 32 is a perspective view of another omnidirectional imaging section.

[Explanation of symbols]

 Reference Signs List 100 omnidirectional captured image transmission system 110 omnidirectional captured image transmission device 113 A / D conversion section 114 information storage section 115 image processing section 116 transmission / reception section 117 omnidirectional imaging section 118 image compression section 119 input section 120 display section 150 omnidirectional imaging Image receiving device 151 Image decompression unit 152 Information storage unit 153 Image conversion unit 154 Transmission / reception unit 155 Input unit 156 Display unit 160 Communication line

Claims (17)

[Claims] 1. A receiving apparatus for receiving an omnidirectional image from a transmitting apparatus for transmitting an omnidirectional image obtained by capturing an image of a surrounding landscape and displaying a part of the received omnidirectional image, comprising: Transmission request transmitting means for transmitting a transmission request for an omnidirectional image to the transmitting device in accordance with an operation; omnidirectional image receiving means for receiving an omnidirectional image from the transmitting device to which the transmission request has been transmitted; Fan-shaped area receiving means for receiving designation of a fan-shaped area corresponding to the above predetermined viewing angle; first rectangular image converting means for cutting out an image corresponding to the fan-shaped area from an omnidirectional image and converting the image into a rectangular image; A first display unit that displays the rectangular image. 2. A transmitting apparatus for transmitting an omnidirectional image obtained by capturing an image of a surrounding landscape, and a receiving apparatus for receiving an omnidirectional image and displaying a part of the received omnidirectional image. An omnidirectional captured image transmission system, wherein the transmitting device includes: a reflecting mirror formed in a convex shape; and a surrounding mirror imaged through the reflecting mirror, which is installed to face the reflecting mirror, and forms an omnidirectional image. An omnidirectional image pickup unit configured to obtain an omnidirectional image, an omnidirectional image storage unit that stores the captured omnidirectional image, and a transmission request reception that receives a transmission request for an omnidirectional image transmitted from the receiving device. Means for reading an omnidirectional image from the omnidirectional image storage means when the transmission request is received, and image transmitting means for transmitting the read omnidirectional image to the receiving apparatus. According to the whole Transmission request transmitting means for transmitting a transmission request of the position image to the transmitting device; omnidirectional image receiving means for receiving an omnidirectional image from the transmitting device to which the transmission request has been transmitted; Fan-shaped area receiving means for receiving designation of a fan-shaped area corresponding to a viewing angle; first rectangular image converting means for cutting out an image corresponding to the fan-shaped area from an omnidirectional image and converting the image into a rectangular image; Omnidirectional captured image transmission system, comprising: a first display means for displaying the image. 3. An omnidirectional image capturing unit that obtains an omnidirectional image by reflecting the surrounding scenery and a peripheral portion of the reflecting mirror via the reflecting mirror, and a digital image forming unit that converts the captured omnidirectional image into a digital image. A digital image conversion unit that obtains a digital omnidirectional image, and a pixel value replacement unit that replaces an image portion of the omnidirectional image converted into a digital image other than an image portion where a landscape is captured with 0 values, A compression encoding unit that compresses and encodes a digital omnidirectional image in which image parts other than the image part in which the scenery is displayed are replaced with 0 values, and obtains a compression-encoded digital omnidirectional image; The image receiving means receives a compression-encoded digital omnidirectional image from the transmitting device to which the transmission request has been transmitted, and decompresses the received compression-encoded digital omnidirectional image. Omnidirectional imaging an image transmission system of the mounting. 4. An omnidirectional image capturing unit that obtains an omnidirectional image by reflecting the surrounding scenery and the peripheral portion of the reflecting mirror via the reflecting mirror, and a digital image that captures the omnidirectional image. A digital image conversion unit that obtains a digital omnidirectional image, and a landscape extraction unit that extracts an image portion in which a landscape is displayed, from the obtained digital omnidirectional image. Receiving an image portion in which the landscape is projected from the transmitting device to which the transmission request is transmitted, and relocating the image portion in which the landscape is projected to an original position of the digital omnidirectional image. An omnidirectional captured image transmission system according to claim 1. 5. The omnidirectional imaging means further has a maximum diameter of the reflecting surface as a diameter, and is equally spaced from and parallel to the cylindrical peripheral surface around the axis of the reflecting surface. A grid on a cylindrical peripheral surface composed of a predetermined number of rods and a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods is formed on the reflecting surface and the imaging surface. Wherein the first rectangular image conversion means converts the image into a rectangular image for each area formed by the grid displayed in the image corresponding to the received fan-shaped area. Item 3. An omnidirectional captured image transmission system according to item 2. 6. The omnidirectional imaging means has a maximum diameter of the reflecting surface as a diameter, and a predetermined number parallel to the axis at equal intervals on a cylindrical peripheral surface with the axis of the reflecting surface as an axis. And a predetermined number of rings provided at equal intervals in the axial direction so as to surround these bars,
The upper part of the cylindrical shape and the upper part of the reflective surface are joined, and a grid on a cylindrical peripheral surface that supports the reflective surface is provided. The first rectangular image conversion unit includes: 3. The omnidirectional captured image transmission system according to claim 2, wherein a rectangular image is converted for each area formed by the grid displayed on the corresponding image. 7. A transmitting device that captures a surrounding landscape, converts the obtained omnidirectional image into a rectangular image, and transmits a part of the rectangular image to the receiving device in accordance with a transmission request of the receiving device. , An omnidirectional imaging means comprising a reflecting mirror formed in a convex shape, and an imaging unit which is installed to face the reflecting mirror and obtains an omnidirectional image by imaging the surrounding landscape via the reflecting mirror; Second rectangular image converting means for converting the obtained omnidirectional image into a rectangular image; rectangular image storing means for storing the rectangular image; and a small rectangular image corresponding to a predetermined viewing angle transmitted from the receiving device. A rectangular image area receiving means for receiving a transmission request of a small rectangular image including the designation of the area, and, when the transmission request is received, an image corresponding to the area of the received small rectangular image from the rectangular image storage means. Cut out to the receiving device Ri out transmission apparatus, characterized in that it comprises a small rectangular image transmission means for transmitting the image. 8. The omnidirectional imaging means further has a maximum diameter of the reflecting surface as a diameter, and is equally spaced and parallel to the cylindrical peripheral surface around the axis of the reflecting surface. A grid on a cylindrical peripheral surface composed of a predetermined number of rods and a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods is formed on the reflecting surface and the imaging surface. 8. The transmission according to claim 7, wherein the second rectangular image conversion unit converts the image into a rectangular image for each area formed by the grid displayed in the omnidirectional image. 9. apparatus. 9. The omnidirectional imaging means has a maximum diameter of the reflective surface as a diameter, and a predetermined number of parallel parallel to the axis at equal intervals on a cylindrical peripheral surface with the axis of the reflective surface as an axis. And a predetermined number of rings provided at equal intervals in the axial direction so as to surround these bars,
The ring on the cylindrical upper part and the upper part of the reflective surface are joined, and a grid on a cylindrical peripheral surface supporting the reflective surface is provided. The second rectangular image conversion unit is displayed on the omnidirectional image. 8. The transmitting apparatus according to claim 7, wherein each of the regions formed by the lattice is converted into a rectangular image. 10. The transmitting apparatus according to claim 7, wherein the reflecting mirror is formed by a hyperboloid formed by a single leaf of a bilobal hyperboloid. 11. A transmitting device that captures a surrounding landscape, converts the obtained omnidirectional image into a rectangular image, and transmits a part of the rectangular image to the receiving device according to a transmitting request of the receiving device. An omnidirectional captured image transmission system, comprising: a receiving device that receives a partial image of an image and displays a received rectangular image, wherein the transmitting device includes: a reflecting mirror formed in a convex shape; An omnidirectional imaging means, which is provided opposite to a mirror, and includes an imaging unit that captures the surrounding landscape via the reflecting mirror to obtain an omnidirectional image; and converts the obtained omnidirectional image into a rectangular image. A rectangular image storage unit that stores the rectangular image, and a rectangular image area receiving unit that receives a small rectangular image transmission request including a designation of a small rectangular image region transmitted from the receiving device. Means for receiving the transmission request When receiving, from the rectangular image storage means, cut out an image corresponding to the area of the received small rectangular image, and comprises a small rectangular image transmitting means for transmitting the clipped image to the receiving device, the receiving device, A rectangular image area transmitting means for transmitting a transmission request for a small rectangular image including designation of an area of the small rectangular image corresponding to the predetermined viewing angle; An omnidirectional captured image transmission system, comprising: a small rectangular image receiving means for receiving an image; and a second display means for displaying the received image. 12. The omnidirectional captured image transmission system according to claim 2, wherein the reflecting mirror is formed of a hyperboloid formed by a single leaf of a bilobal hyperboloid. 13. The omnidirectional imaging means further has a maximum diameter of the reflecting surface as a diameter, and is equally spaced and parallel to the cylindrical peripheral surface around the axis of the reflecting surface. A grid on a cylindrical peripheral surface composed of a predetermined number of rods and a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods is formed on the reflecting surface and the imaging surface. 12. The apparatus according to claim 11, wherein the second rectangular image conversion unit converts the image into a rectangular image for each area formed by the grid displayed in the omnidirectional image. Azimuth captured image transmission system. 14. The omnidirectional imaging means, wherein a maximum diameter of the reflection surface is a diameter, and a predetermined number of parallel azimuth parallel to the axis are equally spaced around a cylindrical peripheral surface having the axis of the reflection surface as an axis. And a predetermined number of rings provided at equal intervals in the axial direction so as to surround these rods, and the ring at the upper part of the cylindrical shape and the upper part of the reflection surface are A grid on a cylindrical peripheral surface that supports the reflection surface and is joined, and the second rectangular image conversion unit converts a rectangular image for each area formed by the grid displayed in the omnidirectional image. The omnidirectional captured image transmission system according to claim 11, wherein: 15. A receiving method used in a receiving apparatus that receives an omnidirectional image from a transmitting apparatus that transmits an omnidirectional image obtained by capturing an image of a surrounding landscape and displays a part of the received omnidirectional image. A transmission request transmitting step of transmitting an omnidirectional image transmission request to the transmitting device according to a user operation; an omnidirectional image receiving step of receiving an omnidirectional image from the transmitting device to which the transmission request has been transmitted; A fan-shaped area receiving step of receiving a designation of a fan-shaped area corresponding to a predetermined viewing angle on the received omnidirectional image; A receiving method, comprising: an image conversion step; and a first display step of displaying the rectangular image. 16. An image of a surrounding scene is taken, an obtained omnidirectional image is converted into a rectangular image, and the rectangular image is transmitted to a receiving device according to a transmission request of the receiving device. An omnidirectional imaging unit, which is provided to face the reflecting mirror and captures an image of the surrounding landscape via the reflecting mirror to obtain an omnidirectional image; and a rectangular image holding unit that holds a rectangular image. A transmission method used in a transmission device, wherein the captured omnidirectional image is converted into a rectangular image.
A rectangular image converting step; a rectangular image storing step of storing the rectangular image in the rectangular image holding means; and a small rectangular image including designation of an area of a small rectangular image corresponding to a predetermined viewing angle transmitted from the receiving device. A rectangular image area receiving step of receiving a transmission request, and when the transmission request is received, an image corresponding to the received small rectangular image area is cut out from the rectangular image holding unit, and the cut out image is sent to the receiving device. Transmitting a small rectangular image to be transmitted. 17. A computer-readable recording medium on which a program is recorded, the recording medium comprising a program for causing a computer to execute the method according to claim 15 or 16.