JP2006211105A - Image formation apparatus and system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To enable a user to perform the designation of a target local region easily in case of acquiring detailed image by enabling the setting of the central direction of a panoramic image shown in order to grasp the state of the whole surrounding and its dimension. <P>SOLUTION: An image formation apparatus includes all direction image acquisition parts 11 that are constituted including one or a plurality of cameras and acquire all the direction images FC as an ambient image, a panoramic image edge setting part 12 for setting the central direction TH and the angle of image GK to all the direction images FC acquired by the all direction image acquisition parts 11, and a panoramic image creation processor 13 which makes the central direction TH set by the panoramic image edge setting part 12 into the center position from all or the part of the all direction images FC acquired by the all direction image acquisition parts 11, and extracts the image of corresponding dimensions to the angle of the image GK to constitute the panoramic image PG. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image generation apparatus and system for generating a panoramic image based on surrounding images such as omnidirectional images.

  2. Description of the Related Art Conventionally, an image generating apparatus that generates a panoramic image based on a 360-degree omnidirectional image (an all-around image) is known. For example, there are those using a plurality of cameras, those using a fisheye camera, and those using an all-around camera. Such an image generation apparatus is used for the purpose of, for example, monitoring cameras and robot guidance.

  16 to 19 are diagrams for explaining a conventional image generating apparatus.

  FIG. 16 shows an image generation apparatus of a multiple shooting type by rotation of a single camera. That is, in FIG. 16A, one camera CM having an angle of view of 90 degrees or more is rotatably attached, and shooting is performed a plurality of times during one rotation of the camera CM. For example, as shown in FIGS. 16B to 16E, a total of four times of shooting are performed at t1, t2, t3, and t4, which are times when they are sequentially rotated 90 degrees during one rotation. Two images FD1 to FD4 are obtained. By connecting these images FD1 to FD4, one panoramic image is obtained.

  FIG. 17 shows a compound eye photographing type image generating apparatus using a plurality of cameras. That is, in FIG. 17A, four cameras CM1 to 4 similar to the camera CM are arranged so that the respective shooting directions sequentially have an azimuth difference of 90 degrees. With these four cameras CM1 to 4, four images FE1 to FE4 are simultaneously obtained as shown in FIGS. By connecting these images FE1 to FE4, one panoramic image is obtained.

  FIG. 18 shows a mirror photographing type image generating apparatus using a convex mirror. That is, in FIG. 18A, the camera CM5 is disposed below the convex mirror TK, and an image of the entire circumference reflected on the convex mirror TK is taken by the camera CM5. An omnidirectional image FF1 as shown in FIG. 18B is obtained from the camera CM5.

  FIG. 19 shows a super-wide-angle image type image generating apparatus. That is, in FIG. 19A, an image of the entire circumference is captured by the fisheye lens GL disposed in front of the camera CM6. An omnidirectional image FG1 as shown in FIG. 19B is obtained from the camera CM6.

  As described above, the conventional image generation apparatus generates a panoramic image from the omnidirectional images FD1 to FD4, FE1 to FF4, FF, and FG obtained by photographing. FIG. 20 shows an example of the panoramic image PG9. Based on the panoramic image PG9, for example, intruder detection and authentication are performed, and the moving direction and operation of the robot are determined.

  Usually, the image quality required to perform intruder authentication and detailed robot movement directions tends to be relatively high. However, there is a problem that the amount of data becomes too large for transferring and outputting. Therefore, in practice, the panorama image PG9 is created at a low resolution and presented to the user, and the panorama image PG9 allows the user to grasp the entire surrounding situation roughly in real time. In general, a separate detailed image is presented for a specified local region or a local region in which a target object is detected by image processing on the panoramic image PG9.

  For example, in Patent Document 1, a plurality of images obtained by the PTZ operation are connected to acquire a wide-angle image, and an arbitrary region is enlarged and displayed by photographing a position designated by the user on the wide-angle image by the PTZ operation. A system is disclosed.

Further, Patent Document 2 discloses a system for photographing in detail a region designated or automatically recognized on an image obtained from an omnidirectional camera with a zoom camera.
JP-A-11-8844 JP 2003-259349 A

  Conventionally, the panoramic image PG9 presented for the user to grasp the entire surroundings is created for a fixed position and direction.

  For example, in the case of Patent Document 1 described above, since the acquired wide-angle image itself is created in a fixed manner, the user's interest is concentrated near one end of the wide-angle image. In this case, it is difficult to see the image because the image is interrupted at the edges, and it is troublesome to specify the enlarged display itself.

  Further, even in the case of Patent Document 2, there is no consideration about the orientation of the developed image obtained from the omnidirectional camera, and there is a similar problem.

  That is, for example, when the user wants to determine whether or not the target object Q is an intruder while the panoramic image PG9 shown in FIG. 20 is displayed on the display device, the target object Q is the end of the panoramic image PG9. Therefore, it is difficult to visually recognize the target object Q, possibly overlooking the target object Q, and it is difficult to specify the position of the enlarged display.

  The present invention has been made in view of the above-described problems, and acquires a detailed image by making it possible to set the center direction and range of a panoramic image presented to grasp the entire surrounding situation. It is an object of the present invention to allow a user to easily specify a local area to be targeted.

  An apparatus according to the present invention includes a surrounding image acquisition unit configured to include one or a plurality of cameras to acquire a surrounding image, and a center direction and an angle of view with respect to the surrounding image acquired by the surrounding image acquisition unit. Panoramic image edge setting means for setting, and the central orientation set by the panoramic image edge setting means as a central position from all or part of the surrounding images acquired by the surrounding image acquisition means, and the angle of view Panorama image creation means for creating a panorama image by extracting an image in a range corresponding to.

  Preferably, the panoramic image edge setting means includes an operation input means that can be set manually by a user.

  In addition, the panoramic image edge setting unit performs image recognition on a subject in the image with respect to the surrounding image acquired by the surrounding image acquiring unit or an image constituting the surrounding image, thereby obtaining the center direction and Means for obtaining the angle of view;

  Further, the surrounding image acquisition means is attached to a movable body whose direction can be changed so that the direction of the surrounding image acquisition means itself can be changed, and the panoramic image edge setting means includes the moving body. Means for acquiring the central orientation and the angle of view based on information indicating the orientation of the image.

  Further, the surrounding image acquisition means includes a plurality of cameras, and the panoramic image creation means includes means for changing the arrangement of a plurality of images obtained from the plurality of cameras.

  Further, the surrounding image acquisition means includes a plurality of cameras, and the panoramic image creation means sets the order of the respective leading addresses when reading a plurality of images obtained from the plurality of cameras to each other. Including means for changing.

  According to another aspect, a surrounding image acquisition unit configured to include one or a plurality of cameras to acquire a surrounding image, and a panoramic image generation unit to generate a panoramic image from the surrounding image acquired by the surrounding image acquisition unit; Panorama image edge setting means for setting the center direction and angle of view of the surrounding image acquired by the surrounding image acquisition means; and the panorama image created by the panorama image creating means A panorama image conversion unit that converts the panorama image in a range corresponding to the angle of view with the center direction set by the setting unit as a center position, and a display that displays the panorama image converted by the panorama image conversion unit Means.

  The surrounding image acquisition means, the panorama image creation means, and the panorama image edge setting means are provided in a server, and the panorama image conversion means and the display means are provided in a client. To the client, the panorama image created by the panorama image creation unit and the information about the center azimuth and the angle of view set by the panorama image edge setting unit are transmitted.

  Note that the surrounding image may be an image of the entire circumference of 360 degrees, or may be an image of a part of the surroundings instead of the entire circumference of 360 degrees.

  According to the present invention, it is possible to set a center direction and a range of a panoramic image presented to grasp the entire surrounding situation, thereby specifying a local region to be a target when acquiring a detailed image. This can be easily performed by the user.

  FIG. 1 is a block diagram showing an image generation system 1 according to an embodiment of the present invention.

  In FIG. 1, the image generation system 1 includes a server 3 and a client 4. The server 3 includes an omnidirectional image acquisition unit 11, a panorama image edge setting unit 12, a panorama image creation processing unit 13, a transmission unit 14, an operation input unit 15, a display unit 16, and the like. The client 4 includes a receiving unit 21, a display unit 22, a printing unit 23, a recording unit 24, an operation input unit (not shown), and the like.

  The omnidirectional image acquisition unit 11 includes one or more cameras, and acquires a 360-degree omnidirectional image. As the omnidirectional image acquisition unit 11, various known structures can be used. For example, a multiple-shooting type with a single camera rotation shown in FIG. 16, a compound-eye shooting type with a plurality of cameras shown in FIG. 17, a mirror shooting type with a convex mirror shown in FIG. 18, and a super wide angle shown in FIG. A photographing type or the like can be used.

  Therefore, from the omnidirectional image acquisition unit 11, four images FD1 to FD1 to 4 as shown in FIGS. 16B to 16E and four images FE1 to FE1 to 4 as shown in FIGS. Alternatively, images FF1 and FG1 as shown in FIGS. 18B and 19B are output. Note that the images FE and FG may be still images or moving images.

  These images (image data) are referred to as “omnidirectional images FC”. That is, the omnidirectional image FC means the entire image information for constituting the omnidirectional image, and does not have to be completed as one omnidirectional image. For example, a fragmentary image for constituting the omnidirectional image. Or a set of

  In the present embodiment, the omnidirectional image FC, which is an image of 360 degrees around the circumference, is used as the surrounding image. May be used.

  The panoramic image edge setting unit 12 is for setting the center direction TH and the angle of view GK of the omnidirectional image FC acquired by the omnidirectional image acquisition unit 11. When setting the central orientation TH and the angle of view GK, the user can operate the operation input unit 15 and set it manually. The panoramic image edge setting unit 12 is provided with an image recognition processing unit 121 for an omnidirectional image (including images constituting the omnidirectional image) FC with respect to a subject (target object) in the image. It is also possible to automatically set the center direction TH and the angle of view GK by performing image recognition.

  Furthermore, as will be described later, the omnidirectional image acquisition unit 11 is attached to a movable body whose direction can be changed, such as a self-propelled robot, so that its direction can be changed. It is also possible to automatically set the center direction TH and the angle of view GK based on the information indicating the direction.

  In any case, the panoramic image edge setting unit 12 outputs information indicating the set center orientation TH and angle of view GK.

  The panoramic image creation processing unit 13 extracts, from all or a part of the omnidirectional image FC, an image in a range corresponding to the angle of view GK with the central direction TH set by the panoramic image edge setting unit 12 as a central position. A panoramic image PG is created. The panorama image creation processing unit 13 outputs image data of the created panorama image PG.

  The transmission unit 14 transmits data DT such as a panoramic image PG to the client 4. At the time of transmission, appropriate compression processing or the like is performed as necessary. It is also possible to perform encryption processing and the like.

  The operation input unit 15 can perform an input operation for setting the center direction TH and the angle of view GK with respect to the omnidirectional image FC by a user's manual operation. As the operation input unit 15, for example, a keyboard, a mouse, a joystick, a button for moving a cursor, or the like is used.

  The display unit 16 is configured using a liquid crystal panel, CRT, or other display device, and can display an omnidirectional image FC, other images, messages, or the like on a display surface. When the user manually sets the center direction TH and the angle of view GK, the user operates the operation input unit 15 while viewing the omnidirectional image FC displayed on the display unit 16.

  The receiving unit 21 receives data DT such as a panoramic image PG transmitted from the server 3. If the panoramic image PG has been compressed, the decompression is also performed. If encrypted, decryption processing is performed.

  The display unit 22 is configured using a liquid crystal panel, a CRT, and other display devices, and can display the received panoramic image PG, other images, messages, and the like on the display surface.

  The printing unit 23 prints the received panoramic image PG, other images, messages, and the like on a sheet. The recording unit 24 records the received panoramic image PG and the like on various recording media. As the recording medium, for example, a semiconductor memory, a magnetic disk, a CD-ROM, a flexible disk, or the like is used.

  As described above, in the image generation system 1 according to the present embodiment, the panorama image PG acquired and created by the server 3 is transmitted to the client 4, and the client 4 displays or records the panorama image PG.

  As the server 3 and the client 4, dedicated machines configured from various semiconductor devices and electronic devices and configured to realize their functions by executing appropriate programs by a CPU (Central Processing Unit) are used. It is also possible to use a personal computer in which an appropriate program is stored in either or both of them. The server 3 and the client 4 may be provided integrally close to each other, or may be provided separately from each other. When the server 3 and the client 4 are separated from each other, they can be connected by an appropriate cable, a dedicated line, a LAN, a WAN, or the Internet.

  Next, the function of the image generation system 1 will be described in more detail with reference to FIGS.

  2 is a diagram illustrating an example of the omnidirectional image FC1 acquired by the omnidirectional image acquisition unit 11, FIG. 3 is a diagram illustrating another example of the omnidirectional image FC2 acquired by the omnidirectional image acquisition unit 11, and FIG. Is a diagram showing another example of the omnidirectional image FC3 acquired by the omnidirectional image acquisition unit 11, FIG. 5 is a diagram showing examples of the panoramic images PG1 and 2, and FIG. 6 is a diagram showing another example of the panoramic image PG3. 7 and FIG. 7 are diagrams showing still another example panoramic image PG4, and FIG. 8 is a diagram showing an example of the local region enlarged image FB1.

  The omnidirectional images FC1 and FC2 shown in FIGS. 2 and 3 are based on the four images FA1 to FA4 obtained from the omnidirectional image acquisition unit 11 having four cameras. The omnidirectional images shown in FIG. FC3 is obtained from the omnidirectional image acquisition unit 11 having one camera of a mirror photographing type using a convex mirror.

  In the omnidirectional image FC1 shown in FIG. 2, the target object Q, which is a person located to the left of the center, is shown together with the other objects Q1 to Q3. However, in the omnidirectional image FC2 shown in FIG. 3, the target object Q moves and enters the end of the omnidirectional image FC2, and the target object Q is interrupted at the end and separated into both ends.

  For such omnidirectional images FC1 and FC2, the user can set (specify) the center direction TH and the angle of view GK by operating the operation input unit 15.

  That is, as shown in FIG. 2, the operation input unit 15 is operated and the panoramic image end setting unit 12 sets the center direction TH1 and the angle of view GK1. The panorama image creation processing unit 13 creates a panorama image PG1 shown in FIG. 5A from the omnidirectional image FC1 corresponding to the set center orientation TH1 and field angle GK1.

  In the panoramic image PG1 shown in FIG. 5A, the set center direction TH1 is the center position of the panoramic image PG1, and the set angle of view GK1 is the angle of view GK1 of the panoramic image PG1. As a result, the target object Q intended by the user is at the center position of the panoramic image PG1. Therefore, the user can easily observe the target object Q, and can easily determine whether or not the target object Q should be observed in more detail, so that it is easy to specify an enlarged display for the target object Q. .

  When the user operates the operation input unit in the client 4 and designates the enlarged display while the panoramic image PG1 shown in FIG. 5A is displayed, the local region enlargement shown in FIG. An image FB1 is displayed. With the local region enlarged image FB1, the user can observe the target object Q in more detail. This makes it possible to more accurately determine whether the target object Q is an intruder, for example.

  In FIG. 2, when the central orientation TH1 and the angle of view GK2 are set for the omnidirectional image FC1, a panoramic image PG2 shown in FIG. 5B is created corresponding to the angle of view GK2.

  Further, as shown in FIG. 3, the operation input unit 15 is operated, and the center image TH3 and the angle of view GK3 are set by the panorama image edge setting unit 12. The panoramic image creation processing unit 13 creates a panoramic image PG3 shown in FIG. 6 from the omnidirectional image FC2 corresponding to the set center orientation TH3 and angle of view GK3.

  Also in the panoramic image PG3 shown in FIG. 6, the set center direction TH3 is the center position of the panoramic image PG3, and the set angle of view GK3 is the angle of view GK3 of the panoramic image PG3. Thus, in the panoramic image PG3, the target object Q is at the center position even though the target object Q is close to the end in the omnidirectional image FC2. Therefore, as in the case of the panoramic image PG1 in the previous example, the user can easily observe the target object Q. In addition, it is possible to display the local area enlarged image FB1 on the display unit 22 by designating the enlarged display for the panoramic image PG3.

  In addition, the central orientation TH and the angle of view GK can be set for an omnidirectional image FC acquired from a mirror photographing type using a convex mirror as in the case of FIGS.

  That is, as shown in FIG. 4, when the center direction TH1 is set and the angle of view GK2 is set for the omnidirectional image FC3, a panoramic image PG2 shown in FIG. 5B is created.

  In the example described above, an example in which an arbitrary center orientation TH and angle of view GK can be set for the omnidirectional image FC has been described. However, in setting the center direction TH and the angle of view GK, it may be possible only within a limited range. For example, as shown in FIG. 17, it is possible to set the central orientation TH and the image only in units of images by each camera, for a type in which a plurality of images obtained by a plurality of cameras are connected to form an omnidirectional image FC. Good.

  That is, in FIG. 2, the center between the two images FA2 and FA3 is designated as the center orientation TH2, and a range including the two images FA2 and FA3 is set as the angle of view GK4. Then, the panoramic image creation processing unit 13 creates a panoramic image PG4 shown in FIG. 7 from the omnidirectional image FC1 corresponding to the set center orientation TH4 and angle of view GK4.

  In the panoramic image PG4 shown in FIG. 7, the set center direction TH4 is the center position, and the set angle of view GK4 is the angle of view GK4 of the panoramic image PG4. Thereby, although the target object Q is not the center position of the panoramic image PG1, the entire target object Q is captured without being divided. Therefore, observation of the target object Q is easy, and designation of enlarged display is also easy.

When designating the central orientation TH and the angle of view GK, it is only necessary to display the respective omnidirectional images FC on the display surface of the display unit 16 and, at the same time, arrows and symbols indicating the central orientation TH and the angle of view GK. , A center line, a dimension line, or the like may be displayed in place. Further, in order to show the angle of view GK, it may be highlighted by changing the density or color of the angle of view GK. In addition to these displays, the center orientation TH and the angle of view GK may be displayed numerically.
[Modification]
FIG. 9 is a block diagram showing a modification of the image generation system according to the present invention, and FIG. 10 is a block diagram showing another modification of the image generation system according to the present invention.

  9 and 10, the same reference numerals are given to portions having the same functions as those in FIG. 1, and description thereof is omitted or simplified.

  In FIG. 9, the image generation system 1B includes a server 3B and a client 4B. The server 3B includes an omnidirectional image acquisition unit 11, a panoramic image edge setting unit 12, a transmission unit 14, an operation input unit 15, a display unit 16, and the like. The client 4B includes a receiving unit 21, a panoramic image creation processing unit 26, a display unit 22, a printing unit 23, a recording unit 24, an operation input unit (not shown), and the like.

  That is, in the image generation system 1B shown in FIG. 9, the server 3B is not provided with the panoramic image creation processing unit 13, and instead, the client 4B is provided with the panoramic image creation processing unit 26. Therefore, in the server 3B, without creating the panoramic image PG, data such as the omnidirectional image FC acquired by the omnidirectional image acquiring unit 11 and the central azimuth TH and the angle of view GK set by the panoramic image end setting unit 12 DT is transmitted from the transmission unit 14 toward the client 4B.

  In the client 4B, the data DT transmitted from the server 3B is received by the receiving unit 21, and a panoramic image PG is created by the panoramic image creation processing unit 26 based on the received data DT. That is, the panoramic image creation processing unit 26 extracts an image in a range corresponding to the angle of view GK with the set center orientation TH as the center position from all or part of the omnidirectional image FC, and creates a panoramic image PG. To do. The created panorama image PG is displayed on the display unit 22.

  In FIG. 10, an image generation system 1C includes a server 3C and a client 4C. The configuration of the server 3C is almost the same as that of the server 3 in FIG. 1, but the panoramic image creation processing unit 13C uses a fixed center orientation and a fixed center orientation TH and field angle GK. A temporary panoramic image PGK is created using the angle of view. The created temporary panoramic image PGK and data DT such as the set center orientation TH and angle of view GK are transmitted to the client 4C.

  In the client 4C, the panorama image conversion unit 27 converts the temporary panorama image PGK transmitted from the server 3C based on the center direction TH and the angle of view GK transmitted in the same manner, and creates a new panorama image PG. . That is, the panorama image conversion unit 27 converts the panorama image PGK into a panorama image PG in the range corresponding to the transmitted angle of view GK with the transmitted center direction TH as the center position. It is also possible to create a panoramic image PG based on the center orientation TH and the angle of view GK set in the client 4C.

  In the embodiment and the modification as described above, the user does not set the center orientation TH and the angle of view GK, but an image recognition processing unit is provided in the server, and an omnidirectional image (an image constituting the omnidirectional image is displayed). The target object Q and other objects are detected by performing image recognition on the subject (target object) in the image, and the center direction TH and the angle of view GK are set according to the detected object. It may be set automatically.

  In addition, a sensor for detecting the target object Q is separately provided, and the azimuth, position, or size of the target object Q is detected by the sensor, and the center direction TH and the angle of view GK are automatically determined based on the detection result. You may make it set to. As such a sensor, for example, various known sensors such as an infrared sensor and an ultrasonic sensor can be used.

  Furthermore, the omnidirectional image acquisition unit 11 is attached to a movable body whose direction can be changed, such as a self-propelled robot, so that its own direction can be changed, and based on information indicating the direction of the movable body. Thus, it is possible to automatically set the center direction TH and the angle of view GK.

  Further, not only the center direction TH and the angle of view GK but also the size (size) of the image may be set, and the panoramic image PG may be created or changed according to the set size. For example, a target object or other object may be detected by performing image recognition on a subject in the image, and the size of the panoramic image may be changed according to the detected object or its attribute or number.

  Further, the panorama image PG may be displayed and recorded on the server. A system configuration such as stand-alone may be used.

  Next, an example of a method of changing the image at the center position in the panorama image creation processing units 13 and 26 or the panorama image conversion unit 27 will be described.

  FIG. 11 is a diagram for explaining an example in which the image at the center position of the panoramic image PG is changed in hardware, and FIG. 12 is a diagram for explaining an example in which the image at the center position is changed in software.

  In order to change the center position of the panorama image PG in hardware, for example, the correspondence between the output image from the camera CM and the input image in the panorama image creation processing unit 13 or the like is controlled by a port number or physical switch. To change.

  In FIG. 11A, four cameras CM1 to CM4 are attached to the outer peripheral surface of the rotating ring 71 every 90 degrees, and the outputs of the cameras CM1 to CM4 are provided on a fixed shaft 72, but are slip rings 73. Are connected to the image lines LN1 to LN4. In FIG. 11A, the outputs of the cameras CM1 to CM4 are connected to the image lines LN1 to LN4, respectively. However, as shown in FIG. 11B, after the rotation ring 71 is rotated 90 degrees rightward, the camera CM1 is the image line LN2, the camera CM2 is the image line LN3, the camera CM3 is the image line LN4, and the camera CM4 is connected to the image line LN1 and inputted as images FH1 to FH4, respectively. A panoramic image PG is created by sequentially connecting these images FH1 to FH4. When the rotation ring 71 rotates, that is, when the cameras CM1 to CM4 rotate, the correspondence relationship between the cameras CM1 to CM4 and the images FH1 to F4 input to the panorama image creation processing unit 13 changes. The content or arrangement is changed, and as a result, the image at the center position of the panoramic image PG is changed. In this case, the configuration including the slip ring 73 corresponds to “means for changing the arrangement of a plurality of images” of the present invention.

  In FIG. 12A, the outputs of the four cameras CM1 to CM4 are input to the buffers BF1 to BF4, respectively. That is, the relationship between the outputs of the cameras CM1 to CM4 and the buffers BF1 to BF4 is constant. However, the relationship between the buffers BF1 to BF4 and the images FH1 to FH4 is changed in software. That is, in FIG. 12A, the images input to the buffers BF1 to BF4 are used as the images FH1 to FH4, respectively. In FIG. 12B, the images input to the buffers BF1 to BF4 are These are used as images FH2, 3, 4, 1 respectively. Such a change can be made by controlling an address when images are read from the buffers BF1 to BF4. Thus, the image at the center position of the panoramic image PG can be changed in software without providing a hardware mechanism. In this case, the configuration for reading the image by controlling the addresses of the memory areas of the buffers BF1 to BF4 corresponds to the “means for changing the order of the top addresses to each other” of the present invention.

  Next, an example when the server 3 is mounted on a mobile object will be described.

  13 is a diagram showing an overview when the server 3 is mounted on the traveling robot 51, FIG. 14 is a diagram showing examples of the obtained panoramic images PG5 and 6, and FIG. 15 is an example of the obtained panoramic images PG7 and 8. FIG.

In FIG. 13, the traveling robot 51 includes a main body 52 and four tires 53 provided on the main body 52. The tire 53 is driven to rotate in the normal direction or the reverse direction, and the rotational speed is controlled, so that the tire 53 can self-run while being steered. An omnidirectional camera CMZ for acquiring an omnidirectional image FC and a main camera CMM for acquiring a detailed image are mounted on the upper part of the main body 52. These cameras CMZ and CMM are attached to the main body 52 by one of the following two attachment methods.
(1) The main camera CMM and the omnidirectional camera CMZ are integrated with each other so as to be rotatable. A type in which the front and traveling directions of the main body 52 and the directions of the main camera CMM and the omnidirectional camera CMZ are mechanically independent.
(2) The omnidirectional camera CMZ is integrated and attached to the main body 52, and the main camera CMM is rotatably attached to the same part. A type in which the orientation of the main camera CMM and the orientation of the omnidirectional camera CMZ are mechanically independent.

  There are two ways. FIG. 14 shows a case of the type (1), and FIG. 15 shows a case of the type (2).

  The traveling robot 51 is traveling on the road DR, and therefore, the front of the road DR is the front and the traveling direction. Assume that the main camera CMM views a track traveling on a road crossing ahead as the target object Q. The main camera CMM tracks the target object Q. A panoramic image PG5 shown in FIG. 14A is created with the direction of the main camera CMM as the center direction TH. After the elapse of time, the target object Q moves and is tracked by the main camera CMM, so that a panoramic image PG6 shown in FIG. 14B is obtained.

  In the panoramic image PG6 shown in FIG. 14B, the target object Q is easy to see at the center position, but the front road DR is further to the right, making it difficult to see. That is, in this case, since the traveling direction or the front is not always set as the center position of the panoramic images PG5 and PG6, it is not possible to deal with the case where it is desired to gaze at the front road DR.

  Therefore, in the present embodiment, as described above, the center direction TH and the angle of view GK are arbitrarily set by various methods. For example, by operating the operation input unit 15, the direction of the road DR is set as the center direction TH. When it is desired to gaze at the target object Q, the target object Q is set as the center direction TH.

  Further, the traveling direction of the traveling robot 51 is automatically set as the center direction TH. The target object Q is detected by image recognition or the like, and the direction of the target object Q is set as the center direction TH. If necessary, the setting method of the center direction TH is switched.

  A panoramic image PG7 shown in FIG. 15A is created with the traveling direction of the traveling robot 51 as the center direction TH. After the time has elapsed, the target object Q has moved, so that a panoramic image PG8 shown in FIG. 15B is obtained.

  In the panoramic image PG8 shown in FIG. 15B, the road DR is at the center position and is easy to see, but the target object Q is further left and reaches the end, making it difficult to see. That is, in this case, the target object Q to be tracked by the main camera CMM cannot be brought to the center position of the panoramic image PG. Since the position of the attention area of the main camera CMM changes in the panoramic images PG7 and 8, the vicinity of the attention area is difficult to see.

  Therefore, in the present embodiment, as described above, the center direction TH and the angle of view GK are arbitrarily set by various methods. For example, the direction of the target object Q or the road DR is set as the center direction TH by operating the operation input unit 15.

  Similarly to the above, the traveling direction of the traveling robot 51 is set as the center direction TH, or the direction of the target object Q is set as the center direction TH by image recognition or the like.

  In this way, by making it possible to arbitrarily set the center position of the panoramic image PG, an easy-to-view image suitable for the purpose can be obtained. In addition, even when the image is enlarged, it is possible to prevent the attention area from approaching the edge or being out of the field of view.

  In the above-described embodiment, as the omnidirectional image acquisition unit 11, for example, an omnidirectional type that takes an image of 360 degrees in the horizontal direction and takes an image of the entire circumference in all directions may be used. Is possible.

  In addition, the omnidirectional image acquisition unit 11, the panorama image edge setting unit 12, the panorama image creation processing unit 13, the server, the client, the image generation system, etc., or the structure, configuration, shape, dimensions, number, image contents, etc. of each part Can be appropriately changed in accordance with the gist of the present invention.

  The present invention can be used as an image generation device used for a search robot or a rescue robot.

1 is a block diagram illustrating an image generation system according to an embodiment of the present invention. It is a figure which shows the example of the omnidirectional image acquired by the omnidirectional image acquisition part. It is a figure which shows the omnidirectional image of the other example acquired by the omnidirectional image acquisition part. It is a figure which shows the omnidirectional image of the other example acquired by the omnidirectional image acquisition part. It is a figure which shows the example of a panoramic image. It is a figure which shows the panoramic image of another example. It is a figure which shows the panoramic image of another example. It is a figure which shows the example of a local region enlarged image. It is a block diagram which shows the modification of the image generation system which concerns on this invention. It is a block diagram which shows the other modification of the image generation system which concerns on this invention. It is a figure explaining the example in the case of changing the image of a center position by hardware. It is a figure explaining the example in the case of changing the image of a center position by software. It is a figure which shows the external view at the time of mounting a server in a traveling robot. It is a figure which shows the example of the obtained panoramic image. It is a figure which shows the example of the obtained panoramic image. It is a figure which shows the image production | generation apparatus of the multiple imaging | photography type by rotation of a single camera. It is a figure which shows the image production | generation apparatus of the compound eye imaging | photography type by a several camera. It is a figure which shows the image production | generation apparatus of the mirror imaging | photography type by a convex mirror. It is a figure which shows the image production | generation apparatus of a super wide angle imaging | photography type. It is a figure which shows the example of a panoramic image.

Explanation of symbols

1,1B, 1C Image generation system (image generation device)
3, 3B, 3C server 4, 4B, 4C client 11 Omnidirectional image acquisition unit (ambient image acquisition means)
12 panoramic image edge setting section (panoramic image edge setting means)
13 Panorama image creation processing unit (panorama image creation means)
15 Operation input section (operation input means)
22 Display section (display means)
26 Panorama image creation processing unit (panorama image creation means)
27 Panorama image conversion unit (panorama image conversion means)
51 Traveling robot (moving body)
121 Image Recognition Processing Unit CM Camera TH Center Direction GK Angle of View FC Omnidirectional Image (Ambient Image)
PG panorama image

Claims (8)

Ambient image acquisition means configured to include one or a plurality of cameras to acquire an ambient image;
Panoramic image edge setting means for setting the center direction and angle of view of the surrounding image acquired by the surrounding image acquisition means;
A panorama is extracted from all or a part of the surrounding images acquired by the surrounding image acquisition unit by extracting an image in a range corresponding to the angle of view and having the central orientation set by the panoramic image edge setting unit as a central position. Panoramic image creation means for creating an image;
An image generation apparatus comprising: The panoramic image edge setting means includes an operation input means that can be set by a user manually.
The image generation apparatus according to claim 1. The panoramic image edge setting means
Means for acquiring the central orientation and the angle of view by performing image recognition on a subject in the image with respect to the surrounding image acquired by the surrounding image acquiring unit or an image constituting the surrounding image;
The image generation apparatus according to claim 1. The surrounding image acquisition means is attached to a movable body whose orientation can be changed so that the orientation of the surrounding image acquisition means itself can be changed,
The panoramic image edge setting means includes means for acquiring the center direction and the angle of view based on information indicating the direction of the moving body.
The image generation apparatus according to claim 1. The surrounding image acquisition means includes a plurality of cameras,
The panoramic image creation means includes means for changing the arrangement of a plurality of images obtained from the plurality of cameras.
The image generation apparatus according to claim 1. The surrounding image acquisition means includes a plurality of cameras,
The panoramic image creation means includes means for mutually changing the order of the respective leading addresses when reading a plurality of images obtained from the plurality of cameras.
The image generation apparatus according to claim 1. Ambient image acquisition means configured to include one or a plurality of cameras to acquire an ambient image;
Panorama image creation means for creating a panorama image from the surrounding image acquired by the surrounding image acquisition means;
Panoramic image edge setting means for setting the center direction and angle of view of the surrounding image acquired by the surrounding image acquisition means;
Panorama image conversion means for converting the panorama image created by the panorama image creation means into a panorama image having a center position set by the panorama image edge setting means and a range corresponding to the angle of view;
Display means for displaying a panoramic image after being converted by the panoramic image converting means;
An image generation system comprising: The surrounding image acquisition means, the panorama image creation means, and the panorama image edge setting means are provided in a server,
The panoramic image conversion means and the display means are provided in a client,
The server is configured to transmit information about the panorama image created by the panorama image creation means and the center orientation and the angle of view set by the panorama image edge setting means to the client. Yes,
The image generation system according to claim 7.