JP2018157399A - Image formation program, image formation device and image formation method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image formation program that efficiently forms a panoramic image.SOLUTION: An image formation program causes a computer to perform processing to: extract a feature point included in a first image at a first position received from a camera; receive an image photographed while the camera is turned at a specific speed from the first position as a starting point; extract feature points included respectively in images photographed while the camera is turned; specify a second image having a feature point matching the image feature point of the first image among the feature points of the extracted images; measure the time up to photography of the second image after the start of turning of the camera at the first position as the starting point; calculate an angular speed in the turning of the camera based upon the measured time; specify intervals of time when images needed to create a panoramic image are photographed based upon the calculated angular speed; and transmit an instruction for photography to the camera at the specified intervals of time.SELECTED DRAWING: Figure 12

Description

  The present invention relates to an image forming program for forming one image from a plurality of images.

  A plurality of images taken while turning the camera are combined to form one image, a so-called panoramic image (see, for example, Patent Document 1).

JP 2001-8192 A

  In order to form a panoramic image from a plurality of original images, it is necessary to overlap image contents in a part of two adjacent original images. This is because the original images are joined together using the overlapping portion as a clue.

  On the other hand, surveillance cameras installed for the purpose of crime prevention and disaster prevention check images while turning the surveillance camera in order to monitor a wide range. However, the confirmation while turning the surveillance camera has time. Therefore, monitoring is facilitated by forming a panoramic image that covers the monitoring range. However, many images output from the surveillance camera include images that are unnecessary for forming a panoramic image. Therefore, it is necessary to extract an image necessary for forming a panoramic image, and the panoramic image cannot be formed efficiently.

  In one aspect, an object of the present invention is to provide an image forming program that efficiently forms a panoramic image.

  An image forming program disclosed in the present application is an image forming program that receives a plurality of images from a camera and forms a panoramic image from the received plurality of images. The image forming program is a first position at a first position received from the camera. Extract feature points included in the image, receive an image taken while turning the camera at a specific speed from the first position, and extract feature points included in each of the images taken while turning Then, among the extracted feature points of each of the images, a second image having a feature point that matches the feature point of the image included in the first image is specified, and the camera is started from the first position. Measuring the time from the start of turning until the second image is taken, calculating the angular velocity when turning the camera based on the measured time, Specifying a time interval for capturing images necessary for creating a panoramic image based on speed, and causing a computer to execute a process of transmitting a shooting instruction to the camera at each specified time interval. Features.

  According to an aspect of the present invention, a panoramic image can be efficiently formed from a plurality of original images.

It is a block diagram which shows the structural example of a panoramic image formation system. It is a sequence diagram which shows the process sequence example of a panoramic image forming system. It is a block diagram which shows an example of a structure of an image processing apparatus. It is explanatory drawing which shows an example of the record layout of camera DB. It is explanatory drawing which shows an example of the record layout of water level gauge DB. It is explanatory drawing which shows an example of the record layout of corresponding | compatible camera DB. It is explanatory drawing which shows an example of the record layout of attribute DB. It is explanatory drawing which shows the formation method of a panoramic image. It is explanatory drawing which shows the change of the imaging | photography range of a camera. It is a flowchart which shows an example of the procedure of a time interval calculation process. It is a flowchart which shows an example of the procedure of a starting process. It is a flowchart which shows an example of the procedure of an individual process. It is a flowchart which shows an example of the procedure of a monitoring process. It is explanatory drawing which shows the other example of the record layout of camera DB. It is a flowchart which shows an example of the procedure of an accumulation time calculation process. It is a flowchart which shows an example of the procedure of a mode determination process. It is a flowchart which shows the other example of the procedure of an individual process. It is a block diagram which shows an example of a function structure of an image processing apparatus.

  Hereinafter, embodiments will be described with reference to the drawings. In the following description, an image taken by a camera that monitors a river is taken as an example.

Embodiment 1
FIG. 1 is a block diagram illustrating a configuration example of a panoramic image forming system. The panorama image forming system 100 includes an image processing apparatus 1, cameras 2-1, 2-2,..., Water level meters 3-1, 3-2,. The image processing apparatus 1, the cameras 2-1, 2-2,..., The water level gauges 3-1, 3-2,... And the terminals 4-1, 4-2,. ing.

  The image processing apparatus 1 performs image processing for synthesizing one panoramic image from a plurality of images taken by the cameras 2-1, 2-2,. The image processing apparatus 1 includes a server computer, a blade server, or a PC (Personal Computer). The functions that the image processing apparatus 1 should have may be provided by a cloud service.

  Multiple cameras are installed according to the number of monitoring points. Multiple water level gauges are installed according to the number of monitoring points as well as cameras. For example, the camera 2-1 and the water level meter 3-1 are installed at the same monitoring location. The camera 2-2 and the water level gauge 3-2 are installed at the same monitoring location. The camera and the water level meter do not have to have a one-to-one correspondence. Note that the camera and the water level gauge do not necessarily have to be plural, and may each be one. The cameras 2-1, 2-2,... Have substantially the same function, and therefore will be described as a camera 2 in the following description. Similarly, the water level meter is described as a water level meter 3 and will be described collectively.

  Terminals 4-1, 4-2,... Are installed at each river office that monitors the river. Similarly to the camera 2 and the water level gauge 3, the terminals 4-1, 4-2,. A monitor is connected to the terminal 4. The terminal 4 displays the panoramic image output from the image processing apparatus 1 on the monitor. A camera control device for the camera 2 such as a joystick is connected to the terminal 4. The camera control device enables river staff to control the camera 2 manually. The camera control device may be configured to be able to control the camera 2 directly via the network N without interposing the terminal 4.

  The camera 2 can capture a moving image. The camera 2 is a camera having sensitivity to near infrared rays that can be photographed even at night, for example, a CCD (Charge Coupled Device) camera. The camera 2 is attached to an electric head. By remotely controlling the electric camera platform, the camera 2 can be panned and tilted from a remote location. The camera 2 is equipped with an electric zoom mechanism. By remotely controlling the electric zoom mechanism, the zoom magnification of the lens can be varied from a remote location. Furthermore, infrared illumination may be attached to the camera 2. By turning on the infrared illumination during night photography, a clearer image can be acquired. It is assumed that three speed modes of low speed, medium speed, and high speed are defined as the speed when the electric pan head pans. In each speed mode, the camera 2 pans at a constant angular speed. Further, the turning described below means panning (horizontal rotation). That is, “low-speed turning” described below means that the camera 2 pans in the low-speed mode.

  The water level gauge 3 measures the water level of the river. The water level meter 3 repeatedly measures the water level. The water level meter 3 transmits the measured water level to the image processing apparatus 1 via the network N. In addition, the water level meter 3 stores a water level that serves as a reference for issuing an alarm or the like. The water level meter 3 compares the measured water level with the stored water level, and transmits a trigger signal that prompts the image processing apparatus 1 to perform processing. The standard water level is, for example, a flood control standby level, flood warning level, evacuation judgment level, and flood risk level. The water team standby water level is a reference water level for the water team to wait. Inundation caution water level is a water level that warns about the occurrence of river inundation. The evacuation judgment water level is a water level that serves as a guide for evacuation information announcement. The inundation risk level is the level at which the river may overflow.

  FIG. 2 is a sequence diagram illustrating a processing sequence example of the panoramic image forming system 100. The water level meter 3 repeatedly measures the water level. The water level meter 3 determines whether or not the measured water level is equal to or higher than a predetermined water level determined in advance (for example, a water defense standby water level). When the water level meter 3 determines that the specified water level has been exceeded, the water level meter 3 transmits a message to the image processing apparatus 1 that the specified water level has been reached.

  In response to the notification, the image processing apparatus 1 transmits a message instructing photographing while turning at a low speed to the camera 2 associated with the water level gauge 3. The camera 2 receives the message and starts shooting while turning at a low speed. The camera 2 transmits an image obtained by photographing to the image processing apparatus 1. The image processing apparatus 1 stores an image received from the camera 2. The image processing apparatus 1 determines whether or not the camera 2 has turned 180 degrees or more. When it is determined that the camera 2 is not turning 180 degrees or more, the image processing apparatus 1 repeats the process of storing the received image. When it is determined that the camera 2 is turning 180 degrees or more, the image processing apparatus 1 combines the plurality of stored images to form a panoramic image. The image processing apparatus 1 transmits the formed panoramic image to the terminal 4. The terminal 4 displays the received panoramic image on a display device such as a monitor. Note that the image processing apparatus 1 performs the process of storing the image received from the camera 2 in parallel while the panoramic image is being formed.

  When the water level meter 3 determines that the measured water level is below the specified water level, the water level meter 3 transmits a message to that effect to the image processing apparatus 1. The image processing apparatus 1 transmits a message instructing the end of shooting to the camera 2. The camera 2 finishes shooting.

  Next, the configuration of the image processing apparatus 1 will be described. FIG. 3 is a block diagram illustrating an example of the configuration of the image processing apparatus 1. The image processing apparatus 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a mass storage unit 14, a communication unit 15, and a reading unit 16. Each component is connected by a bus B.

  The CPU 11 controls each part of the hardware according to the control program 1P stored in the ROM 12. The RAM 13 is, for example, SRAM (Static RAM), DRAM (Dynamic RAM), or flash memory. The RAM 13 temporarily stores data generated when the CPU 11 executes the program.

  The large-capacity storage unit 14 is, for example, a hard disk or an SSD (Solid State Drive). The large-capacity storage unit 14 stores various data necessary for panoramic image formation processing. The large-capacity storage unit 14 stores a camera DB (Data Base) 141, a water level gauge DB 142, a corresponding camera DB 143, a photographed image DB 144, an attribute DB 145, and a panoramic image DB 146. Further, the control program 1P may be stored in the large-capacity storage unit 14.

  The communication unit 15 communicates with the camera 2, the water level gauge 3, and the terminal 4 via the network N. The reading unit 16 reads a portable storage medium 1a including a CD (Compact Disc) -ROM and a DVD (Digital Versatile Disc) -ROM. The CPU 11 may read the control program 1P from the portable storage medium 1a via the reading unit 16 and store it in the large-capacity storage unit 14. Further, the CPU 11 may download the control program 1P from another computer via the network N or the like and store it in the large-capacity storage unit 14. Furthermore, the CPU 11 may read the control program 1P from the semiconductor memory 1b.

  Next, the database stored in the large capacity storage unit 14 will be described. FIG. 4 is an explanatory diagram showing an example of the record layout of the camera DB 141. The camera DB 141 is a database that stores parameters necessary for forming a panoramic image. The camera DB 141 includes an ID column, an angular velocity column, and a time interval column. The ID column stores an ID that uniquely identifies the camera 2. The angular velocity train stores the angular velocity at the time of low-speed turning in the camera 2. The time interval column stores a time interval indicating how much time should be taken when the camera 2 takes a picture.

  FIG. 5 is an explanatory diagram showing an example of the record layout of the water level gauge DB 142. The water level gauge DB 142 is a database that stores a prescribed water level that serves as a reference for starting the formation of a panoramic image. The water level gauge DB 142 includes an ID column and a water level column. The ID column stores an ID that uniquely identifies the water level meter 3. The water level row stores a prescribed water level that serves as a reference for starting the formation of a panoramic image.

  FIG. 6 is an explanatory diagram showing an example of a record layout of the corresponding camera DB 143. The correspondence camera DB 143 is a database that stores the correspondence between the water level gauge 3 and the camera 2. The corresponding camera DB 143 includes a water level meter ID column and a camera ID column.

  FIG. 7 is an explanatory diagram showing an example of the record layout of the attribute DB 145. The attribute DB 145 is a database that stores attributes of panoramic images. The attribute DB 145 includes an image ID column, a camera ID column, a date column, a time column, and a file name column. The image ID stores a panoramic image ID that uniquely identifies the panoramic image. The camera ID column stores the ID of the camera 2 that captured the image that is the source of the panoramic image. The date column stores the date on which shooting for forming a panoramic image was started. The time column stores the time when photographing for forming a panoramic image is started. The date and time stored in the date string and the time string may be the shooting date and shooting time of the last image shot among a plurality of images that are the source of the panoramic image. Further, the date and time stored in the date string and the time string may be the date and time when the panoramic image is formed or stored. The file name column stores the file name of the panorama image file that is the substance of the panorama image. The panorama image file is stored in the panorama image DB 146.

  Note that the attributes of the panorama image stored in the attribute DB 145 may be included in the file name of the panorama image file. The file format of the panorama image file may be Exif (Exchangeable Image File Format), and the attributes of the panorama image stored in the attribute DB 145 may be stored in the attribute area of the file. In these cases, the attribute DB 145 is unnecessary. The panoramic image DB 146 may be constructed by a relational database that can store image data. In this case, the attribute DB 145 and the panorama image DB 146 are one database.

  The captured image DB 144 not showing the record layout stores an image captured by the camera 2 as a file. The image file stored in the photographed image DB 144 includes attributes such as the camera ID and the photographing date / time in the file name. Alternatively, the file format may be Exif, and the camera ID and shooting date / time may be stored in the attribute area of the image file. Similar to the panoramic image DB 146, the photographed image DB 144 may be constructed by a relational database that can store image data.

  Subsequently, formation of a panoramic image will be described. FIG. 8 is an explanatory view showing a panoramic image forming method. As shown in FIG. 8A, in order to form a panoramic image, two images in which a common subject is reflected in a part of the image are synthesized. In FIG. 8A, a common subject appears in a region C1 of an image IMG1 and a region C2 of an image IMG2. Therefore, when the image IMG1 and the image IMG2 are combined, they are combined so that the region C1 and the region C2 overlap. As shown in FIG. 8B, it is possible to form a panoramic image by repeating such composition on n images (n is an integer of n> 1).

  As described above, the camera 2 captures an image while turning at a low speed, and transmits the captured image to the image processing apparatus 1. In order to form a panoramic image as shown in FIG. 8, it is only necessary that a common subject appears in a part of two adjacent images. Therefore, in order to form a panoramic image, the camera 2 needs to take images intermittently while turning. FIG. 9 is an explanatory diagram showing changes in the shooting range of the camera 2. In the example of FIG. 9, the camera 2 is installed in the riverbank. The turning angle θ is used to indicate the direction of the camera 2. As shown in FIG. 9A, the angle θ when the camera 2 is facing leftward in parallel with the river is assumed to be 0 degree. As shown in FIG. 9C, the angle θ when the camera 2 faces rightward in parallel with the river is 180 degrees. Here, it is assumed that the camera 2 turns from a turning angle of 0 degrees to 180 degrees, but this is an example. Other angle ranges may be used.

As shown in FIG. 9A, when θ is 0 degree, the shooting range of the camera 2 is R ₁ . The image taken when θ is 0 degree is the first image that is the source of the panoramic image. Necessary for forming a panoramic image. As shown in FIG. 9B, when θ = α1, the shooting range of the camera 2 is R ₂ . Since R ₁ and R ₂ partially overlap, the image taken in this case is the second image. Similarly, it is possible to form a panoramic image using an image taken every time a turning angle where a part of the photographing range overlaps. As shown in FIG. 9C, a panoramic image is formed from n images shot in each shooting range from the shooting range R ₁ to R _n .

  Next, processing performed by the image processing apparatus 1 will be described. FIG. 10 is a flowchart illustrating an example of a procedure of time interval calculation processing. The time interval calculation process is a process for obtaining an angular velocity and a time interval stored in the camera DB 141. The CPU 11 of the image processing apparatus 1 moves the camera 2 to the home position (step S1). The home position is a predetermined position. The home position is one end of the monitoring angle range of the camera 2 except when the monitoring angle range of the camera 2 is 360 degrees. When the monitoring angle range is 360 degrees, the home position of the camera 2 is an arbitrary position.

  The CPU 11 acquires a captured image (first image) of the camera 2 at the home position (first position). The CPU 11 stores the acquired image in the temporary storage area. The temporary storage area is provided in the RAM 13 or the large capacity storage unit 14. The CPU 11 extracts feature points in the image (step S2). The CPU 11 instructs the camera 2 to start low-speed turning (step S3). The CPU 11 performs matching between the image transmitted from the camera 2 and the feature point extracted in step S2 (step S4). The CPU 11 determines from the matching result whether the camera 2 has returned to the home position again (step S5). If the CPU 11 determines that the camera 2 has not returned to the home position (NO in step S5), the process returns to step S4. When the CPU 11 determines that the camera 2 has returned to the home position (YES in step S5), the CPU 11 stops the turning of the camera 2 (step S6). Here, the camera 2 has returned to the home position refers to the following case. When the monitoring angle range is not 360 degrees, it turns from the home position, which is one end of the angle range, to the other end, temporarily stops, and then turns in the opposite direction to return to the home position again. When the monitoring angle range is 360 degrees, turning is started from a predetermined home position, turning 360 degrees, and returning to the home position again. Here, the feature point of the image (second image) captured after returning to the home position should match the feature point of the image initially captured at the home position. The CPU 11 calculates the turning angular velocity when the camera 2 is turning at low speed from the time during which the camera 2 is turning and the known monitoring angle range of the camera 2 (step S7). The CPU 11 compares the image at the home position with the images continuously taken, and the turning time in which the overlapping portion including the same subject is within a certain range (for example, 1/3 of the entire image), that is, time An interval (interval of photographing time) is obtained (step S8). The CPU 11 stores the turning angular velocity and the time interval in the camera DB 141.

  The time interval calculation process is executed for each camera 2 during normal daytime, and the turning angular velocity and time interval for all the cameras 2 are stored in the camera DB 141.

  FIG. 11 is a flowchart illustrating an example of the procedure of the activation process. The start-up process is a process that is executed every time a notification that the measured water level exceeds the specified water level is received from the water level meter 3. The CPU 11 obtains the water level ID of the water level meter that has exceeded the specified water level (step S11). CPU11 acquires camera ID corresponding to water level meter ID from corresponding camera ID143 (step S12). The CPU 11 stores the camera ID as a monitoring target in the temporary storage area (step S13). The CPU 11 determines whether the monitoring process has already been activated (step S14). If it is determined that the CPU 11 monitoring process is being activated (YES in step S14), the process proceeds to step S16. If the CPU 11 determines that the monitoring process is not being activated (NO in step S14), the CPU 11 activates the monitoring process (step S15). The CPU 11 starts individual processing for the camera 2 having the camera ID acquired in step S12 (step S16). The CPU 11 ends the activation process.

  FIG. 12 is a flowchart showing an example of the procedure of individual processing. The process shown in FIG. 12 is a process activated in step S16 in FIG. The CPU 11 moves the target camera 2 to the home position (step S21). The CPU 11 causes the camera 2 to start a low-speed turn (step S22). The CPU 11 determines whether or not a predetermined time has elapsed (step S23). The predetermined time is an elapsed time since the last shooting. The predetermined time is a time interval defined in the camera DB 141. When the CPU 11 determines that the predetermined time has not elapsed (NO in step S23), the CPU 11 repeatedly executes step S23. If the CPU 11 determines that the predetermined time has elapsed (YES in step S23), the CPU 11 instructs the camera 2 to perform shooting (step S24). The CPU 11 determines whether or not the camera 2 has turned 180 degrees (step S25). The determination as to whether or not the vehicle has turned 180 degrees is performed as follows. The CPU 11 acquires the angular velocity corresponding to the camera ID of the camera 2 to be processed from the camera DB 141. The turning angle is calculated from the acquired angular velocity and the elapsed time after starting the low-speed turning, and it is determined whether or not the value is 180 degrees or more. If the CPU 11 determines that the camera 2 is not turning 180 degrees (NO in step S25), the process returns to step S23. If the CPU 11 determines that the camera 2 has turned 180 degrees (YES in step S25), the CPU 11 stops the turning of the camera 2 (step S26). The CPU 11 joins the images stored in the photographed image DB 144 to form a panoramic image (step S27). The CPU 11 outputs the formed panoramic image to the terminal 4 (step S28). The CPU 11 determines whether or not to stop the process (step S29). The CPU 11 determines whether or not the processing target camera ID has been deleted from the monitoring target camera ID stored in the temporary storage area. If the CPU 11 determines that the camera ID to be processed has been deleted from the temporary storage area, the CPU 11 determines to stop the process. If the CPU 11 determines that the camera ID to be processed has not been deleted from the temporary storage area, the CPU 11 determines not to stop the process. If the CPU 11 determines not to stop the process (NO in step S29), the process returns to step S21. When it is determined that the process is to be stopped (YES in step S29), the CPU 11 ends the individual process. The individual process is a process executed for each camera 2. Therefore, when the plurality of cameras 2 are performing individual processing, the CPU 11 performs the plurality of individual processing in parallel.

  In the individual processing described above, the camera 2 intermittently performs shooting according to a command from the image processing apparatus 1 at each time interval. However, the camera 2 may always take a picture while turning. In this case, a part of the image stored in the photographed image DB 144 is used. After the camera 2 turns 180 degrees, the CPU 11 extracts a necessary image from the photographed image DB 144. The CPU 11 refers to the shooting time of each image and extracts an image for each shooting time interval. The CPU 11 forms a panoramic image from the extracted image.

  FIG. 13 is a flowchart illustrating an example of the procedure of the monitoring process. The monitoring process is a process activated in step S15 in FIG. The monitoring process realizes a function of stopping the process (individual process) for forming a panoramic image of the corresponding camera 2 in response to the water level being lower than the specified water level from the water level meter 3. The CPU 11 determines whether or not a message indicating that the water level has fallen below the specified water level has been received from the water level gauge 3 (step S31). When the CPU 11 determines that a message indicating that the water level has fallen below the specified water level has not been received from the water level gauge 3 (NO in step S31), the CPU 11 repeatedly executes step S31. If the CPU 11 determines that a message indicating that the water level has fallen below the specified water level has been received from the water level meter 3 (YES in step S31), the CPU 11 acquires the camera ID corresponding to the water level meter ID of the water level meter that has fallen below the water level from the corresponding camera ID 143. (Step S32). The CPU 11 deletes the acquired camera ID from the monitoring target stored in the temporary storage area (step S33). The CPU 11 determines whether there is a monitoring target (step S34). The CPU 11 determines whether all the camera IDs of the cameras to be monitored stored in the temporary storage area have been deleted. If the CPU 11 determines that all the camera IDs of the cameras to be monitored have been deleted, it determines that there is no monitoring target. If the CPU 11 determines that at least one camera ID of the camera to be monitored is stored, it determines that there is a monitoring target. If the CPU 11 determines that there is a monitoring target (YES in step S34), the process returns to step S31. If the CPU 11 determines that there is no monitoring target (NO in step S34), the process ends.

  It supplements about step S8 of the above-mentioned time interval calculation process. In step S8, the CPU 11 calculates the pixel movement amount of the captured image by image processing. If the camera 2 is an SD (Standard Definition) camera, the horizontal 720 dots, and if it is an HD (High Definition) camera, the horizontal 1920 dots are searched for and become a predetermined overlapping portion (for example, 1 / of the whole). The CPU 11 obtains the time 3), and stores it in the camera DB 141 as a time interval. In order to calculate the pixel movement amount between the captured images, feature point matching, pattern matching of the central portion, phase-only correlation method, or the like is used. Since these methods are well-known techniques, detailed description thereof is omitted.

  This embodiment has the following effects. The camera 2 does not always shoot while turning. In order to obtain an image necessary for forming a panoramic image, photographing is performed at predetermined time intervals. For this reason, there is no need to select an image to be used when forming a panoramic image. In addition, the storage capacity of the photographed image DB 144 can be saved.

  Since the angular velocity of the turn for each camera 2 is obtained, even if the cameras 2 having different angular velocities for the low-speed turn are mixed, appropriate control can be performed. In addition, since the relationship between adjacent images necessary for selecting an image necessary for forming a panoramic image is represented by a time interval, a panoramic image can be formed even for the camera 2 that cannot obtain a turning angle. .

Embodiment 2
The present embodiment relates to a form corresponding to night photography. Although the camera 2 is a camera that can shoot at night, the camera 2 may operate in the accumulation mode at night when the ambient light is scarce. The accumulation mode is a mode for acquiring an image having a contrast even when ambient light is scarce. This is the same concept as long-time exposure for still cameras. In the accumulation mode, the camera 2 uses a plurality of images taken during a predetermined time (referred to as accumulation time) in order to output one output image. For example, when the accumulation time is 2 seconds, the camera 2 outputs one image obtained by combining a plurality of images taken during 2 seconds. Therefore, the camera 2 outputs the same image for 2 seconds, and outputs a new image after 2 seconds. Thereafter, the camera 2 continues to output the same image until 2 seconds elapse. In such an accumulation mode, when the camera 2 is turned at a low speed, the shooting range changes during 2 seconds when the image is accumulated, so that the subject is blurred or the same subject appears in a plurality of different positions. This makes it difficult to form a panoramic image. In the present embodiment, it is possible to form a panoramic image even when the camera 2 is operating in the accumulation mode.

  FIG. 14 is an explanatory diagram showing another example of the record layout of the camera DB 141. The camera DB 141 includes an ID column, an angular velocity column, a time interval column, and an accumulation time column. Since the ID column, the angular velocity column, and the time interval column are the same as those in the first embodiment, description thereof is omitted. The accumulation time column stores the accumulation time when the camera 2 enters the accumulation mode.

  FIG. 15 is a flowchart illustrating an example of the procedure of the accumulation time calculation process. The CPU 11 moves the camera 2 to the home position (step S41). The CPU 11 extracts feature points from the captured image of the camera 2 at the home position (step S42). The CPU 11 causes the camera 2 to start turning at a low speed (step S43). The CPU 11 performs matching between the captured image of the camera 2 and the home position (step S44). The CPU 11 determines which position in the captured image the feature point extracted in step S42 is. The CPU 11 determines whether or not the captured image is the same as the image at the home position (step S45). When the CPU 11 determines that the feature point is at the same position in the captured image and the image at the home position, the CPU 11 determines that the images are the same. When the CPU 11 determines that the feature points are at different positions, the CPU 11 determines that they are not the same image. If the CPU 11 determines that the images are the same (YES in step S45), the process returns to step S44. If the CPU 11 determines that the images are not the same (NO in step S45), it starts time measurement (step S46). The CPU 11 extracts feature points of a new photographed image (hereinafter referred to as “reference image”) (step S47). Next, the CPU 11 performs matching between the image obtained by the camera 2 and the reference image (step S48). The CPU 11 determines whether or not the newly obtained image is the same as the reference image (step S49). If the CPU 11 determines that the images are the same (YES in step S49), the process returns to step S48. If the CPU 11 determines that the images are not the same (NO in step S49), the time measurement is stopped (step S50). The CPU 11 stops the turning of the camera 2 (step S51). The CPU 11 stores the time from the start of measurement to the stop of measurement as the accumulation time in the camera DB 141 (step S52). The CPU 11 ends the process. The accumulation time calculation process is executed for all cameras in normal times.

  FIG. 16 is a flowchart illustrating an example of the procedure of the mode determination process. The CPU 11 moves the camera 2 to the home position (step S61). The CPU 11 extracts feature points from the image at the home position (step S62). The CPU 11 causes the camera 2 to start turning at a low speed (step S63). The CPU 11 performs matching between the newly obtained image and the image at the home position (step S64). The CPU 11 determines whether or not the newly obtained image is the same as the image at the home position (step S65). When the CPU 11 determines that the newly obtained image and the image at the home position are not the same (NO in step S65), the CPU 11 determines that the camera 2 is operating in the normal mode, and the operation mode of the image processing apparatus 1 is also normal. The mode is set (step S66). The normal mode of the image processing apparatus 1 is a mode for performing the operation shown in the first embodiment. When the CPU 11 determines that the newly obtained image is the same as the image at the home position (YES in step S65), the camera 2 determines that the camera 2 is operating in the accumulation mode, and the operation mode of the image processing apparatus 1 is also accumulated. The mode is set (step S67). The CPU 11 ends the process. Note that the mode determination process may be performed for each camera 2 or only a specific camera 2 may be performed. When the mode determination process is performed for each camera 2, the processing mode is set for each camera 2 also in the image processing apparatus 1. When the mode determination process is performed only by a specific camera 2, the operation mode of the image processing apparatus 1 is common to all the cameras 2. It may be executed at a time near the sunset. In addition, when the mode determination process is performed only with the specific camera 2, the camera 2 installed in the place where it becomes darkest most quickly may be selected. Even though the camera 2 is in the accumulation mode, a panoramic image cannot be obtained when the image processing apparatus 1 operates in the normal mode. However, even if the image processing apparatus 1 operates in the accumulation mode, it is possible to obtain a panoramic image from the captured image of the camera 2 operating in the normal mode.

  FIG. 17 is a flowchart showing another example of the procedure of the individual processing. The process shown in FIG. 17 is the same as the process in Embodiment 1 shown in FIG. In the following description, a different part from Embodiment 1 is mainly demonstrated. The CPU 11 reads out the time interval and accumulation time of the camera 2 to be processed from the camera DB 141 (step S71). The CPU 11 moves the camera 2 to the home position (step S72). The CPU 11 determines whether or not the time corresponding to the time interval has elapsed after the camera 2 starts the low-speed turn (step S73) (step S74). When it is determined that the time corresponding to the time interval has not elapsed (NO in step S74), the CPU 11 repeatedly executes S74. If the CPU 11 determines that the time corresponding to the time interval has elapsed (YES in step S74), the CPU 11 stops the turning of the camera 2 (step S75). The CPU 11 determines whether or not an accumulation time has elapsed since stopping the turning (step S76). When the CPU 11 determines that the accumulation time has not elapsed since the turning was stopped (NO in step S76), the CPU 11 repeatedly executes step S76. If the CPU 11 determines that the accumulation time has elapsed since stopping the turn (YES in step S76), the CPU 11 stores the image obtained from the camera 2 in the captured image DB 144 (step S77). The CPU 11 determines whether or not the camera 2 has turned 180 degrees (step S78). When the CPU 11 determines that the camera 2 is not turning 180 degrees (NO in step S78), the CPU 11 restarts the low-speed turning (step S79). The CPU 11 returns the process to step S74. If the CPU 11 determines that the camera 2 has turned 180 degrees (YES in step S78), the CPU 11 forms a panoramic image (step S80). The CPU 11 outputs the formed panoramic image to the terminal 4 (step S81). The CPU 11 determines whether to stop the process (step S82). If the CPU 11 determines not to stop the process (NO in step S82), the process returns to step S72. If the CPU 11 determines to stop the process (YES in step S82), the process ends.

  In the present embodiment, in addition to the effects exhibited by the first embodiment, the following effects are achieved. Even when the camera 2 operates in the accumulation mode, a panoramic image can be formed.

  FIG. 18 is a block diagram illustrating an example of a functional configuration of the image processing apparatus 1. The image processing apparatus 1 includes an extracting unit 11a, a specifying unit 11b, a calculating unit 11c, and a transmitting unit 11d. Each of these functional units is realized by the CPU 11 operating based on the control program 1P.

  The extraction unit 11a extracts feature points included in the first image at the first position received from the camera, receives an image taken while turning the camera at a specific speed from the first position, Feature points included in each of the images taken while turning are extracted. The specifying unit 11b specifies a second image having a feature point that matches the feature point of the image included in the first image among the feature points of each extracted image. The calculating unit 11c measures the time from when the camera starts turning until the second image is taken starting from the first position, and calculates the angular velocity when the camera is turned based on the measured time. To do. The transmission unit 11d specifies a time interval for capturing an image necessary for creating a panoramic image based on the calculated angular velocity, and transmits a shooting instruction to the camera at each specified time interval.

  In the above description, river monitoring is taken as an example, but this is not a limitation. A panoramic image may be formed in order to grasp the damage situation after the earthquake using a surveillance camera installed at a street corner. Further, a panoramic image may be formed in order to grasp the damage situation at the time of disaster such as heavy rain or earthquake by using a monitoring camera for grasping the traffic condition of the road. Furthermore, a panoramic image may be formed in order to grasp the damage situation at the time of a disaster by using a camera that captures a waiting state installed on the rooftop of a broadcasting station.

The technical features (components) described in each embodiment can be combined with each other, and a new technical feature can be formed by combining them.
The embodiments disclosed herein are illustrative in all respects and should not be considered as restrictive. The scope of the present invention is defined not by the above-mentioned meaning but by the scope of the claims, and is intended to include all modifications within the meaning and scope equivalent to the scope of the claims.

  Regarding the above embodiment, the following additional notes are disclosed.

(Appendix 1)
An image forming program for receiving a plurality of images from a camera and forming a panoramic image from the received plurality of images,
Extracting feature points included in the first image at the first position received from the camera;
Receive an image taken while turning the camera at a specific speed starting from the first position,
Extract feature points included in each of the images taken while turning,
A second image having a feature point that matches a feature point of the image included in the first image among the feature points of the extracted images;
Measuring the time from the start of turning the camera starting from the first position until the second image is taken;
Calculate the angular velocity when turning the camera based on the measured time,
Based on the calculated angular velocity, specify a time interval for capturing images necessary for creating a panoramic image,
An image forming program that causes a computer to execute a process of transmitting a shooting instruction to the camera at each specified time interval.

(Appendix 2)
The image forming program according to appendix 1, wherein the time interval is a time required to capture an image with a predetermined ratio of image overlap with respect to the first image.

(Appendix 3)
When creating a panoramic image based on the calculated angular velocity, the time until the camera rotates 180 ° is calculated based on the angular velocity,
The image forming program according to appendix 2, wherein photographing is performed at each specified time interval within the calculated time range.

(Appendix 4)
Receiving a plurality of images taken while turning the camera at a specific speed,
Perform a feature point difference comparison between the received images,
Find the accumulation time for the camera to output the same image from the result of the difference comparison,
Any one of appendix 1 to appendix 3, characterized in that the camera stops after turning for the time period, takes an image during the accumulation time period, and transmits an instruction to start turning again. The image forming program described in 1.

(Appendix 5)
The image forming program according to appendix 4, wherein the accumulation time is a time during which a difference value obtained by difference comparison is continuously equal to or less than a threshold value.

(Appendix 6)
An image forming apparatus that receives a plurality of images from a camera and forms a panoramic image from the received plurality of images,
A feature point included in the first image at the first position received from the camera is extracted, and an image taken while turning the camera at a specific speed from the first position is received and turned. An extraction unit for extracting feature points included in each of the images taken while,
A specifying unit that specifies a second image having a feature point that matches a feature point of an image included in the first image among the feature points of each of the extracted images;
Measure the time from the start of turning the camera starting from the first position until the second image is taken, and calculate the angular velocity when turning the camera based on the measured time A calculating unit to
A transmission unit that identifies a time interval for capturing an image necessary for creating a panoramic image based on the calculated angular velocity, and transmits a photographing instruction to the camera at each identified time interval. An image forming apparatus.

(Appendix 7)
An image forming method for receiving a plurality of images from a camera and forming a panoramic image from the received plurality of images,
Extracting feature points included in the first image at the first position received from the camera;
Receive an image taken while turning the camera at a specific speed starting from the first position,
Extract feature points included in each of the images taken while turning,
A second image having a feature point that matches a feature point of the image included in the first image among the feature points of the extracted images;
Measuring the time from the start of turning the camera starting from the first position until the second image is taken;
Calculate the angular velocity when turning the camera based on the measured time,
Based on the calculated angular velocity, specify a time interval for capturing images necessary for creating a panoramic image,
An image forming method, wherein the computer executes a process of transmitting a shooting instruction to the camera at each specified time interval.

100 panoramic image forming system 1 image processing apparatus 11 CPU
11a Extraction unit 11b Identification unit 11c Calculation unit 11d Transmission unit 12 ROM
13 RAM
14 Mass storage unit 141 Camera DB
142 Water Level DB
143 Compatible Camera DB
144 Shooting image DB
145 attribute DB
146 Panorama image DB
DESCRIPTION OF SYMBOLS 15 Communication part 16 Reading part 1P Control program 1a Portable storage medium 1b Semiconductor memory 2 Camera 3 Water level meter 4 Terminal N network

Claims (5)

An image forming program for receiving a plurality of images from a camera and forming a panoramic image from the received plurality of images,
Extracting feature points included in the first image at the first position received from the camera;
Receive an image taken while turning the camera at a specific speed starting from the first position,
Extract feature points included in each of the images taken while turning,
A second image having a feature point that matches a feature point of the image included in the first image among the feature points of the extracted images;
Measuring the time from the start of turning the camera starting from the first position until the second image is taken;
Calculate the angular velocity when turning the camera based on the measured time,
Based on the calculated angular velocity, specify a time interval for capturing images necessary for creating a panoramic image,
An image forming program that causes a computer to execute a process of transmitting a shooting instruction to the camera at each specified time interval. 2. The image forming program according to claim 1, wherein the time interval is a time until an image with a predetermined ratio of image overlap with respect to the first image is taken. When creating a panoramic image based on the calculated angular velocity, the time until the camera rotates 180 ° is calculated based on the angular velocity,
3. The image forming program according to claim 2, wherein photographing is performed for each specified time interval within the calculated time range. An image forming apparatus that receives a plurality of images from a camera and forms a panoramic image from the received plurality of images,
A feature point included in the first image at the first position received from the camera is extracted, and an image taken while turning the camera at a specific speed from the first position is received and turned. An extraction unit for extracting feature points included in each of the images taken while,
A specifying unit that specifies a second image having a feature point that matches a feature point of an image included in the first image among the feature points of each of the extracted images;
Measure the time from the start of turning the camera starting from the first position until the second image is taken, and calculate the angular velocity when turning the camera based on the measured time A calculating unit to
A transmission unit that identifies a time interval for capturing an image necessary for creating a panoramic image based on the calculated angular velocity, and transmits a photographing instruction to the camera at each identified time interval. An image forming apparatus. An image forming method for receiving a plurality of images from a camera and forming a panoramic image from the received plurality of images,
Extracting feature points included in the first image at the first position received from the camera;
Receive an image taken while turning the camera at a specific speed starting from the first position,
Extract feature points included in each of the images taken while turning,
A second image having a feature point that matches a feature point of the image included in the first image among the feature points of the extracted images;
Measuring the time from the start of turning the camera starting from the first position until the second image is taken;
Calculate the angular velocity when turning the camera based on the measured time,
Based on the calculated angular velocity, specify a time interval for capturing images necessary for creating a panoramic image,
An image forming method, wherein the computer executes a process of transmitting a shooting instruction to the camera at each specified time interval.

Images