JP2022189552A - Imaging system, installation method, and computer program - Google Patents

Abstract

To provide an imaging system including a plurality of cameras, an installation method, and a computer program, capable of optimizing the installation location of a sub-camera.SOLUTION: In an imaging system including a plurality of cameras 101, 104, 105, included are a selection step S702 for selecting a main camera from among the plurality of cameras and recommended location outputting steps S704 to S706 for outputting recommended locations of the sub-cameras 104, 105 other than the main camera on the basis of the location of the main camera 101 selected by selection means.SELECTED DRAWING: Figure 7

Description

The present invention relates to an imaging system having multiple cameras, an installation method, and a computer program.

In recent years, in the field of video production, there are an increasing number of cases in which a plurality of cameras are installed and live distribution is performed, or a captured moving image is edited later to be completed as a video work. For example, by installing multiple cameras and shooting from various angles and distances, the video can be switched to a camera image that can emphasize the subject of interest in the video with a switcher and live distribution. Alternatively, the video recorded by a plurality of cameras is edited so that the subject of interest can be switched to a scene that can be more emphasized, thereby giving a change to the video.

However, when installing the cameras, it is necessary to set the installation position of each camera appropriately according to the distance from the subject and the number of cameras that can be installed. If the installation position formation of each camera is inappropriate, there are cases where another camera is reflected in the shooting angle of view, and there are cases where the image does not change much even if the camera is switched. be.

Japanese Patent Application Laid-Open No. 2019-118136 JP 2019-149823 A

In the above Patent Document 1, captured image data is stored, a blind spot is detected from the captured image data, a movement instruction signal is generated so as to prevent the occurrence of the blind spot, and a moving camera moves according to this movement instruction signal. , and continue shooting. With this method, it is possible to move the moving camera evenly so as to eliminate blind spots in all shot images. However, when shooting with the main camera and sub-camera fixed, it is not possible to determine the appropriate position according to the number of cameras. be.

Japanese Patent Application Laid-Open No. 2002-200001 discloses a method of communicating information about whether or not shooting is possible when there are a plurality of imaging devices. However, the information on whether or not shooting is possible in this method is mainly based on storage space and whether it can be operated, and the installer needs to consider the installation position of the sub camera through trial and error. .

SUMMARY OF THE INVENTION Accordingly, it is an object of the present invention to provide a photographing system including a plurality of cameras, in which the installation positions of the sub-cameras can be easily optimized.

In order to achieve the above object, the present invention
In an imaging system that includes multiple cameras,
selecting means for selecting a main camera from the plurality of cameras;
recommended position output means for outputting recommended positions of sub-cameras other than the main camera based on the position of the main camera selected by the selection means;
shooting system with

According to the present invention, it is possible to easily optimize the installation positions of the sub-cameras in an imaging system including a plurality of cameras.

1 is a configuration diagram of an imaging system in an embodiment of the present invention; FIG. 1 is a functional block diagram of an imaging system in an embodiment; FIG. FIG. 4 is a diagram showing an example of camera installation and a shooting scene in the shooting system of the embodiment; FIG. 4 is a formation diagram defining recommended installation positions of a main camera and a sub-camera when shooting a scene as shown in FIG. 3; It is a formation data table that defines the recommended installation positions of the sub-cameras with reference to the main camera. FIG. 3 is a diagram showing an example of a camera setting UI screen of the camera 101, which is the main camera; 7 is a flow chart showing a processing flow for determining a recommended installation position and assisting installation according to the present embodiment. FIG. 4 is a schematic diagram showing a portion of the camera body that presents the degree of deviation; FIG. 5 is a schematic diagram showing an example of work for installing a sub-camera at a recommended position in the imaging system of the embodiment; FIG. 10 is a diagram showing an example of a camera setting UI screen for performing re-determination when the sub-camera cannot be installed at the recommended installation position determined the first time;

BEST MODE FOR CARRYING OUT THE INVENTION Preferred embodiments of the present invention will be described below using examples with reference to the accompanying drawings. In each drawing, the same members or elements are denoted by the same reference numerals, and overlapping descriptions are omitted or simplified.

FIG. 1 is a configuration diagram of an imaging system in an embodiment. The photographing system in this embodiment mainly consists of three cameras (101, 104 and 105, respectively), a terminal 102 and a network 103. FIG. A camera 101, a camera 104, a camera 105, and a terminal 102 are connected to a wireless network 103, for example, and can communicate with each other. A user can use the terminal 102 to set and operate the cameras 101 , 104 and 105 . The network 103 may be wired as long as it can communicate with the cameras 101, 104, 105, and terminal 102, and is not limited to a network system.

FIG. 2 is a functional block diagram of the imaging system in the embodiment.
The camera 101 (and camera 104 and camera 105) includes an imaging unit 201, an image processing unit 202, a system control unit 203, a camera installation control unit 204, a camera position acquisition unit 208, a video output unit 209, a storage unit 210, and a communication unit 211. , and a port controller 212 . The camera installation control unit 204 also includes a recommended installation position determination processing unit 205 , an installation assistance processing unit 206 and an installation guidance notification processing unit 207 .

The image capturing unit 201 includes an optical system and an image sensor for capturing a subject image formed by the optical system. The imaging element may be, for example, a CMOS image sensor or the like, and performs imaging of a subject image and photoelectric conversion into an electric signal.
The image processing unit 202 performs image processing and compression encoding processing on the electrical signal converted by the imaging unit 201 to generate image data, and transmits the generated image data to the system control unit 203 .

The system control unit 203 distributes the generated image data to the terminal 102 via the communication unit 211. FIG. Also, the camera control command transmitted from the terminal 102 via the communication unit 211 is analyzed, and processing according to the command is performed. For example, settings such as image quality adjustment settings for the image processing unit 202 are performed. The system control unit 203 also has a volatile memory, which can temporarily store results of internal processing.
Furthermore, the system control unit 203 incorporates a CPU as a computer, and serves as control means for controlling the operation of each unit of the camera and interlocking with other cameras based on a computer program stored in a memory as a storage medium. It is functioning.

Some of the functional blocks shown in FIG. 2 are realized by causing a computer included in the system control unit 203 to execute a computer program. I don't mind. As hardware, a dedicated circuit (ASIC), a processor (reconfigurable processor, DSP), or the like can be used.
Moreover, each functional block shown in FIG. 2 may not be built in the same housing, and may be configured by separate devices connected to each other.

The camera installation control unit 204 is a control unit that performs various processes related to camera installation. have a part.
A recommended installation position determination processing unit 205 is a processing unit that determines where each of the plurality of cameras should be set, and operates on the main camera side (the camera 101 in FIG. 1). Specifically, it is responsible for determining the recommended installation position based on information such as the number of sub-cameras and the distance to the subject that has been input.

The installation assistance processing unit 206 is a processing unit that assists the installer to install the apparatus at the installation position determined by the recommended installation position determination processing unit 205, and operates on the main camera side (camera 101 in the example of FIG. 1). Specifically, the positions of the sub-cameras (cameras 104 and 105 in the example of FIG. 1) are obtained, and the degree of positional deviation (amount of deviation) from the recommended position is notified to the sub-cameras to assist in the installation of the sub-cameras. responsible for processing.

The processing of the installation guidance notification processing unit 207 is performed on the sub camera side (the cameras 104 and 105 in the example of FIG. 1), and receives the degree of positional deviation from the recommended position notified from the main camera. Then, based on the received degree of positional deviation, a process of presenting (notifying) the degree of deviation for the installer to guide the camera to the recommended position is performed. Specifically, the installer is notified by blinking of light emitted by a tally lamp or the like, sound such as a buzzer sound, vibration of a vibration motor, an image, or the like.

A camera position acquisition unit 208 performs processing for acquiring the position of the camera. For example, a wireless module (not shown) called a beacon receives radio waves emitted from a plurality of radio wave transmitters whose positions are specified in advance, and the camera can detect the radio wave intensity and the radio wave direction from each of the plurality of radio wave transmitters. Calculate the position of and get the position. The method of acquiring the camera position may be, for example, a method using GPS, or a method of acquiring the camera position and orientation by photographing a two-dimensional marker indicating a predetermined position, and is not limited.

A video output unit 209 has video output terminals such as HDMI (registered trademark) and SDI (Serial Digital Interface), and outputs an image generated by the image processing unit 202 to an external device based on an instruction from the system control unit 203 . output to
The storage unit 210 stores set values such as image quality adjustment parameters and network settings, and can be started using the previously set values even when restarted.

The communication unit 211 communicates with devices such as the external terminal 102 and transmits control commands received from the devices such as the external terminal 102 via the network 103 to the system control unit 203 . Then, communication processing for transmitting communication data instructed by the system control unit 203 to the network 103 is performed. Also, based on an instruction from the system control unit 203, the image data generated by the image processing unit 202 is transmitted to an external device via the network 103. FIG.

Among the commands transmitted from the external device such as the terminal 102 to the system control unit 203 , the commands related to camera installation are transferred to the camera installation control unit 204 . The system control unit 203 receives the results processed by the camera installation control unit 204 , and the communication unit 211 returns the results to the external terminal 102 or the like beyond the network 103 .

A port control unit 212 is a processing unit that controls various I/O ports, and turns ON/OFF a tally lamp 801, a buzzer 802, and a vibration motor 803 shown in FIG. Processing such as ON/OFF is performed.

FIG. 3 is a diagram showing an example of camera installation and shooting scenes in the shooting system of the embodiment. A camera 101 as a main camera, and cameras 104 and 105 as sub-cameras are prepared for photographing a stage, and a person 301 is standing on the stage. The main camera 101 has already been installed, and the sub-cameras 104 and 105 need to be installed at appropriate positions.

FIG. 4 is a formation diagram that defines the recommended installation positions of the main camera and sub cameras when shooting the scene shown in FIG.
Formation diagrams 401, 402, and 403 are formation patterns, and these formation diagrams showing recommended installation position patterns according to the distance to the subject and the number of sub-cameras are stored in the storage unit 210 as data.

The storage unit 210 functions as formation pattern storage means for storing camera formation patterns according to the position of the main camera, the distance to the subject, the number of sub-cameras, and the like. Here, if the installation position of the main camera is specified, the distance to the subject may be specified to some extent, so the distance to the subject is not essential, but it is better to use the distance between the main camera and the subject as a parameter. The recommended position of the sub camera can be determined more appropriately.

The formation chart includes the distance (L) between the target subject 410 and the main camera 411 and the recommended installation positions (recommended positions) of the main camera 411 and the sub cameras (412, 413, 414, 415). Various numerical data such as the relative position of each camera and the distance to the subject described in these formation charts are stored in the formation data table shown in FIG.

FIG. 5 is a formation data table that defines the recommended installation positions of the sub-cameras relative to the main camera.
The formation data table 500 includes TableNo (501), NumOfSubCamera (502), Coordinates_M (503), and TargetDistance (504). It also includes Coordinates_S1 (505), Coordinates_S2 (506), Coordinates_S3 (507), Coordinates_S4 (508).

TableNo (501) stores the registration number of the table as a value, which is a serial number from No.1.
NumOfSubCamera (502) stores the value of the number of sub cameras.
Coordinates_M (503) stores three-dimensional coordinate values as position information of the main camera.

TargetDistance (504) stores the value of the distance between the subject and the main camera.
Coordinates_S1 (505), Coordinates_S2 (506), Coordinates_S3 (507), and Coordinates_S4 (508) store three-dimensional coordinate values as recommended installation position information for sub cameras 1-4.

By referring to this formation data table, a recommended installation position is determined based on the distance between the subject and the main camera and the number of sub cameras.
In the example of the table described with reference to FIG. 5 of this embodiment, an example of one installed main camera and up to four sub cameras to be added was described, but the number of main cameras and the number of sub cameras are It is not limited to this.

If there are installed cameras other than the main camera, it is possible to add further camera position data such as Coordinates_FS1 to the table. Also, if there are five or more sub-cameras to be added, it is possible to further add the position data of the fifth and subsequent sub-cameras such as Coordinates_S5. That is, in the photographing system of this embodiment, the number of sub-cameras may be one or any number of plural sub-cameras.

FIG. 6 is a diagram showing an example of a camera setting UI screen of the camera 101, which is the main camera.
In FIG. 6, 600 denotes a camera setting UI screen of the terminal 102 that acquires and displays an image from the camera 101, which is the main camera, and is displayed by operating the web browser application software on the terminal 102. FIG.

A connection destination URL 601 is a URL input area for designating a URL for accessing the camera 101 .
A main camera specification input section 602 is a check box for setting the camera being accessed as the main camera. By clicking this check box, the camera 101 is selected and set as the main camera.

An additional installation camera number input unit 603 is an input unit for inputting the number of sub cameras to be added, and the number is set by inputting a numerical value in the numerical input area.
Reference numeral 604 denotes a recommended installation position determination execution button. When this button is pressed, a recommended installation position (recommended position) determination process is executed, and a formation diagram as a determination result is displayed in a determination formation display area 605 . Details of the determination process will be described later with reference to FIG.

The advice display field 606 displays advice on recommended installation positions (recommended positions) based on the determined formation.
Reference numeral 607 denotes a guidance notification start button, and when this button is pressed, a guidance notification instruction is issued to the sub-camera side according to the determined formation. As a result, processing for presenting the degree of deviation (amount of deviation) to the installer is performed on the sub camera side.

As described above, in this embodiment, by displaying the camera setting UI screen and operating the screen, determination of the recommended installation position and guidance to the recommended installation position can be executed.
FIG. 7 is a flow chart showing the flow of processing for determining a recommended installation position and assisting installation in this embodiment. The computers in the system control units 203 of the main camera and the sub camera execute the computer programs stored in the memory and operate in conjunction with each other to perform the operations of the steps in the flow chart of FIG.

In FIG. 7, the left side of the dotted line is the processing flow for the main camera (camera 101 in this embodiment), and the right side of the dotted line is the processing flow for the sub cameras (cameras 104 and 105 in this embodiment). The processing flow executed in this flowchart is executed mainly using the camera installation control unit 204 among the functional blocks in FIG.

In particular, the main camera recommended installation position determination processing unit 205 executes the processing of steps S701 to S705, and the main camera installation assistance processing unit 206 executes the processing flow of steps S706 to S716. Further, the sub-camera installation guidance notification processing unit 207 executes the processing flow of steps S706 to S716.

First, in step S701, access to the camera setting UI screen described above is accepted, and the camera setting UI screen as shown in FIG. 6 is displayed on the web browser screen of the terminal 102. FIG.
Next, in step S702, selection of the camera 101 as the main camera is accepted in the main camera designation input section 602 of the camera setting UI screen. Here, step S702 functions as selection means (selection step) for selecting the main camera from a plurality of cameras. Also, in step S702, determination execution instructions are received from the number of sub-cameras set in the additional installation camera number input unit 603 and the button depression of the recommended installation position determination execution button 604. FIG.

Upon receiving the determination execution instruction, the camera 101 measures the distance to the subject in step S703. For measuring the distance, for example, the camera 101 may be provided with an infrared sensor or the like (not shown) to measure the distance to the subject, or if the camera 101 is equipped with a distance measuring sensor, it is used.

Next, in step S704, using the position of the main camera, the measured distance to the subject, and the number of sub-cameras as parameters, the formation data table 500 described with reference to FIG. 5 is searched for the closest pattern. That is, the value of TargetDistance (504) is compared with the distance measured in step S703, and the shortest candidate is selected. Further, among the candidates, a search is made for a candidate that matches the value of NumOfSubCamera (502), which is the number of sub-cameras, and the number of sub-cameras received in step S702.

Then, in step S705, the formation selected by the search in S704 is determined as the formation of the recommended installation position.
In step S706, a formation chart corresponding to the formation determined in step S705 is displayed in the determined formation display area 605 described with reference to FIG.

Steps S704 to S706 serve as recommended position output means (recommended position output step) for outputting the recommended position of the sub-camera based on the position of the main camera selected by the selection means (furthermore, the distance to the subject). It is functioning. Also, at this time, if there are a plurality of sub-cameras, the recommended positions of the respective sub-cameras are output and displayed in the determination formation display area 605 .

In step S707, a guidance start instruction is received by pressing the guidance notification start button 607. In step S708, the sub-camera position is acquired, and the degree of deviation (shift amount) from the recommended position is calculated.
At this time, the sub-camera receives a request from the main camera and transmits the camera position to the main camera in step S709. At this time, the position of the sub-camera is measured using a beacon or the like as described above. The degree of deviation from the recommended position in step S708 is calculated by calculating the relative distance between the camera position acquired in step S709 and the value of Coordinates_S1 (505) in FIG. demand.

In step S710, the relative distance calculated in step S708 is notified to the sub-camera as the degree of deviation (amount of deviation).
The sub-camera side receives the notification in step S711, and confirms in step S712 whether or not the content of the received notification is the degree of deviation. Then, if it is the value of the degree of deviation, the sub-camera executes presentation processing for notifying (presenting) the degree of deviation to the user in step S713. If the notification is not the degree of deviation, the presentation process is stopped in step S714. Details will be described later with reference to FIGS. 8 and 9. FIG.

Then, on the main camera side, in step S715, it is confirmed whether or not the sub camera has arrived at the recommended position. is calculated again. When the sub-camera has arrived at the recommended position, a presentation end notification is sent to the sub-camera in step S716. In this manner, steps S707 to S716 function as auxiliary means (auxiliary steps) for assisting installation of the sub-camera at the recommended positions output from the recommended position output means.

Since the received notification is not the degree of deviation, the sub-camera that received the presentation end notification terminates the presentation processing (notification processing to the user) in step S714.
As described above, according to the present embodiment, it is possible to easily determine where the sub-camera to be additionally installed should be preferably placed, and assist the sub-camera to be installed at the recommended position.

FIG. 8 is a schematic diagram showing a portion of the camera body that presents the degree of deviation. A tally lamp 801 is provided in the cameras 104 and 105, and by changing the blinking pattern of the tally lamp such as the blinking cycle, it is possible to indicate whether the target is approaching or moving away from the recommended position.

Also, by changing the buzzer volume and sound pattern of the buzzer 802, it is possible to indicate whether the robot is approaching or moving away from the recommended position. Also, by changing the motor vibration pattern of the vibration motor 803 provided inside the camera body, it is possible to indicate whether the user is approaching or moving away from the recommended position. These patterns may be periodic patterns of intermittent operation of light emission, sound generation, or vibration, amplitude change patterns, or frequency patterns of light emission, sound generation, or vibration. .

In this embodiment, these three types of presentation means are described as examples, but the presentation means shown in FIG. 8 is an example, and does not limit the implementation means. I do not care. For example, it is possible to display the recommended position and the current position of the sub-camera on the screen of an external terminal, or use image display such as presenting on the screen whether the user is approaching or moving away from the recommended position. be. Also, two or more of them may be combined as appropriate.

That is, the auxiliary means uses at least one of a light emission pattern, a sound pattern, a vibration pattern, an image, etc. to notify the user of the amount of deviation (degree of deviation) between the recommended position of the sub-camera and the current position of the sub-camera. do. Thereby, it is possible to assist the user in setting the sub-camera.

FIG. 9 is a schematic diagram showing a work example of installing a sub-camera at a recommended position in the imaging system of the embodiment.
When the installer 901 moves the camera 104, which is a sub-camera, he/she moves it to the recommended position 902 while watching the blinking pattern of the tally lamp 801, for example. When the camera 104 approaches the recommended position, the blinking speed of the tally lamp is increased, and when the camera 104 reaches the recommended position, the tally lamp continues to light, thereby assisting installation at the recommended position.

FIG. 10 is a diagram showing an example of a camera setting UI screen for performing re-determination when the sub-camera cannot be installed at the recommended installation position determined the first time.
On the camera setting UI screen 1000, the installer clicks the icon of the sub-camera that could not be installed at the recommended installation position due to an obstacle or the like in the determination formation display area 605 to mark it. When the user presses the recommended installation position determination execution button 604 again, the formation data table 500 is searched again for formations in which sub cameras are not placed at the marked coordinates.

Then, the display is switched to the camera setting UI screen 1001 , and a new determination result is displayed in the determination formation display area 605 . That is, when the user inputs that the sub-camera cannot be installed at the recommended position output by the recommended position output means, a recommended position different from the recommended position is output.

The processing flow for executing re-determination is basically the same as the processing flow described in FIG. . In this way, even if the first recommended installation position cannot be installed, it is possible to determine another recommended installation position that avoids installation impossible positions.

The present invention has been described in detail above based on its preferred embodiments, but the present invention is not limited to the above embodiments, and various modifications are possible based on the gist of the present invention. They are not excluded from the scope of the invention.
It should be noted that a computer program that implements the functions of the above-described embodiments for part or all of the control in this embodiment may be supplied to the imaging system or the like via a network or various storage media. A computer (or CPU, MPU, etc.) in the imaging system or the like may read and execute the program. In that case, the program and the storage medium storing the program constitute the present invention.

101 camera 102 terminal 103 network 104 camera 105 camera 301 person 500 formation table 600 camera setting UI screen 801 tally lamp 802 speaker 803 vibration motor

Claims (11)

In an imaging system that includes multiple cameras,
selecting means for selecting a main camera from the plurality of cameras;
recommended position output means for outputting recommended positions of sub-cameras other than the main camera based on the position of the main camera selected by the selection means;
shooting system with 2. The photographing system according to claim 1, further comprising auxiliary means for assisting installation of said sub-camera at said recommended position outputted from said recommended position output means. 3. The photographing system according to claim 2, wherein said assist means uses at least one of light emission, sound, vibration, and image to assist. 4. The photographing system according to claim 3, wherein said auxiliary means notifies according to the amount of deviation between said recommended position of said sub-camera and the current position of said sub-camera. 5. The photographing system according to claim 4, wherein said auxiliary means notifies said shift amount by at least one of said light emission pattern, said sound pattern, and said vibration pattern. 2. The photographing system according to claim 1, wherein said recommended position output means further outputs recommended positions of sub-cameras other than said main camera based on the distance between said main camera and the subject. 2. The photographing system according to claim 1, wherein said recommended position output means outputs said recommended position of each of said sub-cameras when there are a plurality of said sub-cameras. 2. The photographing system according to claim 1, further comprising formation pattern storage means for storing a camera formation pattern corresponding to the position of the main camera, the distance to the subject, and the number of the sub-cameras. 2. The apparatus according to claim 1, wherein when a user inputs that the sub-camera cannot be installed at the recommended position output by the recommended position output means, a recommended position different from the recommended position is output. The imaging system described. In a method of installing an imaging system including a plurality of cameras,
a selection step of selecting a main camera from the plurality of cameras;
a recommended position output step of outputting recommended positions of sub-cameras other than the main camera based on the position of the main camera selected in the selection step;
Installation method with A computer program for controlling each means of the imaging system according to any one of claims 1 to 9 by a computer.

Images