JP5108563B2 - Neural network device, robot camera control device using the same, and neural network program - Google Patents

Description

  The present invention relates to a neural network device that predicts and calculates the position of a moving object, a robot camera control device using the same, and a neural network program, for example.

  In recent years, research has been conducted to apply a neural network imitating the mechanism of the human brain to various fields such as pattern recognition and prediction. As the structure of the neural network, a hierarchical neural network, a cascade correlation neural network, a recurrent neural network, and the like are known. Conventionally, for example, a method using a recurrent neural network has been proposed as a method for controlling the operation of a robot (see, for example, Patent Document 1).

The robot described in Patent Document 1 performs learning by backward calculation based on time-series past data input to the input layer of the recurrent neural network, and sets the weighting coefficient of the recurrent neural network to a predetermined value. When it is done, it is designed to perform movements that are close to human movements that face each other.
JP 2005-115654 A

  However, in the past, when inputting past data, if there is a lot of unnecessary past data, or if necessary past data is missing, the performance of the neural network will be adversely affected. The input range of the past data was repeatedly set until a neural network model (hereinafter referred to as “optimum model”) that reached a desired learning level was obtained. Therefore, since the conventional method requires complicated work as described above, there is a problem that it takes a long time to construct an optimum model.

  The present invention has been made in order to solve the above-mentioned problems, and can save labor for complicated work, and can construct a neural network device capable of constructing an optimum model in a shorter time than in the past. An object of the present invention is to provide a robot camera control device and a neural network program using the same.

The neural network device of the present invention is a neural network device for controlling the photographing operation of a robot camera equipped with a camera device and a pan head device for photographing a subject, and the neural network and data before the current time are stored in the neural network. Based on the step interval value setting unit for setting a step interval value indicating an interval when inputting in stages, a step number setting unit for setting the step number indicating the number of the intervals, the step interval value and the step number A data selection unit for selecting data before the current time to be input to the neural network, data input means for inputting data acquired at predetermined unit times to the neural network, and the neural network For data entered before the current time A learning state evaluation unit that evaluates a learning state of the neural network from output signal data of the neural network, and an optimum step interval value and an optimum number of steps that make the learning state of the neural network a predetermined learning state. Search area setting means for setting the search area by narrowing down the combination of the step interval value and the step number, and optimum step information search means for searching the optimum step interval value and the optimum step number from the search area, It has the structure provided with.

  With this configuration, the neural network device of the present invention sets the step interval value and the number of steps when the data before the current time is input to the neural network in stages, and the neural network device based on the step interval value and the step number Since data before the current time to be input is selected, labor-saving work can be saved, and an optimal model can be constructed in a shorter time than in the past.

Also, with this configuration, the neural network device of the present invention can optimize the input layer of the neural network easily because the optimal step information search means searches the optimal step interval value and the optimal number of steps from the search area. It is possible to save labor for complicated work, and an optimum model can be constructed in a shorter time than conventional.

  Further, in the neural network device of the present invention, the search area setting means divides a step interval value search area for searching for the optimum step interval value based on the step interval value and sets a first search interval. A first search interval setting unit configured to divide a step number search area for searching for the optimum step number based on the step number and set a second search interval. It has the composition provided.

  With this configuration, in the neural network device according to the present invention, the search region setting means includes the first search interval setting unit and the second search interval setting unit, so that the optimal step interval value and the optimal step number can be calculated from the search region. You can search in a short time.

  The robot camera control device of the present invention includes a neural network device, a subject detection device that detects a predetermined subject position and outputs subject position data, a camera device that captures the subject, and a pan head device. A robot camera, a robot camera control device that outputs shooting operation control signal data for controlling a shooting operation of the robot camera, and the subject position data and the shooting operation control signal data corresponding to the subject position data A learning device that learns the neural network based on learning data, and the learning device learns the learning data based on the step interval value and the number of steps when learning to learn the neural network. The neural network is inputted stepwise into the data input means. An optimal step interval value and an optimal number of steps at which a learning state of a network device is set to a predetermined learning state are obtained, and the robot camera control device performs the learning device during automatic control after learning by the neural network. Inputs the subject position data from the subject detection device to the data input means based on the optimum step interval value and the optimum number of steps obtained at the time of learning, and outputs the neural network output signal data to the photographing operation control. It is configured to output the signal data to the robot camera to control the shooting operation of the robot camera.

  With this configuration, the robot camera control device of the present invention can control the shooting operation of the robot camera by the output signal data of the neural network learned from the learning data.

The neural network program of the present invention is a neural network program that causes a computer to function as a neural network device that controls the photographing operation of a camera device that shoots a subject and a robot camera equipped with a pan head device. A step interval value setting unit for setting a step interval value indicating an interval when data before the current time is input to the neural network stepwise, and a step number setting unit for setting the step number indicating the number of the intervals And a data selection unit that selects data before the current time to be input to the neural network based on the step interval value and the number of steps, and the data acquired at predetermined unit times is included in the neural network Enter in Data input means, learning state evaluation means for evaluating the learning state of the neural network from output signal data of the neural network for the data before the current time input to the neural network, and the learning state of the neural network in advance A search area setting means for setting a search area for searching for an optimal step interval value and an optimal step number in a determined learning state by narrowing down a combination of the step interval value and the step number; and the optimal step interval It has the structure which functions as an optimal step information search means for searching the value and the optimal step number from the search area .

  With this configuration, the neural network program of the present invention sets the step interval value and the step number when inputting the data before the current time to the neural network step by step, and the neural network program based on the step interval value and the step number. Since data before the current time to be input is selected, labor-saving work can be saved, and an optimal model can be constructed in a shorter time than in the past.

  The present invention is able to save labor for complicated work, and has the effect that an optimum model can be constructed in a shorter time than the conventional one, and a robot camera control device and a neural network program using the same Can be provided.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings. An example in which the robot camera control device according to the present invention is applied to imaging operation control of a robot camera that takes a person as a subject in a studio of a broadcasting station will be described. This robot camera control device causes a robot camera to shoot a subject on behalf of a cameraman, and causes the robot camera to perform human-like natural camera work.

  First, the configuration of the robot camera control device in the present embodiment will be described. FIG. 1 is a diagram showing a schematic configuration of a robot camera control device 1 according to the present embodiment.

  As shown in FIG. 1, the robot camera control device 1 operates a sensor camera 10 that captures a subject, a subject detection device 20 that detects the position of the subject, a robot camera 30 that captures a subject, and the robot camera 30. A robot camera operating device (hereinafter referred to as “robotic camera operating device”) 40 and a learning control device 50 having a neural network device and controlling the learning and photographing operation of the robot camera 30 are provided.

  In FIG. 1, a solid arrow indicates a signal flow during learning of the neural network device included in the learning control device 50, and a broken arrow indicates a signal during automatic control in which the learning control device 50 automatically controls the robot camera 30. Shows the flow. The term “learning” refers to a period during which learning data for learning the neural network device and test data for testing the learned neural network device are acquired, and the neural network device is learned with the learning data. A period including a period and a period for testing the learned neural network device with test data.

  The sensor camera 10 includes, for example, two camera devices provided at a predetermined interval, is fixed to the ceiling of a studio, and can perform stereo stereoscopic viewing by looking down at a person in a predetermined area. In addition, the sensor camera 10 is configured to output respective photographing signals obtained by photographing the person by the two camera devices to the subject detection device 20 every unit time, for example, every 17 milliseconds.

  The subject detection device 20 is configured by, for example, a computer that performs image processing. The subject detection device 20 is a stereo stereoscopic video signal from the sensor camera 10 and a virtual three-dimensional axis (x, y, z axis) that is virtually predetermined in the studio. The position of the subject is detected from the relationship between the subject position and the subject position data indicating the position of the subject is output to the learning control device 50.

  The robot camera 30 includes a camera device and a pan head device (hereinafter referred to as a “head device or the like”), and a moving device that moves to a shooting location designated by the robot camera operating device 40 or the learning control device 50. The photographing operation according to the control signal data from 40 (hereinafter referred to as “operation control signal data”) or the control signal data from the learning control device 50 (hereinafter referred to as “photographing operation control signal data”) is performed. ing. Specifically, the robot camera 30 controls the pan head device and the moving device based on the operation control signal data or the shooting operation control signal data, and performs a shooting operation such as pan, tilt, zoom, focus, and shooting location movement. It is like that.

  The robot camera operating device 40 is operated by a skilled cameraman, and includes, for example, a monitor that displays a captured image of the robot camera 30, an operation handle or a switch that remotely operates the pan head device of the robot camera 30, and the like. Yes. When the robot camera 30 is being operated by the robot camera operating device 40, data indicating the state of the shooting operation such as pan, tilt, zoom, focus, shooting location movement, etc. in the robot camera 30 (hereinafter referred to as "shooting operation state data"). Is output from the robot camera 30 to the learning control device 50.

  Next, the configuration of the learning control device 50 will be described with reference to FIG. FIG. 2 is a block diagram of the robot camera control device 1 according to the present embodiment, and shows a detailed configuration of the learning control device 50.

  The learning control device 50 includes a data acquisition unit 51 that acquires data, a data storage unit 52 that stores data, a calculation parameter input unit 53 that inputs calculation parameters, and a neural network device 100. Although not shown, the learning control device 50 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an input / output interface circuit, a storage device, a display, and the like. Consists of a computer. The learning control device 50 constitutes a robot camera control device and a learning device according to the present invention.

  The data acquisition unit 51 acquires the subject position data from the subject detection device 20 and the shooting operation state data from the robot camera 30 operated by the robot camera operation unit 40 for each unit time when the neural network device 100 learns. It is supposed to be.

  The data storage unit 52 evaluates the subject position data and the shooting operation state data acquired by the data acquisition unit 51 for learning data for learning the neural network device 100 or the performance of the learned neural network 110. It is stored as test data.

  The calculation parameter input unit 53 includes a step interval value input unit 53 a that inputs a step interval value of previous data when inputting data before the current time (hereinafter referred to as “previous data”) to the neural network device 100. A step number input unit 53b for inputting a number and a parameter input unit 53c for inputting calculation parameters other than the step interval value and the step number are provided. The calculation parameter input unit 53 constitutes a search area setting unit according to the present invention.

  The neural network device 100 includes, for example, a hierarchical neural network 110 including an input layer, an intermediate layer, and an output layer, input layer optimization means 120 that optimizes the input layer of the neural network 110, and learning evaluation of the neural network 110. And a learning evaluation unit 130 for performing.

  The input layer optimizing means 120 sets the step information setting unit for inputting the previous data to the input layer of the neural network 110 and the step interval value and the number of steps (hereinafter collectively referred to as “step information”). Part 122. The input layer optimization unit 120 constitutes a data input unit, an input layer optimization unit, and an optimum step information search unit according to the present invention.

  The data input unit 121 is configured to input learning data or test data from the data storage unit 52 during learning, and subject position data from the subject detection device 20 during automatic control. The data input unit 121 constitutes a data selection unit according to the present invention.

  Here, the data input unit 121 is configured to previously input data to the input layer of the neural network 110 when the cameraman captures a subject, rather than operating the camera only according to the instantaneous position of the subject. This is because the image is taken in consideration of the past position of the subject. Therefore, the step information setting unit 122 sets step information, and the data input unit 121 is configured to input previous data to the input layer of the neural network 110 according to the step information. The step information setting unit 122 constitutes a step interval value setting unit and a step number setting unit according to the present invention.

  Hereinafter, previous data and step information will be described with reference to FIG.

In FIG. 3, the points indicated by black circles represent the temporal positions of the shooting data shot every unit time. Also, currently represents the data of the object position at time t by S _t. The previous data, a data of the current time t earlier object position, time t, t-i, t- 2i, t-3i, ···, data of the corresponding object position respectively t-ui _{S t} , _St-i , _St-2i , _St-3i , ..., _St-ui . Here, the symbol i indicating the interval of each data indicates the step interval value, and in the example shown in FIG. 3, the step interval value i = 3. A symbol u indicating the total number of step interval values i indicates the number of steps.

Similarly to the subject position data S, the shooting operation state data r of the robot camera 30 when the cameraman operates the robot camera 30 via the robot camera operating device 40 is also the shooting operation state data r _t corresponding to each time. , Rt _-i , rt _-2i , rt _-3i , ..., _rt-ui . Similarly, the photographing operation status data R of the robot camera 30 is also captured operating condition data _R t corresponding to respective _{times, R t-i, R t} -2i, R t-3i, ···, the _{R t-ui} Including.

  Returning to FIG. 2, the description of the configuration is continued.

  The learning evaluation unit 130 evaluates the learning state of the neural network 110 from the output signal data of the neural network 110 with respect to the previous data. In addition, the learning evaluation unit 130 updates the weighting coefficients between the input layer and the intermediate layer and between the intermediate layer and the output layer in the neural network 110 according to the learning state of the neural network 110. . Note that the learning evaluation unit 130 constitutes a learning state evaluation unit according to the present invention.

  Next, the operation of the robot camera control device 1 in the present embodiment will be described. The operation of the robot camera control device 1 includes (1) an operation for acquiring learning data and test data, (2) an operation for generating an optimal model, and (3) an operation for automatically controlling the robot camera 30. These will be described in order below.

(1) Operation for Acquiring Learning Data and Test Data In FIG. 2, when a skilled cameraman operates the robot camera operating device 40, the robot camera operating device 40 outputs operation control signal data to the robot camera 30. The robot camera 30 shoots a subject based on the operation control signal data, and outputs shooting operation state data to the data acquisition unit 51 every unit time. In parallel, the subject detection device 20 detects the position of the subject based on the subject image captured by the sensor camera 10 and outputs subject position data to the data acquisition unit 51 for each unit time.

  Then, the data acquisition unit 51 generates learning data in which the subject position data is associated with the shooting operation state data corresponding to the subject position data, and stores the learning data in the data storage unit 52 in a predetermined file format. For example, when a csv (Comma Separated Values) file format is used as a file format for storing learning data, subject position data and shooting operation state data are associated with each other for each unit time, and each data is separated by a comma and arranged. It becomes a file.

  The test data is also acquired by the data acquisition unit 51 and stored in the data storage unit 52 in the same manner as the learning data.

(2) Operation for Generating Optimal Model FIG. 4 is a flowchart showing an operation for generating an optimal model.

  First, the calculation parameter input unit 53 sets input parameters (step S11). Specifically, the calculation parameter input unit 53 displays a GUI (Graphical User Interface) screen as shown in FIG. 5 on the display, and prompts the user to input parameters.

  The GUI screen shown in FIG. 5 specifies learning data specifying unit 101 for specifying learning data, test data specifying unit 102 for specifying test data, and file information for storing optimal model data of the neural network 110. An optimal model data specifying unit 103 to perform, an input variable check box 104 for specifying coordinate axes of position data input to the neural network 110, an output variable check box 105 for specifying coordinate axes to be controlled by the neural network 110, and a search for step information And a search area designating unit 106 for designating an area. The search area specifying unit 106 also includes an input unit 106a for the maximum past step value, an interval value specifying box 106b for specifying the interval value, an input unit 106c for the step interval value, and an interval value specifying for specifying the interval value. Box 106d. The maximum past step value is a value for determining the number of steps u, as will be described later. The interval value specification box 106b constitutes a second search interval setting unit according to the present invention. The interval value designation box 106d forms a first search interval setting unit according to the present invention.

  In the example shown in FIG. 5, among the variables related to the x-axis, y-axis, and z-axis, variables related to the x-axis and y-axis are designated as the input variables. As the output variable, a variable related to pan and tilt is specified among variables related to pan, tilt, zoom, focus, X axis, Y axis, and Z axis. Note that the variables related to the X axis, the Y axis, and the Z axis are for causing the moving device of the robot camera 30 to move the shooting location.

  In the search area designating unit 106, “5, 10-30, 50” is input to the maximum past step value input unit 106a, “5” is input to the interval value specifying box 106b, and “1-” is input to the step interval value input unit 106c. 5, 20 ”and“ 2 ”in the interval value designation box 106d, respectively, are described.

  Here, the symbol “,” indicates a logical product (AND) in the logical operation, and the symbol “−” indicates a target region. The symbol “-” is used because the characteristic of the neural network 110 is not greatly changed by a change of one step in the previous data. Therefore, the symbol “-” is used in the maximum past step value input unit 106a and the step interval value input unit 106c. The search interval can be roughly set according to the designation in the interval value designation box 106b and the interval value designation box 106d.

  Specifically, in the example illustrated in FIG. 5, “5”, “from 10 to 30”, and “50” are designated in the maximum past step value input unit 106 a. In addition, since “5” is designated in the interval value designation box 106b, “5”, “10”, “15”, “20”, “25”, “30”, “30” 50 "is designated.

  Similarly, “1-5, 20” is described in the step interval value input unit 106c, and “2” is described in the interval value designation box 106d. Therefore, as the step interval values, “1”, “3” "," 5 "and" 20 "are designated. The step interval value specified here corresponds to the step interval value indicated by symbol i in FIG. On the other hand, the number of steps indicated by the symbol u in FIG. 3 is obtained by dividing the maximum past step value by the step interval value. For example, when the maximum past step value = 50 and the step interval value i = 2, the number of steps u = 25. However, when the division is not divisible, it is preferable to use the maximum value among the divisible numbers. For example, when the maximum past step value = 50 and the step interval value i = 3, the number of steps u = 16 may be set. Note that the number u of steps may be directly input instead of inputting the maximum past step value.

  Returning to FIG. 4, using the parameters set in step S11, the step information setting unit 122 calculates a combination of all i and u from the search area (step S12). Specifically, the combinations of the step interval value i and the number of steps u in the search area set in FIG. 5 are as follows.

  (I, u) = (1,5), (3,1), (5,1), (1,10), (1,15), (1,20), (1,25), (1 , 30), (3, 3), (3, 5), (3, 6), (3, 8), (3, 10), (5, 2), (5, 3), (5, 4 ), (5,5), (5,6), (1,50), (3,16), (5,10), (20,1), (20,2)

  Subsequently, the input layer optimization unit 120 reads the learning data specified in step S11 from the data storage unit 52, and causes the neural network 110 to learn using the learning data for all combinations of i and u (step S13). ). In the present embodiment, an example in which learning processing is performed by an error back propagation method will be described and described with reference to FIG.

  As shown in FIG. 6, the input layer optimization unit 120 initializes the time variable t in step S21, and increases the time variable t until the time variable t becomes larger than (i × u) in steps S22 and S23. .

When the input layer optimization unit 120 determines that t ≧ (i × u) in step S <b> 22, the subject position data S _t , S _t−i , S _t−2i , S _t− of the learning data is sent to the neural network 110. _3i ,..., S _t-ui are input, and the neural network 110 calculates predicted shooting operation state data R _{t + 1} of the robot camera 30 by forward calculation (step S24).

Next, the learning evaluation unit 130 actually controls the predicted shooting operation state data R _{t + 1 of} the robot camera 30 obtained by the neural network 110 and the position at the time (t + 1) included in the learning data, that is, an experienced cameraman. is the time when the (t + 1) for the shooting operation state data r _{t + 1} of the robot camera 30 in, for example, calculates the sum e ² of the square error of the pan axis value and the tilt axis value (step S25).

  Next, the learning evaluation unit 130 updates the weighting coefficient of the neural network 110 by backward calculation (step S26).

  Next, the input layer optimization unit 120 determines whether or not the time variable t is equal to the shooting time T (step S27). The photographing time T information is designated in advance by learning data, for example.

  If it is not determined in step S27 that the time variable t is equal to the shooting time T, the input layer optimization unit 120 increments the time variable t (step S28), and the process returns to step S24.

  On the other hand, if it is determined in step S27 that the time variable t is equal to the shooting time T, the input layer optimization unit 120 determines whether learning of the neural network 110 has ended (step S29), and learning ends. If it is determined that learning has been completed, the learning process ends. If it is not determined that learning has ended, the process returns to step S21.

  Here, the learning process described above will be specifically described with reference to FIG. In order to simplify the description, i = 3, u = 5, and shooting unit time = 1 (second). In this case, FIG. 7 shows the transition of the previous data of the subject position to be learned. In this example, the input of the previous data of the subject position to the neural network 110 starts at (i × u) = 15 (seconds) after the first data (the black dot at the left end). The time is the start time (= 0), and the photographing end time T is 9 (seconds) later.

Is a previous data D ₀ in the subject position to enter at the start time. Since the time variable t is incremented in step S28 in FIG. _{6, D 1, D 2,} ···, it will be sequentially input data in the past and _{D 9.} Then, before after weighting coefficients are updated by the data D _9, whether the learning is completed it is determined in step S29. Here, when it is not determined that the learning has ended, the time variable t is initialized in step S21. This means that the previous data D ₀ to D ₉ are sequentially input again. In general, even if the previous data to be input is the same data, the learning state of the neural network 110 can be improved as learning is repeated.

Next, an example of a technique for determining whether or not learning of the neural network 110 has been completed in step S29 will be described. As shown in the present embodiment, when the neural network 110 is a hierarchical type, for example, the total value e ² obtained in step S25 for all the input / output patterns of the neural network 110 is the threshold E set in advance. ^The time when it becomes smaller than ² can be set as the end of learning. The threshold E is preferably determined according to the subject and the shooting scene.

  Returning to FIG. 4, for the neural network 110 learned as described above, evaluation values are calculated using test data for all combinations of i and u (step S14). An example of this process will be described with reference to the flowchart shown in FIG.

As shown in FIG. 8, the input layer optimization unit 120 initializes the time variable t in step S31, and increases the time variable t until the time variable t becomes larger than (i × u) in steps S32 and S33. At the same time, it is assumed that the total value e _t ² = 0 of the square error of each axis value in the pan head device of the robot camera 30 or the like.

If the input layer optimization unit 120 determines that t ≧ (i × u) in step S32, the subject network position data S _t , S _t−i , S _t−2i , S _{t− 3 t} ,... S _t-ui are input to the learned neural network 110, and the neural network 110 calculates predicted shooting operation state data R _{t + 1} of the robot camera 30 by forward calculation (step S34).

Next, the learning evaluation unit 130 actually controls the predicted shooting operation state data R _{t + 1 of} the robot camera 30 obtained by the neural network 110 and the position at the time (t + 1) included in the test data, that is, an experienced cameraman. is the time when the (t + 1) for the shooting operation state data r _{t + 1} of the robot camera 30 in, for example, calculates the sum e _t ² of square error of the pan axis value and the tilt axis value (step S35).

  Next, the input layer optimization unit 120 determines whether or not the time variable t is equal to the shooting time T (step S36). Note that the photographing time T is designated in advance by test data, for example.

  If it is determined in step S36 that the time variable t is not equal to the shooting time T, the input layer optimization unit 120 increments the time variable t (step S37), and the process returns to step S34.

  On the other hand, if it is determined in step S36 that the time variable t is equal to the shooting time T, the input layer optimization unit 120 calculates the evaluation value of the neural network 110 by Equation 1 (step S38), and the evaluation value calculation process Exit.

Note that the evaluation value of the neural network 110 is not limited to that obtained by this method. For example, a correlation coefficient between R _{t + 1} and r _{t + 1} in t = (i × u) to T may be used.

  Returning to FIG. 4, the input layer optimization unit 120 obtains step information (hereinafter referred to as “optimum step information”) from which the best evaluation value is obtained among the evaluation values obtained in step S <b> 14, that is, an optimum step interval value. In addition, the file information designated by the optimum model data designation unit 103 shown in FIG. 5 is used for each of i and u, which is a combination of the optimum number of steps, and optimum model data for which the best evaluation value is obtained. Then, the data is output to a storage device (not shown) (step S15), and the optimum model generation process is terminated.

(3) Operation for automatically controlling the robot camera 30 Next, an operation for automatically controlling the robot camera 30 will be described with reference to FIG.

  First, the input layer optimizing means 120 reads the data of the optimum step information and the optimum model data stored in the optimum model generation process from a storage device (not shown), and constructs the neural network 110 based on these. As a result, the number of units in the input layer of the neural network 110 is represented by the product of (u + 1) and the number of input variables.

  Next, the sensor camera 10 captures a subject by stereo stereoscopic vision and outputs the video signal to the subject detection apparatus 20 every unit time.

  Next, the subject detection apparatus 20 detects the position of the subject by performing image processing based on the relationship between the video signal obtained by stereo stereoscopic vision and the virtual three-dimensional axis that is virtually determined in the studio, and determines the position of the subject. The subject position data shown is sent to the data input unit 121 of the learning control device 50.

  Next, the data input unit 121 selects and inputs subject position data to the input layer of the neural network 110 based on the optimum step information for each unit time. For example, a case where the optimum step information is a step interval number i = 3 and a step number u = 2 is taken as an example. In this case, the data input unit 121 detects subject position data at the current time, subject position data three units before the current time, and six unit hours before the current time, from the subject position data sequentially input every unit time. The subject position data is selected and input to the input layer.

  The neural network 110 performs calculation for predicting the shooting operation state of the robot camera 30 based on the subject position data input by the input layer, and outputs the calculation result data to the robot camera 30 as shooting operation control signal data.

  Then, the robot camera 30 performs a shooting operation according to the shooting operation control signal data from the learning control device 50, such as a panning operation and a tilting operation.

  Note that by programming the processing of each step shown in FIGS. 4, 6, and 8 and operating the computer by the program, the program can cause the computer to function as the robot camera control device 1. In this case, the computer has the configuration shown in FIG.

  As described above, according to the robot camera control device 1 in the present embodiment, the configuration is such that the step interval value and the number of steps when inputting the previous data stepwise into the neural network are set, so that complicated work is required. Labor-saving can be achieved, and an optimal model can be constructed in a shorter time than before.

  In the above-described embodiment, the network apparatus according to the present invention has been described as an example applied to the imaging operation control of the robot camera. However, the present invention is not limited to this, for example, a moving person It can also be applied to a prediction device that predicts the position of an object.

  As described above, the neural network device according to the present invention, the robot camera control device using the neural network device, and the neural network program can save labor for complicated work and construct an optimum model in a shorter time than before. For example, it is useful as a neural network device that predicts and calculates the position of a moving object, a robot camera control device using the same, and a neural network program.

The figure which shows schematic structure of the robot camera control apparatus in one embodiment of this invention The block diagram of the robot camera control apparatus in one embodiment of this invention Explanatory drawing of the previous data and step information in the robot camera control apparatus in one embodiment of this invention The flowchart which shows the operation | movement which produces | generates the optimal neural network model in the robot camera control apparatus in one embodiment of this invention The figure which shows an example of the GUI screen for setting an input parameter in the robot camera control apparatus in one embodiment of this invention. The flowchart which shows the operation | movement which learns a neural network using the data for learning in the robot camera control apparatus in one embodiment of this invention. Specific explanatory diagram when learning a neural network using learning data in the robot camera control device according to the embodiment of the present invention. The flowchart which shows the operation | movement which calculates the evaluation value of a neural network using the test data in the robot camera control apparatus in one embodiment of this invention

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Robot camera control apparatus 10 Sensor camera 20 Subject detection apparatus 30 Robot camera 40 Robot camera operation device 50 Learning control apparatus (robot camera control apparatus, learning apparatus)
51 Data Acquisition Unit 52 Data Storage Unit 53 Calculation Parameter Input Unit (Search Area Setting Unit)
53a Step interval value input unit 53b Step number input unit 53c Parameter input unit 100 Neural network device 101 Learning data specification unit 102 Test data specification unit 103 Optimal model data specification unit 104 Input variable check box 105 Output variable check box 106 Search region Designation unit 106a Maximum past step value input unit 106b Interval value designation box (second search interval setting unit)
106c Step interval value input section 106d Interval value designation box (first search interval setting section)
110 neural network 120 input layer optimization means (data input means, input layer optimization unit, optimum step information search means)
121 Data input part (data selection part)
122 Step information setting unit (step interval value setting unit, step number setting unit)
130 learning evaluation part (learning state evaluation means)

Claims (4)

A neural network device for controlling a photographing operation of a robot camera equipped with a camera device and a pan head device for photographing a subject,
A neural network ;
A step interval value setting unit for setting a step interval value indicating an interval when inputting data before the current time stepwise into the neural network, a step number setting unit for setting a step number indicating the number of the intervals, A data selection unit that selects data before the current time to be input to the neural network based on the step interval value and the number of steps, and inputs data acquired every predetermined unit time to the neural network. Data input means,
Learning state evaluation means for evaluating the learning state of the neural network from the output signal data of the neural network for the data before the current time input to the neural network;
A search region for setting a search region for searching for an optimal step interval value and an optimal step number at which the learning state of the neural network becomes a predetermined learning state by narrowing down a combination of the step interval value and the step number Setting means;
Optimal step information search means for searching the search region for the optimal step interval value and the optimal step number;
A neural network device comprising: The search area setting means includes a first search interval setting unit that divides a step interval value search area for searching for the optimum step interval value based on the step interval value and sets a first search interval; 2. A second search interval setting unit that sets a second search interval by dividing a step number search area for searching for the optimum step number based on the step number. The neural network device described in 1. A robot equipped with the neural network device according to claim 1, a subject detection device for detecting a predetermined subject position and outputting subject position data, a camera device for photographing the subject, and a pan head device Learning including a camera, a robot camera control device that outputs shooting operation control signal data for controlling a shooting operation of the robot camera, and the subject position data and the shooting operation control signal data corresponding to the subject position data A learning device for learning the neural network based on the data for use,
The learning device inputs the learning data stepwise to the data input means based on the step interval value and the number of steps during learning for learning the neural network, and the learning state of the neural network device is To obtain an optimal step interval value and an optimal number of steps to be in a predetermined learning state,
The robot camera control device is configured to detect the subject position from the subject detection device based on the optimum step interval value and the optimum step number obtained by the learning device during the learning during automatic control after the neural network learns. Data is input to the data input means, and the output signal data of the neural network is output to the robot camera as the shooting operation control signal data to control the shooting operation of the robot camera. Camera control device. A neural network program that causes a computer to function as a neural network device that controls a photographing operation of a camera device that shoots a subject and a robot camera equipped with a pan head device,
The computer,
A neural network;
A step interval value setting unit for setting a step interval value indicating an interval when inputting data before the current time stepwise into the neural network, a step number setting unit for setting a step number indicating the number of the intervals, A data selection unit that selects data before the current time to be input to the neural network based on the step interval value and the number of steps, and inputs data acquired every predetermined unit time to the neural network. Data input means,
Learning state evaluation means for evaluating the learning state of the neural network from the output signal data of the neural network for the data before the current time input to the neural network;
A search region for setting a search region for searching for an optimal step interval value and an optimal step number at which the learning state of the neural network becomes a predetermined learning state by narrowing down a combination of the step interval value and the step number Setting means;
Optimal step information search means for searching the search region for the optimal step interval value and the optimal step number;
A neural network program characterized by functioning as a function.

Images