JPH0296724A - Camera with learning function - Google Patents

Abstract

PURPOSE:To attain photographing which fits to a user's individually by permitting a camera to learn a network in a learning mode, and controlling the camera in an automatic mode based on an output from the network. CONSTITUTION:The camera is provided with the network composed of an input layer, an intermediate layer, and an output layer which are combined in order with the prescribed strength of the combination. In the learning mode, a signal showing the environmental condition of surroundings such as the brightness distribution of an object is connected to the input layer composed of plural units. Here, the output layer is permitted to learn the strength of the combination between all layers which generate the desired output selected by a user. In the automatic mode, the camera is controlled based on the output of the learned network. Therefore, the photographing which fits to a user's individuality can be attained for any object.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a camera with a learning function that can photograph any subject that satisfies certain conditions.

[Prior Art] As a conventional example of such a camera, Japanese Patent Application Laid-Open No. 57-420
There is a camera described in Publication No. 26.

This is a multi-A camera that divides the subject into multiple areas, emits 71p+ light, and generates multiple bar outputs corresponding to each area.
In the PL light device, depending on the maximum brightness, select one of the maximum brightness, the intermediate brightness between the maximum brightness and the average brightness, the average brightness, the intermediate brightness between the average brightness and the minimum brightness, and the minimum brightness, and thereby Control exposure.

[Problems to be Solved by the Invention] This conventional example uses a microcomputer to recognize the subject pattern, so it may be applicable only to a certain limited number of subject patterns, or it may perform incorrect detection, resulting in unintended photographs. There was a drawback. Furthermore, when attempting to perform intended exposure control for many subject patterns, the subject patterns or brightness cannot be formulated, resulting in drawbacks such as an enormous program and a long computation time. Furthermore, even if a photograph is produced as intended by the manufacturer, the evaluation of the photograph largely depends on the individual's sensibilities, so good results may not necessarily be obtained for all users.

Furthermore, it is difficult to formulate the identification of the main parts of a subject for focusing and exposure control.

The purpose of this invention is to solve these drawbacks by having the network learn in learning mode to produce results that match each user's preferences, and in auto mode to adjust the camera based on the output of the network. The purpose of the present invention is to provide a camera with a learning function that can be controlled to produce results that match the user's individuality.

[Means for Solving the Problems] A camera according to the present invention is provided with a network consisting of a human layer, an intermediate layer, and an output layer that are sequentially connected with a predetermined connection strength. In the learning mode, a signal representing the surrounding environmental conditions such as the luminance distribution of the subject is sent to t! Connect to the human power layer consisting of i number of units. At this time, the output layer learns the strength of the connection between each layer so that the desired output selected by the user is generated. In auto mode, camera control is performed based on the output of the trained network.

[Operation] According to the present invention, the network is trained so that each user can obtain the desired output from the camera, so it is possible to adjust the exposure level according to the subject pattern or detect the focus based on the main part of the subject. can be tailored to suit the user's personality.

[Embodiment J] Hereinafter, an embodiment of a camera with a learning function according to the present invention will be described with reference to the drawings. FIG. 1 shows its block diagram. As can be seen from FIG. 1, this embodiment uses a neurocomputer to perform exposure control and focus detection. First, please refer to FIGS. 2 to 12. Explain about neurocomputer.

Figure 2 shows a model of a neurocomputer.

This model is Rumerhal)□ (RuIIethart
) and others, and is called the back propagation model (hereinafter abbreviated as BP model). A neurocomputer consists of many units (neurons), and the units are classified into a human layer, an intermediate layer, and an output layer. Each unit is connected in the direction of input layer-middle layer-output layer to form a network (neural net). The strength of the connections between each unit is determined by learning. However, there is no mutual connection between units within each layer. The model of each unit is
As shown in the figure.

Next, the principle of this BP model learning algorithm will be explained. When a certain pattern P is given to the input layer, the actual output value appearing in the output layer is Opjo, and the desired output value at that time (hereinafter referred to as the teacher signal) is [pj], then the difference between the two Epj is as follows. expressed.

Epj=1/2 (tpj-OpjO) 2...
- (1) To perform learning, it is sufficient to change the connection strength of all units so as to reduce this error Epj.

When pattern P is given, the strength of coupling W from the i-th unit of the (K-1) layer to the i-th unit of the layer
The amount of change in jj is defined as follows.

Here, K is set to 0 at the output layer and increases toward the input layer.

ΔpWj1−c −9E p/a WjiK−f
21aEp/aWjlK -(9E p/9 ne tpjK)・Can e
tpjK/1llWHK) - (3) Kokote, net pjK-ΣWjkK・Opk"It's a tear.

Second, if f is a sigmoid (SigTAold) function, 0pkK-f (ne tpkK) and Table 4)
Then, equation (3) is transformed as follows. The sigmoid function is shown in FIG.

a E pj/ f3 WjlK -1 δpjK・op+K"l...
(4) Here, δpjK is in the first layer. It is the backward f-1 amount of error, and δpJ ”=-aEl)j/9
This is netpjK. Therefore, equation (2) is transformed as follows. Here, η is a constant.

ΔpWjlK −η・δpjK・OplK“1...(5
) In the case of crab knit, Epj-1/2 (tpj
-0pJ), 0IIJ =f (netp
jO), so 50 2 ・0. 0 The backward propagation amount δpJ of the output layer is 1 as follows.

become.

δpjK , to 1-a　Ep　/9　ne　tpj (aopl　/anetpjK)I Equation (7) is a recursive function of δ.

A general formulation of ΔpWjlK is as follows.

ΔpWjlK (n+1) - ηδpJ・0piK+1+αΔpWjlK (n)
−(8), where ΔpWjlK (0)−0, and the mouth represents the number of times of learning. The second term on the right side of equation (8) is added to reduce error fluctuations and speed up convergence.

From equation (8), the bond strength is updated as follows.

wjiK(n+1) −Wji(n)+ΔpWjlK(n)(K−0,1
, 2,...) ... (9) Here, the sigmoid function fl is expressed as fl = 1/ (1+e ``''') = -(
10), it is fi' -ft (1-fi), so the amount of backward propagation can be simplified as shown in the following equation.

In the case of Kaninit: δpj' - 0pJ (1 - OpjO) (tpj - Opjo) ・
...(11), O For intermediate units.

δpjK = OpjK (1-OpjK) ・Σ(δpk −Wkj'-' (n+ 1)l
−1 in (12) As can be seen from the above, the calculation of ΔW starts from the output layer unit and moves to the intermediate layer unit.

In this way, learning proceeds in the opposite direction to the processing of input data.

Therefore, learning using the BP model is performed as follows.

First, data for learning is input and the results are output. Next, the strength of the coupling is changed to reduce the resulting error (the difference between the actual output and the teacher signal).

Then, input the learning data again. This action
Repeat until ΔW converges.

FIG. 5 shows the basic circuit configuration of the BP model.

A random access memory (hereinafter referred to as RAM) 1 stores the strength of connection Wjl, and consists of N pages 11 to N for each layer. The RAM 2 stores the variation ΔWj+ of the bond strength Wjl when the pattern P is given, and is composed of N pages k-1 to N. RAM3 is 2; backward propagation amount δpj of difference
It stores (N+1) pages from k-0 to N. R
A ki 4 stores the output value Opj of each unit, and k
It consists of (N'+1) pages from -0 to N. 5 is O
pj arithmetic circuit, 6 is δpj arithmetic circuit, 7 is ΔρWj
+ calculation circuit. 9 is a sequence controller that controls the entire sequence.

The learning process using the BP model shown in FIG. 5 will be explained.

Here, the operation when the BP model is simulated by a Neumann type computer will be explained with reference to the flowcharts of FIGS. 6 to 9. Figure 6 shows Opj
'7t1 calculation flowchart 1. FIG. 7 is a flowchart of δpj calculation, FIG. 8 is a flowchart of Wpj calculation, and FIG. 9 is a flowchart of learning level determination.

In step 1 (Sl), the coupling strength Wji in the RAMI is initialized to a random value. In step 2, the human power value Opj
"Set l to RAM 4 5, step knob 3 to step 9
Then, the unit output value OpjK is calculated by the calculation circuit 5 in order from the human power layer to the output layer.

Next, in steps 11 to 20 of FIG. 7, the arithmetic circuit 6 calculates the backward propagation amount δpj0 of the error in the output layer from the output value 0pj0 and the teacher signal tpj indicating the desired output according to equation (11).

Next, at steps 21 to 24 in FIG.
+0 Find (1). Note that the initial values ΔpWjiK (0) of ΔpWj+ are all 0.0. In step 25, the strength of coupling Wjlo (1) is determined by the arithmetic circuit 8 according to equation (9). From the above, OpJ of the output layer, δpj', Δp Wjio
, 0 (1), and WNo (1) are found. Thereafter, these are stored in RAMI~RA~14 after updating the river data.

Next, learn the middle layer. Flowchart 1 in Figure 7
Returning to ・, δpjO, obtained above using the er calculation circuit 6,
Wjio (1) and 0 stored in RAM4
Using pj0, the amount of backward propagation of error δpjK is determined. Next, in the flowchart of FIG.
The amount of change in the bond strength ΔpWjlK (1) is (8
), and the arithmetic circuit 8 calculates the coupling strength Wjl.
K (1) is determined according to equation (9). Similar to the output layer, the data obtained above is stored in RAMI to RAM4 in an updated form. The above flow is input layer (K − N
+ 1), and the first day's learning is completed in winter.

By performing the above learning multiple times (n), the strength of the connection between each unit V/j+ is determined, and when a human power 1aopj indicating a certain input pattern P is applied, a desired output value Ppj is obtained. network will be automatically formed.

FIG. 9 is a flowchart for determining the mean square error Ep between the actual output value Opj and the teacher signal tpj. The smaller this value is, the closer the actual output value will be to the desired output value. If the current Ep is smaller than a certain threshold value ε, the learning is finished, and if it is larger than ε, the learning is repeated.

Although learning for one input cover turn P has been described above, it is also possible to perform learning in which there are a plurality of input cover turns and a plurality of output cover turns corresponding to each pattern. It is also possible to learn to output a specific cover turn for a plurality of cover turns.

The BP model described above can be realized with the Neumann-type microcomputers that are currently widely used in consumer devices, but as it is, one of the major strengths of neurocomputers, which is the ability to speed up through parallel processing, will not be utilized. . Therefore, it is preferable that the processes shown in FIGS. 6 to 9 be performed in parallel by a plurality of computers.

FIG. 10 shows the configuration of a parallel processing system for this purpose.

! ! Two microprocessors P1 to Pn are connected to the postprocessor 11. The neural network, not shown in FIG. 2, is divided into n partial networks, and each is assigned to a microprocessor P1 to Pn. The host processor 11 controls mutual timing between the microprocessors Pl and Pn, and controls the timing between the microprocessors Pl and Pn.
~Pn is integrated to perform processing such as pattern recognition. Each microprocessor PI-
Pn is the output value O shown in FIG. 5 according to the calculation procedure described above.
Perform operations on multiple consecutive columns of pj. Therefore, the microprocessors Pl to Pn have δpj necessary to calculate the output values for which they are responsible.

There are 61 RAMs and arithmetic circuits for storing ΔW j I and W j i respectively. I! When the computation of the output values of all the units corresponding to J is completed, communication for updating data is performed while maintaining synchronization among the respective processors Pi to P11. The host processor 11 determines the learning achievement level and controls the mutual timing of the microprocessors P1 to Pn.

When performing pattern recognition shade processing based on the learned results, the necessary output value PpjO is finally determined by calculating the output from the human power layer shown in FIG. In this case as well, by executing distributed processing using a plurality of microprocessors as shown in FIG. 11, speeding up can be achieved due to the parallelism of the neural network.

Although the circuit shown in FIG. 5 is basically required in the learning process, the configuration can be greatly simplified if only the learning results are applied.

FIG. 11 shows the basic circuit configuration in this case.

The human data is outputted by sequentially performing the calculation 0pjK=f (ΣWjkK length·0pkK+1) via the human power unit 12 (for example, an A/D converter) to obtain output data OpJ. The coefficient memory 14 in which the coupling strength WjiK is stored may be a ROM or a rewritable ROM.

FIG. 12 is a schematic block diagram of a learning system during manufacturing for products to which learning results are applied.

Product 16 is R that stores the bond strength WNK.
Built-in OIvll 7. 18 is a learning device, R
The combination of OM17 and learning device 18 is basically the fifth
Although it is the same as the device shown in the figure, when the writing of WjiK to the ROM 17 is completed, the product 16 (ROM 17) and the learning device 18 are separated. Note that it is not necessary to perform learning each time for each product of the same type, so it is also possible to copy and use the ROM 17.

In the above explanation, the learning of the BP model and the application of the results have been realized through simulations using currently used Neumann type computers.

This is mainly because learning requires complex algorithms, and it is extremely difficult for hardware to automatically self-organize the weights of connections between neurons. However, if the strength of the connection Wlj is known, the BP model shown in Figure 1 can be configured in hardware if the learning results are applied to a machine.

If you want to increase speed through parallel processing or apply it to inexpensive consumer products, there is no point unless you use this method.

This can be achieved by configuring each unit in FIG. 2 with an inverter and replacing the coupling strength Wlj with a resistor network R1j, which can be easily achieved using recent LSI technology.

Next, a camera with a learning function to which the neurocomputer described above is applied will be explained with reference to FIG. The camera is an autofocus (AF) camera in which the photographic lens 20 can be moved in the optical axis direction by a combination device 171720a.

A diaphragm 19 is provided on the front surface of the photographing camera 20, and a subject image passing through the diaphragm 19 is incident on a light receiving section 21 having photoelectric conversion elements arranged in a matrix.

Therefore, the light receiving unit 21 outputs the brightness information of the subject in the narrowed down state for each photoelectric conversion element, digitizes it via the amplifier 22 and A/D converter 23, and converts it into a BV' value as the arithmetic circuit ( ALU) 24 +:
fl, Thshl. The eL arithmetic circuit 24 calculates the brightness BVI of the actual subject from the brightness BV' value in the narrowed down state.
This is a circuit for calculating ift (-BV'-AVo), and for this reason, the open aperture of the aperture 19 (ifJ A
Vo is being input.

The BV value for each photoelectric conversion element output from the a calculation circuit 24 is supplied to a selection circuit (multiplexer) 28, and a frame memory 40 that stores the output of each photoelectric conversion element
Also supplied. The selection circuit 24 receives a control signal PX, which will be described later.
Based on y, only the output BV value of the photoelectric conversion element corresponding to the main part of the subject is passed.

The output of the selection circuit 28 is supplied to two arithmetic circuits for focus detection that detect the sharpness and phase of the main part of the subject, and also the brightness BV of the main part of the subject.

Film sensitivity SV1 Aperture value AV, Shutter speed T
Apex calculation (BV+5V-TV+AV) is performed from V, and the signal is supplied to an arithmetic circuit 31 for determining the shutter speed or aperture value. The two calculation circuits perform calculations for focus detection using the hill-climbing method. The outputs of the two arithmetic circuits are supplied to the driver 30, and based on this, the driver 30 drives the focusing groove 20a to move the position of the photographic lens. The output of the arithmetic circuit 31 is supplied to a shutter control device 32 and an aperture control device 33 to control exposure. In this way, this embodiment performs exposure control and focus detection based on the luminance information of only the main part, not the entire subject. When the main part spans a plurality of photoelectric conversion elements, the arithmetic circuit 29.31 takes the average of the output BV values of the plurality of photoelectric conversion elements as the BV value of the main part.

The data in the frame memory 40 is input to a neurocomputer 25 having a learning function, which calculates a signal Pxy indicating the position of the main part from the brightness pattern of the subject. That is, the neurocomputer 25 uses human power to obtain the brightness pattern Opjo of the subject and finally outputs PXy. The basic block configuration of the neurocomputer 25 may be as shown in FIG. 5, but here, in order to increase the speed,
Learning is performed using a parallel computer as shown in FIG. The connection weights Wlj obtained as a result of learning are stored in the coefficient memory 26.

An operation section 27 is provided for manually inputting the position of the main part of the object for focus detection and exposure control during learning. The details of the operation section 27 are not shown, but it consists of touch panel switches and the like that correspond one-to-one with the positions on the screen, and the position of the main part of the subject on the screen is manually operated while looking at the finder.

By using a liquid crystal display instead of a finder and inputting information using a transparent touch panel provided integrally on the display section of the liquid crystal display, the main part of the subject can be selected more easily.

The output of the neurocomputer 25 and the output of the operation section 27 are connected to the selector 42. The selector 42 is switched depending on whether it is a learning mode or an auto mode in which the main parts of the subject are automatically detected after learning, and is connected to the operation unit 27 when in the learning mode, and connected to the neurocomputer 25 when in the auto mode. . 34 is a sequence controller.

Here, the learning mode will be explained. In order to proceed with learning efficiently, the neural network has an independent neural network S11 for each row, as shown in FIG.
．． Learning is performed in ... and integrated in the output layer SO. The neural network is composed of three layers: an input layer 37, an intermediate layer 38, and an output layer 39, and the learning principle is as described above.

Here, for simplicity of explanation, the photoelectric conversion element is the first
As shown in FIG. 4, it is assumed that they are arranged in 4 rows and 7 columns.

The position of this light 7L and the conversion element corresponds to the position pxy of the main part.

FIG. 15 shows a specific example of a subject. In the case of the portrait shown in (a), the main part of the subject is the photoelectric transformer -'fP3
3, in the case of (b), the photoelectric conversion element Pa5, (c
), it is assumed that the photoelectric conversion element P34 corresponds to the main part.

By providing the neurocomputer 25 with such a brightness pattern of the subject as an input via the frame memory 40, and providing the positions of these main parts as a teacher signal from the operation unit 27, the neurocomputer 25 learns the strength of the connection.
No matter what the subject pattern is, it will be possible to automatically output the correct main part. That is,
In the neurocomputer 25, the strength of the connection between each layer Wjl is self-organized by the above-described algorithm so that the output becomes equal to the teacher signal.

Note that the signal input from the operation unit 27 indicating the position of the main part is also supplied to the selection circuit 28, so even in the learning mode, the signal of the main part is selected and the two arithmetic circuits for focus detection and the two arithmetic circuits for exposure control are The signal is supplied to the arithmetic circuit 31. Therefore, learning can be done together with actual photography. 1st
Although Figure 5 shows only three objects, in reality, learning is repeated for hundreds of patterns. As a result, the number 6
While photographing the subject pattern, the main parts of the subject can be learned according to the user's preference.

Furthermore, the neurocomputer 25 has the excellent property of outputting correct output even for patterns that were not input during learning after a certain amount of learning. It is very effective in solving difficult problems. The neurocomputer's learning allows it to self-organize the relationship between the subject's main parts and a huge variety of subject patterns that could not previously be programmed with a Neumann-type computer, making it possible to control exposure and detect focus as intended. Furthermore, since neurocomputers can perform high-speed calculations through parallel processing, they are suitable for cameras that require speed.

After a predetermined number of frames have been photographed, it is assumed that learning is completed, the selector 42 is connected to the neurocomputer 25, the main part of the subject is automatically determined from the subject pattern based on the learning result, the focus is adjusted based on the brightness BV of the main part, Or control exposure.

This invention is not limited to the embodiments described above, and can be modified in various ways. The above explanation deals with the case where the neurocomputer learns the main parts of the subject and performs focus detection or exposure control. It is also possible to perform learning such as backlight correction. Furthermore, by inputting not only the brightness of the subject but also temperature, humidity, etc. as manual parameters of the neurocomputer, it is possible to learn to create subtle seasonal sensations that are difficult to formalize.

(Effects of the Invention) As explained above, according to the present invention, in the learning mode, the network is trained to produce results that match the preferences of each user, and in the auto mode, the network is trained based on the output of the network. By controlling the camera, it is possible to provide a camera with a learning function that can photograph any subject in a way that suits the user's personality.

[Brief explanation of the drawing]

Fig. 1 is a block diagram of an embodiment of a camera with a learning function according to the present invention, Fig. 2 is a diagram showing a model of a neurocomputer used in the embodiment, and Fig. 3 is a model of each unit making up the network. Figure 4 is a diagram showing a sigmoid function, Figure 5 is a block diagram of a neurocomputer, Figures 6 to 9 are flowcharts when the neurocomputer in Figure 5 is simulated with a Neumann computer, Figure 6 shows the output O of each unit.
Figure 7 is a flowchart for determining the backward propagation of error 2 δpj, Figure 8 is a flowchart for determining the coupling strength coefficient Wjl, Figure 9 is a flowchart for determining the level of learning, Figure 10. is a block diagram of a parallel processing system, Figure 11 is a block diagram of a device that applies learning results, Figure 12 is a block diagram of a system that trains a device that applies learning results, and Figure 13 is a network diagram of the embodiment. FIG. 14 is a diagram showing an example of the arrangement of charging conversion elements of the embodiment, and FIGS. 15(a) to (c) are diagrams showing examples of objects to be studied. 20 a... Focusing device for sound, 21... Light receiving section, 24,
29° 31...m'J5 circuit, 25... Neurocomputer, 26... Coefficient memory, 27... Operating unit, 28... Selection circuit, 32... Shutter control device,
33... Aperture control device, 40... Frame memory.

Claims (1)

[Claims] A network that determines an output using a signal representing the environmental state around the camera as a parameter in a learning mode, and the network is self-organized so that the desired output value selected by the user matches the output value of the network. A camera with a learning function, comprising a learning device and a control means for controlling the camera based on the output of the network in auto mode.