JP2008177656A - Imaging unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an imaging unit capable of appropriately performing a remote operation support by facilitating an appropriate communication between an operator and an instructor by reducing differences between an object of operation that the operator is viewing and an image of the object of operation which is being viewed by the instructor and captured by an electronic camera to enable the operator and instructor to share the same image information. <P>SOLUTION: The imaging unit controls an exposure value based upon a previously stored exposure value or focuses an image of a subject based upon a previously stored focusing position in a direction that an imaging optical system faces to reduce differences between the object of operation that the operator is viewing and the image of the object of operation which is being viewed by the instructor and captured by the electronic camera to enable the operator and instructor to share the same image information, so the imaging unit provides accurate remote operation assistance by accelerating accurate communication between the operator and instructor. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an imaging unit, and in particular, controls an exposure value based on a previously stored exposure value in a direction in which the imaging optical system faces, or stores it in a direction in which the imaging optical system faces. The present invention relates to an imaging unit that focuses an image of a subject based on the focused position.

  Conventionally, an image that is detachably mounted on the observer's head or face and displayed on an image display element such as a small liquid crystal display element is projected directly onto the observer's eyeball by an eyepiece optical system, and the image is as if A head-mounted image display device that enables observation of a virtual image as if it has been enlarged and projected in the air, that is, an HMD (Head Mount Display) is known. The HMD is used as a display device for viewing a content image such as a DVD (Digital Video Disc) or a video, or for remote operation in an industrial device or a medical device, and has a wide variety of uses.

  For remote operation, an operator wears an HMD equipped with an electronic camera, and an operator at a remote location observes an image of a work object captured by the electronic camera on a monitor such as a host computer. Various remote work support systems have been proposed (for example, patents) that send and provide appropriate information by an image to an HMD worn by a person and perform work instructions in combination with voice instructions if necessary (for example, patents) Reference 1).

  In such a remote operation support system, the operator's head frequently moves in a so-called camera shake state with the work performed by the worker, and the automatic exposure control of the electronic camera is performed each time. (Hereinafter referred to as AE) and automatic focus adjustment (hereinafter referred to as AF) functions, and the exposure and focus of the image on the monitor viewed by the remote instructor often changes, resulting in an extremely This makes it difficult to share information between the worker and the instructor and to support accurate remote work.

  Therefore, even when camera shake occurs, the movement state of the captured image is detected so that the automatic focus adjustment means frequently operates and an unsightly captured image is not obtained. There has been proposed a video camera provided with means for prohibiting the operation of the automatic focus adjustment means by determining that a camera shake is caused when the drive mode satisfies a predetermined condition (see, for example, Patent Document 2).

  In addition, there has been proposed an autofocus device for a video camera that resumes autofocus operation when panning and tilt are detected even when autofocus operation is stopped to prevent the influence of a crossing subject (for example, a patent) Reference 3).

Furthermore, in order to eliminate unnecessary autofocus operation during the movement of the camera position, a camera control method for a video conference terminal that stops the autofocus operation while the swivel head is operating has been proposed (for example, Patent Documents). 4).
JP 2002-132487 A JP-A-5-336426 Japanese Patent Laid-Open No. 6-153053 JP-A-7-274054

  According to the methods proposed in Patent Documents 2 to 4, a certain effect can be expected as a response to a so-called camera shake state. However, for example, when the operator moves his / her neck to observe the work object from a different angle, the exposure and focus of the image of the electronic camera are shifted, and the AE function and AF function respond to the exposure and focus. Until the image is restored and the image becomes normal, the instructor at the remote location cannot cope with the problem that the same image information as the worker cannot be shared.

  Therefore, the work object visually recognized by the worker is different from the image of the work object captured by the electronic camera viewed on the monitor by the instructor in the remote place, that is, proposed in Patent Documents 2 to 4. This method does not provide a solution to the problem that the information held by the operator and the instructor is different and hinders accurate communication between the operator and the instructor.

  The present invention has been made in view of the above circumstances, and an image of a work object captured by an electronic camera viewed on a monitor by an instructor in a remote place is different from a work object viewed by an operator. By reducing the state as much as possible, the operator and the instructor can share the same image information, promote accurate communication between the operator and the instructor, and provide accurate remote work support. An object of the present invention is to provide an imaging unit capable of performing the above.

  The object of the present invention can be achieved by the following constitution.

1. An image sensor for capturing an image of a subject;
An imaging optical system for forming an image of a subject on the imaging surface of the imaging device;
An exposure control unit for controlling an exposure value of an image of a subject imaged by the imaging device;
In an imaging unit comprising: a focus control unit for moving the position of the imaging optical system to focus an image of a subject on the imaging surface of the imaging element;
An orientation change detection unit for detecting a change in orientation of the imaging optical system;
A direction calculation unit that calculates a direction of the imaging optical system based on a detection result of the direction change detection unit;
At the storage position corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit, the exposure value controlled by the exposure control unit in that orientation and the imaging optical system controlled by the focus control unit A storage unit for storing the in-focus position;
Causing the exposure control unit to perform exposure control based on the exposure value in the direction stored in the storage position corresponding to the direction of the imaging optical system calculated by the direction calculation unit of the storage unit;
Imaging that causes the focusing control unit to perform focusing control based on the in-focus position stored in the storage position corresponding to the orientation of the imaging optical system calculated by the orientation calculating unit of the storage unit An imaging unit comprising a control unit.

  2. 2. The exposure control unit according to claim 1, wherein the exposure control unit sets the exposure value to the exposure value stored in the storage unit, and then controls the exposure value using the exposure value stored in the storage unit as a starting point. Imaging unit.

  3. The focus control unit sets the position of the imaging optical system to the focus position stored in the storage unit, and then uses the focus position stored in the storage unit as a starting point to capture the image of the subject in the image sensor. 2. The imaging unit according to 1, wherein the imaging unit is focused on.

4). When the exposure value corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit is not stored in the storage unit, the exposure control unit starts the exposure value at the time of the previous imaging as a starting point. Control the exposure value,
The imaging unit according to 1 or 2, wherein the storage unit stores the exposure value in a storage position corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit after completion of exposure control.

5. The focus control unit is configured to focus at the time of previous imaging when the in-focus position of the imaging optical system corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit is not stored in the storage unit. Focus the image of the subject on the imaging surface of the image sensor starting from the position,
The said storage part memorize | stores the focus position of the said imaging optical system in the memory | storage position corresponding to the direction of the said imaging optical system calculated by the said direction calculation part after focusing. Imaging unit.

  6). 6. The storage device according to claim 1, wherein the storage unit constantly updates the exposure value and the focus position stored in the storage position corresponding to the orientation of the imaging optical system to the latest value. Imaging unit.

  7. The imaging unit according to any one of 1 to 6, wherein an exposure value and a focus position stored in the storage unit are initialized when the imaging unit is powered on.

  According to the present invention, the exposure value is controlled based on the previously stored exposure value in the direction in which the imaging optical system is facing, or the previously stored focus in the direction in which the imaging optical system is facing. By focusing the image of the subject based on the position, the work object viewed by the worker and the image of the work object captured by the electronic camera viewed on the monitor by the remote operator are displayed. Since the number of different states can be reduced as much as possible, it is possible for the operator and the instructor to share the same image information, promoting accurate communication between the operator and the instructor, and providing accurate remote work support. An imaging unit capable of performing the above can be provided.

  Hereinafter, the present invention will be described based on the illustrated embodiment, but the present invention is not limited to the embodiment. In the drawings, the same or equivalent parts are denoted by the same reference numerals, and redundant description is omitted.

  First, the configuration of the remote work support system 1 will be described with reference to FIG. FIG. 1 is a schematic diagram showing an outline of the overall configuration of the remote work support system 1.

  In FIG. 1, a remote work support system 1 is composed of an HMD 2, a server device 4, and the like. The server device 4 includes a host computer 5, an image / audio recording server 6, a content server 7, and the like.

  The host computer 5 is connected to the image / sound recording server 6 and the content server 7 via, for example, the network EN. It is connected via a line network PN or the like.

  In the remote work support system 1 having such a configuration, the operator B wears the HMD 2 including the imaging unit 28, which will be described in detail later. While observing on the monitor 53 of the server device 4 provided on the ground, the server device 4 provides the worker B with appropriate information and instructions using images and sounds to perform the work support SPT. The work performed by the worker B in response to the work support SPT is recorded in the image / sound recording server 6 as a work record RPT.

  In a scene where the worker B works using the remote work support system 1, if the direction in which the worker B is facing is the same, it is considered that the brightness and distance of the subject do not change significantly. If the direction of the image pickup unit 28 is the same, the exposure and focus position of the image pickup unit 28 in that direction may be considered not to change significantly.

  Therefore, the orientation of the image pickup unit 28 is detected, the exposure and focus position of the image pickup unit 28 are matched with the previous exposure and focus position in that orientation, and the AE and AF are set based on the exposure and focus position. By going and fine-tuning the exposure and focus, it is possible to quickly obtain an image with exposure and focus. Therefore, it is possible to reduce as much as possible the state where the work object visually recognized by the worker B and the image of the work object captured by the imaging unit 28 viewed on the monitor by the instructor A at a remote location are different. . The present invention is devised based on this idea.

  Next, the structure of HMD2 mentioned above is demonstrated using FIG. FIG. 2 is a schematic diagram showing the overall configuration of the HMD 2.

  In FIG. 2, the HMD 2 includes temples 21R and 21L, a bridge 22, nose pads 23R and 23L, transparent substrates 24R and 24L, a display unit 25, an imaging unit 28, earphones 29R and 29L, a microphone 30, a control unit 31, and the like. is doing.

  The HMD 2 projects information such as images and characters transmitted from the server device 4 directly onto the wearer's eye by the display unit 25, thereby observing a virtual image as if the image was enlarged and projected in the air. It is possible.

  The temples 21R and 21L are long members made of a flexible elastic material or the like. The temples 21R and 21L are hooked on the wearer's ears or the temporal region, and are held on the head of the wearer by the wearer of the HMD2. This is for adjusting the mounting position. The temples 21R and 21L are configured to be rotatable in the directions of arrows P and Q in the rotating portions 21Ra and 21La. When the HMD 2 is not used, the temples 21R and 21L are rotated in the directions of the arrows P and Q. It can be made compact by moving it along the transparent substrates 24R and 24L.

  The bridge 22 includes nose pads 23R and 23L that hold the HMD 2 on the face of the wearer, and is a short rod-like member that spans a predetermined position on the transparent substrates 24R and 24L facing each other, and is transparent to the transparent substrate 24R. The substrate 24L is held in a relative positional relationship with a fixed gap. The transparent substrates 24R and 24L are substantially flat transparent bodies that each form a U-shaped space 24Rs at a position corresponding to one eyeball. An eyepiece optical system 27 is fitted in a U-shaped space 24Rs formed by being surrounded by the transparent substrate 24R.

  The display unit 25 includes an LCD display unit 26 that displays information such as images and characters transmitted from the server device 4, an eyepiece optical system 27 that directly projects the information displayed on the LCD display unit 26 onto the wearer's eyeball, and the like. And displays an image transmitted from the server apparatus 4 via a communication unit 36 (described later) provided in the control unit 31. The imaging unit 28 will be described in detail with reference to FIG. Earphones 29R and 29L are inserted into the ears of the wearer and provide sound to the wearer. The microphone 30 converts the voice of the wearer into an audio signal.

  The control unit 31 includes a microcomputer and has a power switch 351, an operation switch 352, and the like, and displays on the HMD 2 in response to signals from these various operation switches, control signals from the server device 4, and the like. Controls overall operations and imaging operations. In this example, the imaging unit 28, the display unit 25, and the control unit 31 are illustrated as separate bodies. However, the control unit 31 may be integrated with the imaging unit and the display unit 25.

  Next, the configuration of the HMD 2 will be described with reference to FIG. FIG. 3 is a block diagram showing the internal configuration of the HMD 2.

  In FIG. 3, the HMD 2 includes a display unit 25, an imaging unit 28, a control unit 31, and the like.

  The display unit 25 has been described in detail with reference to FIG.

  The imaging unit 28 includes an imaging optical system 281, an imaging element 282, an AFE (Analog Front End) 283, an image processing unit 284, an imaging control unit 285, an AF motor 286, an aperture 287, and a horizontal direction (pan direction; hereinafter referred to as “P direction”). A sensor 288, a vertical direction (tilt direction; hereinafter referred to as a T direction) sensor 289, a storage unit 285m, and the like.

  The light from the subject condensed by the imaging optical system 281 is adjusted in light quantity by the diaphragm 287 and imaged on the imaging surface of the imaging element 282. The formed subject image is photoelectrically converted by the image sensor 282 and captured at a shutter speed by the electronic shutter function of the image sensor 282. The image captured by the image sensor 282 is subjected to analog / digital conversion (hereinafter referred to as A / D conversion) after gain adjustment by the AFE 283, and digital image data 284a subjected to predetermined image processing by the image processing unit 284. And transmitted to the control unit 31 via the imaging control unit 285.

  An image captured by the image sensor 282 is A / D converted after gain adjustment by the AFE 283 and used for AE and AF in the image capturing control unit 285. Although AE is performed by controlling the aperture value of the aperture 287 and the electronic shutter speed of the image sensor 282, a mechanical shutter may be used instead of the electronic shutter function of the image sensor 282. In addition, AF is performed by moving the position of the imaging optical system 281 in the optical axis direction, but the position of the imaging element 282 may be moved in the optical axis direction. Details will be described in detail after FIG.

  Here, the image pickup optical system 281, the image pickup element 282, the AFE 283, the image pickup control unit 285, and the AF motor 286 function as a focus control unit in the present invention, and the image pickup element 282, AFE 283, image pickup control unit 285, and diaphragm 287 are included in the present invention. Functions as an exposure control unit.

  Further, the P-direction sensor 288 and the T-direction sensor 289 are constituted by angular velocity sensors, for example, and detect which direction the imaging optical system 281 is currently facing by integrating the angular velocity for a certain period of time. If the current orientation of the imaging optical system 281 detected by the P-direction sensor 288 or the T-direction sensor 289 is different from the orientation previously detected in the same manner by a predetermined value or more, the imaging optical system 281 It is determined that the orientation has changed, and a predetermined process is performed. The detection, determination, and predetermined processing are controlled by the imaging control unit 285. This will also be described in detail after FIG.

  Here, the P direction sensor 288, the T direction sensor 289, the imaging control unit 285, and the storage unit 285m function as an orientation change detection unit in the present invention, and the imaging control unit 285 and the storage unit 285m function as an orientation calculation unit in the present invention. To do.

  The storage unit 285m stores various types of data when the above-described imaging control unit 285 determines AE or AF, changes in the orientation of the imaging optical system 281 using the P direction sensor 288, and the T direction sensor 289, predetermined processing, and the like. Used for For example, the direction of the imaging optical system 281 and the AE control value (hereinafter referred to as the stored AE value SAE) and the AF control value (hereinafter referred to as the stored AF value SAF) during imaging in that direction are stored. This will be described in detail after FIG.

  The control unit 31 includes a control unit 32, an image memory 33, an operation unit 35, a communication unit 36, and the like. The control unit 32 reads a ROM (Read Only Memory) that stores each control program (not shown), a RAM (Random Access Memory) that temporarily stores data such as arithmetic processing and control processing, and a control program from the ROM. CPU (central processing unit) and the like that are executed in response to the signals from the operation switches of the operation unit 35 and the control signals from the server device 4 described in FIG. Controls overall operations and image signal processing operations.

  The image memory 33 temporarily stores the digital image data 284a when transmitting the digital image data 284a transmitted from the imaging unit 28 to the server device 4 via the communication unit 36 and the public line network PN. When displaying information such as images and characters transmitted from the server device 4 via the network PN and the communication unit 36 on the LCD display unit 26 of the display unit 25, information such as images and characters is temporarily stored. It is a temporary memory for

  The communication unit 36 connects the HMD 2 to the public line network PN, communicates with the server device 4, and inputs / outputs images, sounds, various control signals, and the like.

  Next, the storage direction of the imaging optical system 281 in the storage unit 285m in the P direction and the T direction and storage of the stored AE value SAE and the stored AF value SAF during imaging in that direction will be described with reference to FIG. FIG. 4 is a schematic diagram illustrating the configuration of the storage table PTT that stores the P-direction and T-direction orientations of the imaging optical system 281 and the storage AE value SAE and storage AF value SAF in the orientations. The stored AE value SAE is a stored exposure value in the present invention, and the stored AF value SAF is a stored in-focus position in the present invention.

  In FIG. 4, the storage table PTT is a matrix table in the P direction (for example, a range from −180 ° to + 180 °) and the T direction (for example, a range from −90 ° to + 90 °). The eye is a storage position corresponding to the orientation of the imaging optical system 281 in the present invention. For example, at the position PT (0, 0) where the P direction = T direction = 0 ° of the storage table PTT, the shutter speed SS00 and the aperture value F00, which are the stored AE values SAE in the direction of P direction = T direction = 0 °, Further, the position d00 in the optical axis direction of the imaging optical system 281 that is the stored AF value SAF is stored.

  Similarly, at the position PT (m, n) of the storage table PTT, for example, in the P direction = m ° and the T direction = n °, the shutter speed SSmn and the aperture value Fmn, which are the stored AE values SAE in that direction, and the storage are stored. The position dmn in the optical axis direction of the imaging optical system 281 that is the AF value SAF is stored. The step in the P direction and the T direction of the table may be appropriately set according to the performance required from the remote work support system 1 to the HMD 2.

  Each storage value stored in the storage table PTT may be initialized, for example, in the initialization step of the HMD 2 described later with reference to FIG. 5, or if the HMD 2 is always used in the same situation, it is not initialized. In addition, it may be updated at any time according to use. The storage and update of each stored value will be described in FIG.

  Next, embodiments of AE and AF of the imaging unit 28 will be described with reference to FIGS. FIGS. 5 to 9 are flowcharts showing embodiments of AE and AF of the image pickup unit 28. FIG. 5 is a main routine, FIG. 6 is a subroutine of FIG. 5, and FIGS. 7 to 9 are subroutines of FIG.

  Here, in order to minimize the state where the work object visually observed by the worker and the image of the work object captured by the imaging unit 28 viewed on the monitor by the instructor in the remote place are different, the imaging is performed. A method for performing AE and AF based on the exposure value in the direction in which the unit 28 faces and the past stored value of the in-focus position will be described.

  In FIG. 5, when the power switch 351 on the control unit 31 is turned on in step S101 and the HMD 2 is powered on, the HMD 2 is initialized in step S111. As described in FIG. 4, in this step, the storage AE value SAE and the storage AF value SAF stored in the storage table PTT of the storage unit 285m may be initialized. In the case of initialization, the stored AE value SAE and the stored AF value SAF are stored and accumulated in the storage table PTT of the storage unit 285m each time the orientation of the imaging optical system 281 changes in the subsequent imaging operation, or It will be overwritten and updated.

  For example, when the HMD2 is repeatedly used in the same situation for beginner education etc., if the stored AE value SAE and stored AF value SAF stored in the storage table PTT are left as they are without being initialized, the power is turned on. Immediately thereafter, AE and AF using the stored AE value SAE and the stored AF value SAF can be performed.

  Next, the imaging operation of the imaging unit 28 is started in step S113. In step S120, the “AE / AF subroutine” shown in FIG. 6 is executed, and the subject is in focus and in exposure. Subsequently, in step S131, one frame of moving image is captured, and in step S133, the single frame moving image captured in step S131 is transmitted to the server device 4 via the communication unit. In step S135, it is confirmed whether or not imaging has been performed for a predetermined number of frames. Steps S131 to S135 are repeated until a predetermined number of frames is captured.

  When the predetermined number of frames has been imaged (step S135; Yes), in step S141, it is confirmed whether or not to end the imaging. The end of imaging is instructed by, for example, turning off the power switch 351 on the control unit 31 or transmitting an end command from the server device 4. When the imaging is finished (step S141; Yes), a series of operations is finished as it is. When the imaging is not finished (step S141; No), the process returns to step S120, the “AE / AF subroutine” is executed, and the above-described operation is repeated thereafter.

  Here, it has been described that step S120 “AE / AF subroutine” is executed for each predetermined number of frames. However, the present invention is not limited to this. For example, the P direction sensor 288 or the T direction sensor 289 controls the movement of the imaging optical system 281. If detected, step S120 “AE / AF subroutine” may be executed as needed.

  FIG. 6 shows the step S120 “AE / AF subroutine” of FIG.

  6, first, step S200 “movement detection subroutine” shown in FIG. 7 for detecting a change in the orientation of the imaging optical system 281 is executed. Subsequently, in step S211, it is confirmed whether or not a change in the orientation of the imaging optical system 281 is detected in step S200. Whether or not a change in direction has been detected is confirmed by whether or not at least one of a P-direction movement flag Fp and a T-direction movement flag Ft, which will be described later with reference to FIG. 7, is set.

  If no change in direction is detected (step S211; No), step S220 “normal AE subroutine” shown in FIG. 8 is executed, and then step S230 “normal AF subroutine” shown in FIG. 9 is executed. Proceed to step S291. The shutter speed and aperture value, which are the starting points of the AE operation in step S220, are the shutter speed and aperture value used in the immediately preceding imaging operation. The position in the optical axis direction of the imaging optical system 281 that is the starting point of the AF operation in step S230 is the position in the optical axis direction of the imaging optical system 281 in the immediately preceding imaging operation.

  When a change in the orientation of the imaging optical system 281 is detected (step S211; Yes), in step S241, a stored AE value corresponding to the current orientation of the imaging optical system 281 detected in step S200 in the P direction and the T direction. SAE is read from the storage table PTT of the storage unit 285m. In step S251, it is confirmed whether or not the stored AE value SAE can be read from the storage table PTT.

  If the stored AE value SAE already exists on the storage table PTT (step S251; Yes), the aperture value of the aperture 287 of the imaging unit 28 and the shutter speed of the image sensor 282 are read in step S261. Are set to an aperture value and a shutter speed that satisfy the stored AE value SAE, and starting from the aperture value and the shutter speed, step S220 “normal AE subroutine” is executed.

  Thus, by using the stored AE value SAE that is most likely to be the appropriate exposure in the current orientation of the imaging optical system 281 as the starting point of AE, the exposure of the image when the orientation of the imaging optical system 281 is changed. That is, the state where the image of the work object captured by the electronic camera viewed on the monitor by the instructor at the remote place is different as much as possible.

  When the stored AE value SAE cannot be read, that is, when the stored AE value SAE does not exist on the storage table PTT (step S251; No), the starting point is the aperture value and shutter speed in the immediately preceding imaging operation. Step S220 “normal AE subroutine” is executed.

  Next, in step S263, the stored AF values SAF corresponding to the current P direction and T direction of the imaging optical system 281 detected in step S200 are read from the storage table PTT of the storage unit 285m. In step S271, it is confirmed whether or not the storage AF value SAF can be read from the storage table PTT.

  If the stored AF value SAF already exists on the storage table PTT (step S271; Yes), the position in the optical axis direction of the imaging optical system 281 is stored in the storage AF value SAF in step S281. From this position, step S230 “normal AF subroutine” is executed.

  Thus, by using the stored AF value SAF that is most likely to be in focus in the current orientation of the imaging optical system 281 as the AF starting point, the focus of the image when the orientation of the imaging optical system 281 changes is used. That is, the state where the image of the work object captured by the electronic camera viewed on the monitor by the instructor at the remote place is different as much as possible.

  When the stored AF value SAF cannot be read, that is, when the stored AF value SAF does not exist on the storage table PTT (step S271; No), the optical axis direction of the imaging optical system 281 in the immediately preceding imaging operation Step S230 “Normal AF Subroutine” is executed starting from the position of, and the process proceeds to Step S291.

  When the stored AE value SAE corresponding to the current P-direction and T-direction orientations of the imaging optical system 281 detected in step S200 “movement detection subroutine” exists in the storage table PTT of the storage unit 285m in step S291 Is updated by overwriting the stored AE value SAE on the storage table PTT with the latest AE value detected in step S220 “normal AE subroutine”.

  When the storage AE value SAE corresponding to the current P direction and T direction of the imaging optical system 281 does not exist on the storage table PTT, the latest AE value detected in step S220 “normal AE subroutine” is captured. A new storage AE value SAE corresponding to the current P direction and T direction of the optical system 281 is newly stored on the storage table PTT. The stored AE value SAE may be an Ev value that is an exposure constant, or may be a set of an aperture value and a shutter speed that satisfy the Ev value.

  Subsequently, in step S293, the stored AF values SAF corresponding to the current P-direction and T-direction directions of the imaging optical system 281 detected in step S200 “movement detection subroutine” are stored on the storage table PTT of the storage unit 285m. If it exists, the stored AF value SAF on the storage table PTT is overwritten and updated with the latest AF value detected in step S230 “Normal AF subroutine”.

  If there is no stored AF value SAF corresponding to the current P direction and T direction of the imaging optical system 281 on the storage table PTT, the latest AF value detected in step S230 “normal AF subroutine” is captured. A new stored AF value SAF corresponding to the current P direction and T direction of the optical system 281 is newly stored on the storage table PTT, and the process returns to step S120 of the main routine of FIG. The stored AF value SAF is a value indicating the position of the imaging optical system 281 in the optical axis direction (for example, the number of feeding steps of the AF motor 286, etc.), and when the imaging optical system 281 is composed of a plurality of optical elements. , A value indicating the position of each.

  Thereby, the stored AE value SAE and the stored AF value SAF corresponding to the orientation of the imaging optical system 281 are always updated to the latest values, and the direction in which the imaging optical system 281 has not been directed so far is also A learning function for sequentially storing the stored AE value SAE and the stored AF value SAF can be realized.

  Here, in order to prevent a problem that AF cannot be performed due to a whiteout image or a blackout image, the AE is first performed and the image is adjusted to an appropriate exposure, and then the AF is performed. AF may be performed in parallel as appropriate. Even in such a case, it is desirable to perform AF after AE is preceded to eliminate extreme whiteout images and blackout images. Furthermore, if there is a whiteout image or a blackout image, it is better to temporarily stop AF and remove the whiteout image or blackout image by AE and restart the AF.

  FIG. 7 shows the step S200 “movement detection subroutine” of FIG.

  In FIG. 7, in step S301, the angular velocity detected by the horizontal direction (P direction) sensor 288 is integrated for a fixed time, and in step S303, the current P direction of the imaging optical system 281 is calculated from the angular velocity integrated in step S301. Is calculated. Similarly, in step S305, the angular velocity detected by the vertical direction (T direction) sensor 289 is integrated for a predetermined time, and in step S307, the angular velocity integrated in step S305 is calculated in the current T direction of the imaging optical system 281. The orientation is calculated. Steps S301 and S303 and steps S305 and S307 may be executed in parallel.

  In step S311, it is confirmed whether or not the current P direction orientation of the imaging optical system 281 calculated in step S303 has changed by a predetermined value or more from the previous P direction orientation calculation. When it has not changed more than the predetermined value (step S311; No), the process proceeds to step S331 as it is. If it has changed by a predetermined value or more (step S311; Yes), in step S321, the current P direction orientation of the imaging optical system 281 calculated in step S303 is stored in the storage unit 285m, and in step S323, the P direction. The movement flag Fp is set, and the process proceeds to step S331.

  In step S331, it is confirmed whether or not the current T-direction orientation of the imaging optical system 281 calculated in step S307 has changed by a predetermined value or more from the previous T-direction orientation calculation. If it has not changed more than the predetermined value (step S331; No), the process directly returns to step S200 in FIG. If it has changed by a predetermined value or more (step S331; Yes), in step S341, the current T direction orientation of the imaging optical system 281 calculated in step S307 is stored in the storage unit 285m, and in step S343, the T direction. The movement flag Ft is set, and the process returns to step S200 in FIG.

  If at least one of the P-direction movement flag Fp and the T-direction movement flag Ft is set, a change in the direction of the imaging optical system 281 is detected in step S211 in FIG. It should be noted that the predetermined value that is the determination value of the change in the direction of the P direction and the T direction described above may be appropriately set according to the performance required from the remote work support system 1 to the HMD 2.

  FIG. 8 shows step S220 “normal AE subroutine” of FIG.

  In FIG. 8, one-frame imaging is performed in step S401. The shutter speed of the image sensor 282 and the aperture value of the aperture 287 used at the time of imaging in step S401 are stored in step S261 when step S220 “normal AE subroutine” is executed after step S261 in FIG. The shutter speed and aperture value satisfy the existing stored AE value SAE on the table PTT.

Further, when step S220 “normal AE subroutine” is executed after step S211 of FIG. 6 or after step S251 of No, it is the shutter speed and aperture value used in the immediately preceding imaging operation. Next, in step S403, the luminance signal of the pixels within the AE area set on the entire imaging surface or a part of the imaging surface of the imaging element 282 is read out, and in step S405, the luminance signal read out in step S403 is read out. Is calculated.

  In step S411, it is confirmed whether or not the luminance distribution calculated in step S405 is within an appropriate range. Whether or not it is in an appropriate range depends on whether (1) the maximum luminance of the luminance distribution exceeds the white level and is not saturated (so-called whiteout), or (2) the minimum luminance of the luminance distribution is below the black level. Whether or not black is crushed, (3) whether or not the average value of the luminance distribution is in the vicinity of the median between the white level and the black level is determined.

  If the luminance distribution is not within the proper range (step S411; No), an appropriate shutter speed is calculated using a general AE calculation in step S421, and similarly using a general AE calculation in step S423. An appropriate aperture value is calculated. Steps S421 and S423 may be executed simultaneously. In step S425, the appropriate shutter speed and the appropriate aperture value calculated in steps S421 and S423 are stored in the storage unit 285m as the latest AE value, and the process returns to step S220 “normal AE subroutine” in FIG.

  When the luminance distribution is in an appropriate range (step S411; Yes), in step S431, the shutter speed and the aperture value at the time of imaging in step S401 are stored in the storage unit 285m as the latest AE value. The process returns to step S220 “normal AE subroutine”.

  Here, the set of shutter speed and aperture value is stored as the latest AE value, but instead, for example, an Ev value that is an exposure constant obtained from the shutter speed and aperture value may be stored.

  FIG. 9 shows the step S230 “normal AF subroutine” of FIG.

  In FIG. 9, first, in step S501, the shutter speed and the aperture value are set to the latest AE values stored in FIG. Next, in step S503, the luminance signal of the pixels within the AF area set on the entire imaging surface or a part of the imaging element 282 is read, and the difference between the luminance signals, that is, the contrast is calculated. In step S505, the imaging optical system 281 is moved by a predetermined amount in the far side, that is, in the direction approaching the imaging element 282. In step S507, the contrast in the AF area is calculated in the same manner as in step S503.

  In step S511, the contrast calculated in step S503 is compared with the contrast calculated in step S507, and it is confirmed whether or not the contrast is increased in step S507. If it has increased (step S511; Yes), the process returns to step S505, and steps S505 to S511 are repeated until the contrast does not increase. In the case of repetition, the contrast calculated in step S507 is compared with the contrast calculated in the previous step S507.

  If the contrast has not increased (step S511; No), in step S521, the imaging optical system 281 is moved by a predetermined amount in the near side, that is, in the direction away from the imaging element 282. Subsequently, in step S523, the contrast in the AF area is calculated as in steps S503 and S507.

  In step S531, the contrast calculated in step 507 is compared with the contrast calculated in step S523, and it is confirmed whether or not the contrast is increased in step S523. If it has increased (step S531; Yes), the process returns to step S521, and steps S521 to S531 are repeated until the contrast does not increase. In the case of repetition, the contrast calculated in step 523 is compared with the contrast calculated in the previous step S523.

  If the contrast does not increase (step S531; No), the movement of the imaging optical system 281 is stopped in step S541, and the current position of the imaging optical system 281 is stored in the storage unit 285m as the latest AF value in step S543. The process returns to step S230 “Normal AF subroutine” in FIG.

  In contrast comparison in steps S511 and S531, it is desirable to give a hysteresis to the determination level and take measures to prevent so-called hunting in which the imaging optical system 281 repeatedly moves in small increments, as is generally done. . Further, the predetermined amount by which the imaging optical system 281 described above is moved may be set as appropriate in accordance with the focus performance required from the remote work support system 1 to the HMD 2.

  As described above, according to the present invention, the exposure value is controlled based on the previously stored exposure value in the direction in which the imaging optical system is facing, or in the direction in which the imaging optical system is facing. By focusing the image of the subject based on the previously stored in-focus position, the work object viewed by the operator and the electronic camera viewed on the monitor by the remote instructor are captured. Since it is possible to minimize the state where the images of the work object are different, it is possible for the worker and the instructor to share the same image information, and promotes accurate communication between the operator and the instructor. It is possible to provide an imaging unit capable of performing remote work support.

  The detailed configuration and detailed operation of each component constituting the imaging unit according to the present invention can be changed as appropriate without departing from the spirit of the present invention.

It is a schematic diagram which shows the outline | summary of the whole structure of a remote work assistance system. It is a schematic diagram which shows the whole structure of HMD. It is a block diagram which shows the internal structure of HMD. It is a schematic diagram showing the configuration of a storage table that stores the directions of the imaging optical system in the P direction and the T direction, and stored AE values and stored AF values in the directions. It is the main routine of the flowchart which shows embodiment of AE and AF control of an imaging unit. This is the subroutine (1/1) in FIG. This is the subroutine (1/3) in FIG. This is the subroutine (2/3) in FIG. This is the subroutine (3/3) in FIG.

Explanation of symbols

1 Remote work support system 2 HMD
4 Server Device 5 Host Computer 53 Monitor 6 Image / Audio Recording Server 7 Content Server 25 Display Unit 26 LCD Display Unit 27 Eyepiece Optical System 28 Imaging Unit 281 Imaging Optical System 282 Imaging Element 283 AFE
284 Image processing unit 284a Digital image data 285 Imaging control unit 285m Storage unit 286 AF motor 287 Aperture 288 Horizontal direction (P direction) sensor 289 Vertical direction (T direction) sensor 31 Control unit 32 Control unit 33 Image memory 35 Operation unit 36 Communication Part A Instructor B Worker (wearer)
EN network IN internet PN public line network SPT work support RPT work record SAE storage AE value SAF storage AF value Fp P direction movement flag Ft T direction movement flag

Claims (7)

An image sensor for capturing an image of a subject;
An imaging optical system for forming an image of a subject on the imaging surface of the imaging device;
An exposure control unit for controlling an exposure value of an image of a subject imaged by the imaging device;
In an imaging unit comprising: a focus control unit for moving the position of the imaging optical system to focus an image of a subject on the imaging surface of the imaging element;
An orientation change detection unit for detecting a change in orientation of the imaging optical system;
A direction calculation unit that calculates a direction of the imaging optical system based on a detection result of the direction change detection unit;
At the storage position corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit, the exposure value controlled by the exposure control unit in that orientation and the imaging optical system controlled by the focus control unit A storage unit for storing the in-focus position;
Causing the exposure control unit to perform exposure control based on the exposure value in the direction stored in the storage position corresponding to the direction of the imaging optical system calculated by the direction calculation unit of the storage unit;
Imaging that causes the focusing control unit to perform focusing control based on the in-focus position stored in the storage position corresponding to the orientation of the imaging optical system calculated by the orientation calculating unit of the storage unit An imaging unit comprising a control unit. 2. The exposure control unit according to claim 1, wherein the exposure control unit sets the exposure value to the exposure value stored in the storage unit, and then controls the exposure value using the exposure value stored in the storage unit as a starting point. The imaging unit described. The focus control unit sets the position of the imaging optical system to the focus position stored in the storage unit, and then uses the focus position stored in the storage unit as a starting point to capture the image of the subject in the image sensor. The imaging unit according to claim 1, wherein the imaging unit is focused on. When the exposure value corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit is not stored in the storage unit, the exposure control unit starts the exposure value at the time of the previous imaging as a starting point. Control the exposure value,
The imaging according to claim 1, wherein the storage unit stores the exposure value in a storage position corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit after completion of exposure control. unit. The focus control unit is configured to focus at the time of previous imaging when the in-focus position of the imaging optical system corresponding to the orientation of the imaging optical system calculated by the orientation calculation unit is not stored in the storage unit. Focus the image of the subject on the imaging surface of the image sensor starting from the position,
The said storage part memorize | stores the focus position of the said imaging optical system in the memory | storage position corresponding to the direction of the said imaging optical system calculated by the said orientation calculation part after focusing. The imaging unit described in 1. 6. The storage unit according to claim 1, wherein the storage unit constantly updates the exposure value and the focus position stored in the storage position corresponding to the orientation of the imaging optical system to the latest values. The imaging unit described in 1. The imaging unit according to claim 1, wherein an exposure value and a focus position stored in the storage unit are initialized when the imaging unit is turned on.

Images