JP2021136663A - Device, communication apparatus, method and program - Google Patents

Abstract

To improve the convenience in a system in which a detachable device that can execute a prescribed function is used.SOLUTION: There is provided a device which is attached to an apparatus having an attachment mechanism that can be detachably attached with the device capable of storing at least a captured image. The device can execute prescribed processing to image data based on the image captured by an imaging apparatus when being attached to the imaging apparatus having the attachment mechanism. The device determines whether or not the imaging apparatus has a function of making the device execute prescribed processing, and stores prescribed data when determining that the imaging apparatus does not have the function.SELECTED DRAWING: Figure 10

Description

The present invention relates to a technique for improving convenience in a system in which a removable device capable of performing a predetermined function is used.

In recent years, in various scenes, image processing such as image analysis that detects and tracks objects, estimates attributes, etc., and estimates of the number of objects based on the results of such image analysis, using images captured by surveillance cameras. Is being done. Conventionally, such image processing has been performed by transferring the image of the surveillance camera to a high-performance arithmetic unit such as a PC or a server that executes the actual image processing. On the other hand, with the recent improvement in the processing capacity of mobile arithmetic units, it has become possible to perform image processing on the surveillance camera side. The processing on the camera side can be executed by, for example, an arithmetic unit arranged in the camera body. Further, by arranging the arithmetic unit on a detachable device such as USB, the functions that can be executed on the camera side can be expanded and the processing can be shared with the camera (see Patent Document 1).

Japanese Unexamined Patent Publication No. 7-184098

The present invention provides a technique for improving convenience in a system in which a removable device capable of performing a predetermined function is used.

The device according to one aspect of the present invention is the device mounted on a device having a mounting mechanism capable of attaching and detaching at least a device capable of storing captured images, and is mounted on an imaging device having the mounting mechanism. The image pickup apparatus has a processing means capable of executing a predetermined process on image data based on an image captured by the image pickup apparatus and a function capable of causing the processing means to execute the predetermined process. It has a determination means for determining whether or not the image is present, and a storage means for storing predetermined data when it is determined that the image pickup apparatus does not have the function.

According to the present invention, it is possible to improve convenience in a system in which a removable device capable of performing a predetermined function is used.

It is a figure which shows an example of a system configuration. It is a figure which shows the hardware configuration example of the image pickup apparatus. It is a figure which shows the functional configuration example of an image pickup apparatus. It is a figure which shows the hardware configuration example of a detachable device. It is a figure which shows the functional configuration example of a detachable device. It is a figure which shows the hardware configuration example of an input / output device. It is a figure which shows the functional configuration example of an input / output device. It is a figure which shows the example of the flow of processing executed in a system. It is a figure which shows the example of the process flow of determining whether or not a detachable device supports an extended function. It is a figure which shows the example of the processing flow of an initial sequence. It is a figure which shows the example of the request and response of the command of an initial sequence. It is a figure which shows the configuration example of the response of the detachable device. It is a figure which shows the example of the processing flow which is executed after the completion of an initial sequence. It is a figure which shows the example of the flow of the process which determines whether the image pickup apparatus supports an extended function. It is a figure which shows the example of the process flow which an input / output device refers to the designated data of a detachable device.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiment, not all of the plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the attached drawings, the same or similar configurations are given the same reference numbers, and duplicate explanations are omitted.

<System configuration>
FIG. 1 shows a configuration example of the image analysis system of the present embodiment. In the following, as an example, a case where this system is a specific person tracking system will be described. However, the present invention is not limited to this, and the following discussion can be applied to any system that analyzes an image and outputs a predetermined information. This system includes imaging devices 110a to 110d, a network 120, and an input / output device 130. Each of the imaging devices 110a to 110d has a slot into which a device capable of recording an captured image can be attached / detached, and the detachable devices 100a to 100d are inserted into the slot to allow the detachable devices 100a to 100a to be attached / detached. Connected to 100d. In the following, the removable devices 100a to 100d will be referred to as "detachable device 100", and the imaging devices 110a to 110d will be referred to as "imaging device 110".

The removable device 100 is a calculation device that can be attached to and detached from the image pickup device 110. The detachable device 100 is, for example, a device in which a predetermined processing circuit is mounted on an SD card. The detachable device 100 is configured so that the entire removable device 100 can be inserted into the imaging device 110, for example, depending on the mode of the SD card, so that the removable device 100 can be connected to the imaging device 110 without a portion protruding from the imaging device 110. be able to. Further, the detachable device 100 is configured so that, for example, more than half (for example, all) of the detachable device 100 can be inserted into the image pickup device 110, whereby the detachable device 100 can be connected to the image pickup device 110 with few portions protruding from the image pickup device 110. It may be configured in. As a result, it is possible to prevent the removable device 100 from interfering with obstacles such as wiring, and it is possible to improve the convenience when using the device. Further, since the image pickup device 110 of many existing network cameras and the like is provided with an SD card slot, it is possible to provide an extended function to the existing image pickup device 110 by the detachable device 100. The detachable device 100 is attached to the image pickup device 110 by at least an arbitrary interface used when a storage device capable of storing an image captured by the image pickup device 110 is attached, in addition to the mode of the SD card. It may be configured to be. For example, the removable device 100 may have a USB (universal serial bus) interface and may be configured to be mounted in the USB socket of the imaging device 110. Further, the predetermined processing circuit is implemented by, for example, an FPGA (Field Programmable Gate Array) programmed to perform a predetermined processing, but may be implemented in other formats.

The image pickup device 110 is an image pickup device such as a network camera. In the present embodiment, the image pickup apparatus 110 has a built-in arithmetic unit capable of processing an image, but the present invention is not limited to this. For example, an external computer such as a PC (personal computer) connected to the image pickup device 110 may exist, and a combination thereof may be treated as the image pickup device 110. Further, in the present embodiment, it is assumed that the removable device 100 is attached to all the imaging devices 110. Although FIG. 1 shows four image pickup devices 110 and detachable devices attached to each of the four image pickup devices 110, the number of combinations of these devices may be three or less, or five or more. It may be. By mounting the removable device 100 having an image analysis processing function on the image pickup device 110, it is possible to execute image processing on the image pickup device 110 side even if the image pickup device 110 does not have the image analysis processing function. Can be. Further, in the embodiment in which the arithmetic unit for image processing is arranged in the image pickup device 110 as in the present embodiment, the removable device 100 in which the arithmetic unit is arranged is attached to the image pickup device 110, so that the image pickup device 110 side. It is possible to diversify and enhance the image processing that can be performed with.

The input / output device 130 is a device that accepts input from a user and outputs information to the user (for example, display of information). In the present embodiment, for example, the input / output device 130 corresponds to an external communication device (Video Management System (VMS) or the like). Further, the input / output device 130 is a computer such as a PC, and information is input / output by a browser or a native application installed in the computer.

The image pickup apparatus 110 and the input / output apparatus 130 are communicably connected via the network 120. The network 120 includes a plurality of routers, switches, cables, and the like that meet communication standards such as Ethernet (registered trademark). In the present embodiment, the network 120 may be any network that enables communication between the image pickup apparatus 110 and the input / output apparatus 130, and may be constructed by an arbitrary scale, configuration, and compliant communication standard. For example, the network 120 may be the Internet, a wired LAN (Local Area Network), a wireless LAN, a WAN (Wide Area Network), or the like. Further, the network 120 can be configured to enable communication with a communication protocol conforming to the ONVIF (Open Network Video Interface Forum) standard, for example. However, the present invention is not limited to this, and the network 120 may be configured so as to be able to communicate with other communication protocols such as a unique communication protocol.

<Device configuration>
(Configuration of imaging device)
Subsequently, the configuration of the image pickup apparatus 110 will be described. FIG. 2 is a diagram showing a hardware configuration example of the image pickup apparatus 110. The image pickup apparatus 110 includes, for example, an image pickup unit 201, an image processing unit 202, an arithmetic processing unit 203, a distribution unit 204, and an SD I / F unit 205 as its hardware configuration. I / F is an abbreviation for interface.

The image pickup unit 201 includes a lens unit for forming an image of light and an image pickup element that converts an analog signal according to the imaged light. The lens unit has a zoom function for adjusting the angle of view, an aperture function for adjusting the amount of light, and the like. The image sensor has a gain function that adjusts the sensitivity when converting light into an analog signal. These functions are adjusted based on the set value notified from the image processing unit 202. The analog signal acquired by the imaging unit 201 is converted into a digital signal by the analog-digital conversion circuit and transferred to the image processing unit 202 as an image signal.

The image processing unit 202 includes an image processing engine, peripheral devices thereof, and the like. Peripheral devices include, for example, a RAM (Random Access Memory), a driver for each I / F, and the like. The image processing unit 202 generates image data by performing image processing such as development processing, filter processing, sensor correction, and noise removal on the image signal acquired from the image pickup unit 201. Further, the image processing unit 202 can transmit the set value to the lens unit or the image sensor and perform exposure adjustment so that an appropriate exposure image can be acquired. The image data generated by the image processing unit 202 is transferred to the arithmetic processing unit 203.

The arithmetic processing unit 203 is composed of one or more processors such as a CPU and an MPU, a memory such as a RAM and a ROM, and a driver for each I / F. The CPU is an acronym for Central Processing Unit, the MPU is an acronym for Micro Processing Unit, the RAM is an acronym for Random Access Memory, and the ROM is an acronym for Read Only Memory. In one example, the arithmetic processing unit 203 determines the division between the imaging device 110 and the detachable device 100 to execute each part of the processing to be executed in the above-mentioned system, and corresponds to the determined division. The process can be executed. The details of the processing contents and the division of processing will be described later. The image received from the image processing unit 202 is transferred to the distribution unit 204 or the SD I / F unit 205. The processing result data is also transferred to the distribution unit 204.

The distribution unit 204 includes a network distribution engine and peripheral devices such as a RAM and an ETH PHY module. The ETH PHY module is a module that executes processing of the physical (PHY) layer of Ethernet. The distribution unit 204 converts the image data and the processing result data acquired from the arithmetic processing unit 203 into a format that can be distributed to the network 120, and outputs the converted data to the network 120. The SD I / F unit 205 is an interface portion for connecting to the detachable device 100, and includes, for example, a power supply and a mounting mechanism such as a detachable socket for attaching / detaching the detachable device 100. Here, it is assumed that the SD I / F section 205 is configured according to the SD standard established by the SD Association. Communication between the detachable device 100 and the image pickup device 110, such as transfer of an image acquired from the arithmetic processing unit 203 to the detachable device 100 and acquisition of data from the detachable device 100, is performed by SD I /. It is done through F part 205.

FIG. 3 shows an example of the functional configuration of the image pickup apparatus 110. The image pickup device 110 includes, for example, an image pickup control unit 301, a signal processing unit 302, a storage unit 303, a control unit 304, an analysis unit 305, a device communication unit 306, and a network communication unit 307 as its functions.

The image pickup control unit 301 executes control for photographing the surrounding environment via the image pickup unit 201. The signal processing unit 302 performs predetermined processing on the image captured by the image pickup control unit 301 to generate data of the captured image. Hereinafter, the data of this photographed image is simply referred to as a “photographed image”. The signal processing unit 302 encodes, for example, an image captured by the image pickup control unit 301. The signal processing unit 302 encodes the still image by using, for example, a coding method such as JPEG (Joint Photographic Experts Group). In addition, the signal processing unit 302 uses the H. Coding is performed using a coding method such as 264 / MPEG-4 AVC (hereinafter referred to as “H.264”) or HEVC (High Efficiency Video Coding). Further, the signal processing unit 302 encodes an image by using a coding method selected by the user from among a plurality of preset coding methods, for example, via an operation unit (not shown) of the image pickup apparatus 110. It may be converted.

The storage unit 303 stores a list of analysis processes that can be executed by the analysis unit 305 (hereinafter referred to as a "first process list") and a list of post-processing for the result of the analysis process. In addition, the storage unit 303 stores the result of the analysis process described later. In the present embodiment, the process to be executed is the analysis process, but any process may be executed, and the storage unit 303 describes the process related to the process to be executed with the first process list and later. It may be possible to memorize a list of processes. The control unit 304 controls the signal processing unit 302, the storage unit 303, the analysis unit 305, the device communication unit 306, and the network communication unit 307 so that each of them executes a predetermined process.

The analysis unit 305 selectively executes at least one of the pre-analysis processing, the analysis processing, and the post-analysis processing described later on the captured image. The pre-analysis process is a process executed on a captured image before executing the analysis process described later. In the pre-analysis process of the present embodiment, as an example, it is assumed that a process of dividing a captured image to create a divided image is executed. The analysis process is a process of outputting information obtained by analyzing an input image. In the analysis process of the present embodiment, as an example, at least one of the human body detection process, the face detection process, and the vehicle detection process is executed by inputting the divided image obtained by the pre-analysis process, and the analysis process result is output. Suppose that is executed. The analysis process is configured to output the position of the object in the divided image by using a machine learning model trained so that the object included in the image can be detected by, for example, the technique of Non-Patent Document 1. It can be a process. The post-analysis process is a process that is executed after the analysis process is executed. In the post-analysis processing of the present embodiment, as an example, a process of outputting a value obtained by totaling the number of objects detected in each divided image as a processing result is executed based on the analysis processing result for each divided image. do. The analysis process may be a process of performing pattern matching, detecting an object in the image, and outputting the position.

The device communication unit 306 communicates with the detachable device 100. The device communication unit 306 converts the input data into a format that can be processed by the detachable device 100, and transmits the data obtained by the conversion to the detachable device 100. Further, the device communication unit 306 receives data from the detachable device 100 and converts the received data into a format that can be processed by the image pickup apparatus 110. In the present embodiment, the device communication unit 306 executes a process of converting a decimal number between a floating point format and a fixed point format as a conversion process, but the conversion process is not limited to this, and other processes are device communication. It may be performed by unit 306. Further, in the present embodiment, the device communication unit 306 transmits a command sequence predetermined within the range of the SD standard to the detachable device 100, and receives a response from the detachable device 100. It is assumed that communication with the detachable device 100 is performed. The network communication unit 307 communicates with the input / output device 130 via the network 120.

(Detachable device configuration)
FIG. 4 shows an example of the hardware configuration of the detachable device 100. As an example, the detachable device 100 includes an I / F unit 401, an FPGA 402, an SD controller 403, a storage unit 404, and an FPGA 405. The detachable device 100 is formed in a shape that can be inserted into and removed from the detachable socket of the SD I / F portion 205 of the image pickup apparatus 110, that is, in a shape that conforms to the SD standard.

The I / F unit 401 is an interface portion for connecting a device such as an image pickup device 110 and a detachable device 100. The I / F unit 401 includes, for example, an electrical contact terminal or the like that receives power from the image pickup apparatus 110 and generates and distributes the power to be used in the detachable device 100. Similar to the SD I / F unit 205 of the image pickup apparatus 110, the I / F unit 401 shall comply with the items defined (compliant) in the SD standard. The reception of images and setting data from the image pickup device 110 and the transmission of data from the FPGA 402 to the image pickup device 110 are executed via the I / F unit 401.

The FPGA 402 includes an input / output control unit 410, a processing switching unit 411, and an arithmetic processing unit 412. The FPGA 402 is a type of semiconductor device capable of repeatedly reconstructing the internal logic circuit structure. By the processing realized by the FPGA 402, a processing function can be added (provided) to the device equipped with the detachable device 100. Further, since the logic circuit structure can be changed later by the reconstruction function of the FPGA 402, for example, by attaching the detachable device 100 to a device in a field where technological progress is rapid, appropriate processing is performed in the device in a timely manner. Can be executed. In this embodiment, an example in which the FPGA is used will be described, but as long as the processing described later can be realized, for example, a general-purpose ASIC or a dedicated LSI may be used. The FPGA 402 is started by writing the setting data including the information of the generated logic circuit structure from the dedicated I / F or by reading the setting data from the dedicated I / F. In the present embodiment, it is assumed that this setting data is stored in the storage unit 404. When the power is turned on, the FPGA 402 reads the setting data from the storage unit 404, generates a logic circuit, and starts up. However, the present invention is not limited to this, and for example, by mounting a dedicated circuit in the detachable device, the image pickup apparatus 110 may write the setting data to the FPGA 402 via the I / F unit 401.

The input / output control unit 410 includes a circuit for transmitting and receiving an image to and from the image pickup device 110, a circuit for analyzing a command received from the image pickup device 110, a circuit for performing control based on the analysis result, and the like. Will be done. The commands here are those defined in the SD standard, and the input / output control unit 410 can detect some of them. The details of the function will be described later. The input / output control unit 410 controls so as to transmit an image to the SD controller 403 in the case of storage processing and to transmit an image to the arithmetic processing unit 412 in the case of image analysis processing. When the input / output control unit 410 receives the processing switching setting data, the input / output control unit 410 transmits the setting data to the processing switching unit 411. The processing switching unit 411 includes a circuit for acquiring information on the image analysis processing function from the storage unit 404 and writing it to the arithmetic processing unit 412 based on the setting data received from the image pickup apparatus 110. The information of the image analysis processing function is, for example, a setting parameter indicating the order and type of operations processed in the operation processing unit 412, the coefficient of the operation, and the like. The arithmetic processing unit 412 is configured to include a plurality of arithmetic circuits necessary for executing the image analysis processing function. The arithmetic processing unit 412 executes each arithmetic processing based on the information of the image analysis processing function received from the processing switching unit 411, transmits the processing result to the image pickup apparatus 110, and / or outputs the processing result. Record in the storage unit 404. In this way, the FPGA 402 extracts the setting data of the processing function to be executed, which is included in the setting data corresponding to the plurality of processing functions held in advance, and the arithmetic processing unit 412 is based on the extracted setting data. Rewrite the processing content executed by. As a result, the detachable device 100 can selectively execute at least one of the plurality of processing functions. Further, by adding the setting data of the newly added process at any time, the latest process can be executed on the image pickup apparatus 110 side. In the following, having a plurality of setting data corresponding to each of the plurality of processing functions is expressed as having a plurality of processing functions. That is, even if the FPGA 402 of the detachable device 100 is configured to execute one processing function, the processing content of the arithmetic processing unit 412 can be changed by the setting data for the other processing function. When possible, it is expressed as having a plurality of processing functions.

The SD controller 403 is a known control IC (integrated circuit) as defined in the SD standard, and executes control of slave operation of the SD protocol and control of reading / writing data to / from the storage unit 404. The storage unit 404 is configured by, for example, a NAND flash memory, and stores various information such as storage data written from the image pickup apparatus 110, information of the image analysis processing function written to the arithmetic processing unit 412, and setting data of the FPGA 402. do.

FIG. 5 shows an example of the functional configuration of the removable device 100. The detachable device 100 includes, for example, an analysis unit 501 and a communication unit 502 as its functional configuration. The analysis unit 501 executes an analysis process on the image. For example, when the analysis process setting request is input, the analysis unit 501 executes the setting for making the input analysis process executable. Further, when an image is input, the analysis unit 501 executes an analysis process set in an executable state on the input image. In the present embodiment, the feasible analysis processes are, but are not limited to, the human body detection process and the face detection process. For example, it may be a process of determining whether or not a person stored in advance is included in the image (for example, a face recognition process). For example, the degree of matching between the image feature amount of the person stored in advance and the image feature amount of the person detected from the input image is calculated, and when the degree of matching is equal to or more than the threshold value, the person is stored in advance. It is determined that there is. Also, for the purpose of privacy protection. It may be a process of superimposing a predetermined mask image or performing a mosaic process on the person detected from the input image. Further, it may be a process of detecting whether or not the person in the image is performing a specific action by using a learning model in which a specific action of the person is learned by machine learning. Further, it may be a process of determining what kind of region the region in the image is. For example, it may be a process of determining what kind of area is in the image by using a learning model in which buildings, roads, people, the sky, etc. are learned by machine learning. As described above, the executable analysis process can be applied to both the image analysis process using machine learning and the image analysis process not using machine learning. Further, each of the above analysis processes may not be performed by the detachable device 100 alone, but may be performed in cooperation with the image pickup apparatus 110. The communication unit 502 communicates with the image pickup apparatus 110 via the I / F unit 401.

(I / O device configuration)
FIG. 6 shows an example of the hardware configuration of the input / output device 130. The input / output device 130 is configured as a computer such as a general PC. For example, as shown in FIG. 6, the input / output device 130 includes a processor 601 such as a CPU, a memory such as a RAM 602 and a ROM 603, a storage device such as an HDD 604, and a communication I / O. It is configured to include F605. The input / output device 130 can execute various functions by the processor 601 executing a program stored in the memory or the storage device.

FIG. 7 shows an example of the functional configuration of the input / output device 130 according to the present embodiment. The input / output device 130 includes, for example, a network communication unit 701, a control unit 702, a display unit 703, and an operation unit 704 as its functional configuration. The network communication unit 701 connects to the network 120, for example, and executes communication with an external device such as an image pickup device 110 via the network 120. Note that this is only an example. For example, the network communication unit 701 is configured to establish a direct connection with the image pickup device 110 and communicate with the image pickup device 110 without going through the network 120 or other devices. May be good. The control unit 702 controls the network communication unit 701, the display unit 703, and the operation unit 704 so as to execute their respective processes. The display unit 703 presents information to the user, for example, via a display. In this embodiment, information is presented to the user by displaying the result rendered by the browser on the display. Information may be presented by a method other than screen display such as voice or vibration. The operation unit 704 receives an operation from the user. In the present embodiment, the operation unit 704 is a mouse or a keyboard, and the user operates these to input the user operation to the browser. However, the present invention is not limited to this, and the operation unit 704 may be any device that can detect the intention of another user, such as a touch panel or a microphone.

<Processing flow>
Next, an example of the flow of processing executed in the system will be described. Of the following processes, the process executed by the image pickup apparatus 110 is realized, for example, by the processor in the arithmetic processing unit 203 executing a program stored in a memory or the like. However, this is only an example, and a part or all of the processing described later may be realized by dedicated hardware. Further, the processing executed by the detachable device 100 and the input / output device 130 may also be realized by the processor in each device executing a program stored in a memory or the like, and a part or all of the processing may be performed. It may be realized by dedicated hardware.

(Overall flow)
FIG. 8 schematically shows a series of flow of image analysis processing executed by the system. In this process, first, the user attaches the detachable device 100 to the image pickup apparatus 110 (S801). The image pickup apparatus 110 executes an initialization sequence of the detachable device 100 (S802). In this initialization sequence, a predetermined command is transmitted and received between the image pickup device 110 and the detachable device 100, so that the image pickup device 110 is in a state where the detachable device 100 can be used. After that, the image pickup device 110 grasps the processes that can be executed by the detachable device 100, and grasps the processes that can be executed locally (the image pickup device 110 alone or the combination of the image pickup device 110 and the detachable device 100 can execute the processes). (S803). The detachable device 100 may be configured to be capable of executing arbitrary processing, but processing unrelated to the processing to be executed on the image pickup apparatus 110 side may not be considered. In one example, the imaging device 110 may retain, for example, a list of possible processes previously obtained from the input / output device 130. In this case, when the image pickup device 110 acquires information indicating a process that can be executed by the detachable device 100 from the detachable device 100, only the process that can be executed depends on whether or not the process is included in the list. Can be grasped. Subsequently, the image pickup apparatus 110 determines the process to be executed, and executes the setting of the detachable device 100 as needed (S804). That is, when at least a part of the process determined as the execution target is executed by the detachable device 100, the setting of the detachable device 100 for the process is executed. In this setting, for example, the FPGA 402 can be reconfigured using the setting data corresponding to the process to be executed. Then, the image pickup apparatus 110 or the detachable device 100 executes the analysis process (S805). After that, the image pickup apparatus 110 executes post-processing (S806). The processes of S805 and S806 are repeatedly executed. The process of FIG. 8 is executed, for example, when the detachable device 100 is attached. For example, the process of S803 is executed again when the detachable device 100 is removed. At least a part of the process of 8 may be repeatedly executed.

Note that FIG. 8 shows an example of a processing flow executed when the removable device 100 having a processing function is attached to the image pickup apparatus 110 configured to enable the processing function by the removable device 100. ing. On the other hand, for example, when the detachable device 100 having no processing function is attached to the image pickup apparatus 110, the image processing by the detachable device 100 is not performed as a matter of course. For example, if the user attaches an erroneous SD card to the image pickup apparatus 110, the detachable device 100 cannot add functions. The user can recognize that a device having no such processing function is attached to the imaging device 110, for example, by examining the processing that can be performed on the imaging device 110 side. Therefore, when the removable device 100 having no additional processing function is erroneously attached to the image pickup apparatus 110, the user can recognize the state. In addition, the user can recognize in which imaging device 110 such an erroneous attachment has occurred, for example, via the input / output device 130. On the other hand, the image pickup apparatus 110 may not support the addition of functions by the removable device 100. In this case, the image pickup apparatus 110 cannot recognize whether or not the attached removable device 100 has an additional function. That is, when the SD card type removable device 100 is attached, the image pickup apparatus 110 can only recognize the SD card as a mere storage device. Therefore, when the detachable device 100 having the additional processing function is erroneously attached to the image pickup apparatus 110 that does not correspond to the additional function, the user may not be able to recognize the erroneous attachment. For example, a unique ID is assigned to the removable device 100, and the image pickup device 110 can read the ID to determine the type of the removable device 100. However, if the image pickup apparatus 110 does not have the function of reading the ID, the function of the removable device 100 cannot be determined. Therefore, when the removable device 100, which was originally supposed to be attached to the image pickup device 110 capable of adding functions, is attached to the image pickup device 110 which cannot support such addition of functions, it is attached to the image pickup device 110. Cannot be recognized in. As a result, in order to confirm in which image pickup device 110 the erroneous attachment has occurred, it is necessary for the user to remove and confirm the detachable device 100 attached to each of the image pickup devices 110, which is complicated.

Therefore, in the present embodiment, a procedure is used to efficiently identify whether or not the removable device 100 mounted on the image pickup apparatus 110 has an additional function. This procedure will be described below.

An example of a flow for determining whether or not the detachable device has an image analysis processing function as an extended function will be described with reference to FIG. The detachable device 100 of the present embodiment can provide an image analysis processing function as an extended function, but may be compatible with another extended function in addition to or in place of this. Therefore, in the following, unless otherwise specified, the term “extended function” is used instead of the term “image analysis processing function”.

In the process of FIG. 9, the user first attaches the detachable device 100 to the image pickup apparatus 110 (S901). The image pickup apparatus 110 executes a predetermined processing sequence with the detachable device 100 (S902). This sequence may be an initial sequence executed at startup or a sequence for sending and receiving data after the initial sequence is completed. These two processes will be described later. Then, the input / output device 130 inquires of the image pickup device 110 whether or not the image pickup device 110 supports the extended function (S903). For example, this inquiry is made by the user operating the input / output device 130. It should be noted that this inquiry can be made, for example, by a method and content previously determined between the input / output device 130 and the image pickup device 110. The details of this inquiry processing will be described later. In this case, when an inquiry is made to an imaging device that does not correspond to this predetermined method, the input / output device 130 does not receive a response to the inquiry from the imaging device. Alternatively, an error message will be received. When the input / output device 130 receives a response other than an error message from the image pickup device 110 in response to the inquiry, the input / output device 130 determines whether or not the image pickup device 110 supports the extended function based on the response (S904). When the input / output device 130 determines that the image pickup device 110 does not support the extended function (NO in S904), the input / output device 130 refers to the designated data of the detachable device 100 (S905). The procedure for setting the specified data and the process for referencing this data will also be described later. The input / output device 130 determines whether or not the detachable device 100 has an extended function based on the designated data (S906). The designated data of the detachable device 100 can be generated, for example, in the initial sequence of S902. Therefore, the details of the data contents will be described later together with the description of the initial sequence of S902. On the other hand, when the input / output device 130 determines that the image pickup device 110 supports the extended function (YES in S904), the input / output device 130 determines whether or not the detachable device 100 has the extended function with respect to the image pickup device 110. Inquire. Then, the input / output device 130 determines whether or not the detachable device 100 has an extended function based on this inquiry (S907). The procedure for this inquiry will also be described later.

An example of the process executed in S902 of FIG. 9 will be described with reference to FIGS. 10 and 13. In this process, the detachable device 100 determines whether or not the image pickup apparatus 110 supports the extended function, and executes a process based on the determination result. First, in FIG. 10, the initial sequence is started after the detachable device 100 is attached to the image pickup apparatus 110 and activated (S1001). This initial sequence is performed according to the SD standard, for example, if the removable device is a device that operates according to the SD standard. FIG. 11 shows an example of the flow of the initial sequence according to the SD standard. In FIG. 11, "CMD" is a command and "CAMD" is an application command. The numbers following the CMD and the CMD indicate the type of command, and the content corresponding to the numbers is requested. The detachable device 100 transmits a response to the request command of the image pickup apparatus 110 to the image pickup apparatus 110 for a command requiring a response, and does not transmit a response to a request command that does not require a response. .. Further, in the SD standard, the request command is not transmitted from the detachable device 100 to the image pickup apparatus 110.

The detachable device 100 transmits a command indicating that its own device has an extended function by adding an argument in response to a request command from the image pickup device 110 (S1002). The command and response transmitted at this time may be, for example, a command and response in a format conforming to the SD standard. FIG. 12 shows an example of a command configuration conforming to the SD standard. FIG. 12 shows a configuration example of the response to the command CMD2. This response includes fields with a length of 1 bit, 1 bit, 6 bits, 127 bits, and 1 bit, respectively, Start bit, transmission bit, reserved, CID or CSD register, and end bit. Of these, the reserved field is a field prepared as a spare field that is not normally used in SD communication, and is ignored in normal command communication. Therefore, in the present embodiment, the detachable device 100 adds a parameter indicating whether or not its own device has an extended function to the reserved field and transmits it to the image pickup device 110. As a result, the removable device 100 can notify the image pickup device 110 that the device itself has an extended function. Note that CMD2 is an example, and a parameter as to whether or not the detachable device 100 has an extended function may be added to the response to another command. Further, the field to which the information indicating whether or not the removable device 100 has the extended function is added is not limited to the reserved field.

As mentioned above, the imaging device 110 generally ignores the reserved field. On the other hand, in the present embodiment, the imaging device 110 capable of adding functions by the removable device 100 confirms whether the reserved field contains information indicating that the removable device 100 has an extended function. Then, when this information is included in the reserved field, the image pickup apparatus 110 transmits a response (command) to the detachable device 100. Although the case where the command is transmitted only by the image pickup apparatus 110 is described in the present embodiment, the command may be transmitted from the detachable device 100. Then, this command may instruct the image pickup apparatus 110 to notify the image pickup apparatus 110 of information indicating whether or not the image pickup apparatus 110 can support the addition of functions by the detachable device 100. That is, the image pickup device 110 transmits a predetermined second signal for notifying information indicating whether or not the own device can support the addition of functions by the detachable device 100 with respect to the predetermined first signal from the detachable device 100. Any configuration can be used in which is transmitted to the removable device 100.

The detachable device 100 can determine whether or not the image pickup device 110 supports the extended function based on whether or not the response is received from the image pickup device 110 (S1003). The detachable device 100 determines, for example, that the image pickup device 110 corresponds to the extended function when a specific response is received from the image pickup device 110 (YES in S1003) (S1006). On the other hand, when the detachable device 100 does not receive a specific response from the image pickup device 110 (NO in S1003), the detachable device 100 determines that the image pickup device 110 does not support the extended function (S1004). When the detachable device 100 determines that the image pickup device 110 does not support the extended function (S1004), the detachable device 100 creates data indicating that the own device has the extended function and stores the data in the storage unit 404 (S1005). .. The data created here can have specific formats such as text, binary, images, and moving images. Further, the data created here may have a specific file name in addition to having a specific format, or may be stored in a folder having a specific folder name. Further, for example, the created data may be stored in a predetermined specific area of the storage unit 404. That is, the data may be stored in any format as long as it is in a format that can be extracted when later specified by the input / output device 130 or the like. Further, the data to be created is at least one of the fact that the image pickup device 110 does not support the extended function, the type of the detachable device 100, and the conditions of the image pickup device necessary for operating the extended function. Can include. The type of the removable device 100 may be a hardware version, a software version, a supported extended function, or the like. According to this, the user searches the area where the data created in S1005 should be stored via the input / output device 130 or the like, so that the image pickup device 110 corresponds to the addition of the function by the detachable device 100. Can be recognized.

When the detachable device 100 determines in S1003 that the image pickup apparatus 110 corresponds to the extended function (S1006), the detachable device 100 determines whether or not the data previously created in S1005 is stored (S1007). Then, when it is determined that the data previously created in S1005 is stored (YES in S1007), the detachable device 100 deletes the data (S1008). By deleting this data, when the user searches the area where the data created in S1005 should be stored via the input / output device 130, the user corresponds to the fact that the image pickup device 110 does not support the extended function. No data will be found. As a result, it is possible to prevent the user from erroneously determining that the image pickup apparatus 110 does not support the extended function. Instead of deleting the data, the data may be rewritten to a predetermined value such as an invalid value.

Even when the detachable device is a device having only a general storage function, it may be configured to execute the above-mentioned processing, but the above-mentioned processing may not be performed. Since the above processing is not performed, the data of S1005 is not generated. In this case, when the image pickup device 110 does not support the function addition by the detachable device 100, the input / output device 130 tries to refer to the designated data stored in the detachable device 100, but the data does not exist. Therefore, the reference will fail. Therefore, when the input / output device 130 attempts to refer to the designated data stored in the detachable device 100 but fails to refer to the designated data, the input / output device 130 determines that the detachable device 100 does not provide the extended function. sell.

FIG. 13 shows an example of a processing flow that is executed when the imaging device 110 becomes compatible with the extended function after the initial sequence of the imaging device 110 is completed. In the process of FIG. 13, when the image pickup device 110 corresponds to the extended function during activation in S902 of FIG. 9 (S1301), the detachable device 100 issues a command indicating that the image pickup device 110 supports the extended function. (S1302). After that, in the removable device 100, the data created in S1005 is deleted (S1303). Also here, the data may be rewritten to predetermined data such as invalid data without deleting the data.

Note that FIG. 13 shows an example in which the image pickup device 110 actively transmits a command to the detachable device 100 when the image pickup device 110 corresponds to the extended function, but the present invention is not limited to this. For example, the detachable device 100 may periodically transmit the signal of S1002 at regular intervals, and determine whether or not the image pickup apparatus 110 is compatible with the extended function depending on whether or not a response is obtained. good.

Subsequently, using FIG. 14, an example of a processing flow in the case where the input / output device 130 determines whether or not the image pickup device 110 supports the extended function based on the response of the image pickup device 110 in S904 of FIG. Will be described. The input / output device 130 specifies the IP address of the image pickup device (for example, any of the image pickup device 110a to the image pickup device 110d) for which it is desired to confirm whether or not it has the extended function by, for example, a user operation (S1401). The input / output device 130 transmits a command inquiring whether or not the extended function is supported to the image pickup device having the specified IP address (S1402). The imaging device corresponding to the IP address specified by the input / output device 130 receives the request command from the input / output device 130 (S1403). Upon receiving the request command, the image pickup apparatus transmits a response including a parameter indicating that the own apparatus supports the extended function to the input / output apparatus 130 (S1404). When the input / output device 130 receives a response including a parameter corresponding to the extended function from the image pickup device (S1405), the input / output device 130 determines that the image pickup device corresponds to the extended function. When the input / output device 130 does not respond to the inquiry in S1402 or receives an error message, the input / output device 130 determines that the image pickup device does not support the extended function.

Next, an example of a processing flow in which the input / output device 130 refers to the designated data of the detachable device 100 in S905 of FIG. 9 will be described with reference to FIG. The data specified here is the data created in S1005 described above. The input / output device 130 transmits a reference request for the designated data (data created in S1005) of the detachable device 100 to the image pickup device 110 (S1501). When the image pickup device 110 receives the request command from the input / output device 130 (S1502), the image pickup device 110 requests the detachable device 100 to refer to the designated data (S1503). In addition, for reference to this designated data, a request in which a file name, a folder name, a file format, a storage area (address), etc., which are predetermined for the designated data are specified is transmitted. When the detachable device 100 receives the request command from the image pickup device 110 (S1504), the detachable device 100 transmits the designated data to the image pickup device 110 (S1505). The image pickup apparatus 110 receives a response including the designated data from the detachable device 100 (S1506). The processes from S1503 to S1506 can be executed as a process of reading data according to an interface standard such as an SD standard. However, the present invention is not limited to this, and any procedure can be used in which the image pickup apparatus 110 can acquire the designated data from the detachable device 100. The image pickup apparatus 110 transmits the designated data received from the detachable device 100 as a response to the request from the input / output apparatus 130 received in S1502 (S1507). The input / output device 130 receives the designated data stored in the detachable device 100 from the image pickup device 110 (S1508). In this way, the input / output device 130 can acquire the information of the detachable device 100 via the image pickup device 110.

As described above, in the present embodiment, the user may or may not have the extended function of the detachable device 100 by inquiring to the image pickup device 110 for the image pickup device 110 corresponding to the function addition by the detachable device 100. Can be identified. Thereby, for example, the user can easily recognize that the removable device 100 having no extended function is erroneously attached to the image pickup device 110 that supports the addition of functions by the removable device 100. Further, the user can refer to the designated data in the detachable device 100 for the imaging device 110 that does not support the addition of the function by the detachable device 100, so that the user can have the extended function of the detachable device 100. Can be identified. That is, the input / output device 130 accesses the detachable device 100, and when the designated data exists, the detachable device 100 has an extended function, while the image pickup device 110 has such a function. You can confirm that it does not support expansion. On the other hand, the input / output device 130 accesses the detachable device 100, and when the designated data does not exist, the detachable device 100 does not have the extended function, and the image pickup device 110 has such a function. You can confirm that it does not support expansion. In this way, the user can confirm whether or not the removable device 100 has an extended function regardless of whether or not the image pickup device 110 supports the addition of functions by the removable device 100. As a result, for example, the user can easily identify that the removable device 100 having an extended function is erroneously attached to the image pickup device 110 that does not support the addition of functions by the removable device 100.

In the above-described embodiment, the designated data explicitly indicates that the image pickup apparatus 110 does not support the expansion of the function by the detachable device 100 (the detachable device 100 cannot execute the process). An example has been described. However, it is not limited to this. That is, since the designated data is stored when the image pickup device 110 does not support the expansion of the function by the detachable device 100, the image pickup device 110 does not support the expansion of the function by the detachable device 100. Does not need to be shown. In this case, for example, only the information of the detachable device 100 may be shown in the designated data.

<< Other Embodiments >>
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

The invention is not limited to the above embodiments, and various modifications and modifications can be made without departing from the spirit and scope of the invention. Therefore, a claim is attached to make the scope of the invention public.

100a to 100d: Detachable device, 110a to 110d: Imaging device, 120: Network, 130: Input / output device, 201: Imaging unit, 202: Image processing unit, 203: Arithmetic processing unit, 204: Distribution unit, 205: SD I / F unit, 301: Imaging control unit, 302: Signal processing unit, 303: Storage unit, 304: Control unit, 305: Analysis unit, 306: Device communication unit, 307: Network communication unit, 401: I / F unit , 402: FPGA, 403: SD controller, 404: storage unit, 501: analysis unit, 502: communication unit, 601: processor, 602: RAM, 603: ROM, 604: HDD, 605: communication I / F, 701: Network communication unit, 702: control unit, 703: display unit, 704: operation unit

Claims (17)

The device to be mounted on a device having a mounting mechanism capable of attaching and detaching at least a device capable of storing captured images.
A processing means capable of executing a predetermined process on image data based on an image captured by the image pickup device when mounted on an image pickup device having the mounting mechanism.
A determination means for determining whether or not the image pickup apparatus has a function capable of causing the processing means to perform the predetermined processing, and
A storage means for storing predetermined data when it is determined that the imaging device does not have the function.
A device characterized by having. The device according to claim 1, wherein the storage means does not store the predetermined data when the imaging device has the function. The storage means is characterized in that, when the imaging device has the function and stores the predetermined data, the storage means deletes or rewrites the predetermined data. Or the device according to 2. The determination means is characterized in that it transmits a first signal to the image pickup apparatus and determines whether or not the image pickup apparatus has the function based on whether or not a second signal is received as a response thereof. The device according to any one of claims 1 to 3. The first signal is a response to a command received from the imaging device, the response in which a predetermined field contains information indicating that the device can perform the predetermined process. The device according to claim 4, wherein the device is characterized by the above. The device according to claim 5, wherein the predetermined field is a field that is ignored by the image pickup apparatus that does not have the function. The determination means according to any one of claims 4 to 6, wherein the determination means periodically transmits the first signal at regular intervals to determine whether or not the image pickup apparatus has the function. Described device. The device according to any one of claims 1 to 7, wherein the predetermined data is data having at least one of a specific format and a specific file name. The device according to any one of claims 1 to 8, wherein the predetermined data is stored in a folder having a specific folder name. The device according to any one of claims 1 to 9, wherein the predetermined data is stored in a predetermined specific area of the storage means. Any of claims 1 to 10, wherein the storage means deletes or rewrites the predetermined data when the imaging device changes from a state in which the image pickup device does not have the function to a state in which the image pickup device has the function. The device according to item 1. A communication device capable of accessing at least an imaging device having a mounting mechanism capable of attaching and detaching a device capable of storing captured images and a device mounted on the mounting mechanism.
Refer to the predetermined data stored in the device when the image pickup device does not have a function of causing the device mounted on the image pickup device to perform a predetermined process on the image data based on the image captured by the image pickup device. A communication device comprising a specific means for identifying that the device capable of performing the predetermined process is attached to the image pickup device having no function. The specific means cannot execute the predetermined process on the image pickup device having no function when the predetermined data is not stored in the device when the image pickup device does not have the function. The communication device according to claim 12, wherein the device is specified to be attached. When the imaging device has the function, the specific means specifies whether or not the device mounted on the imaging device can execute the predetermined process by inquiring to the imaging device. The communication device according to claim 12 or 13, wherein the communication device. A method performed by said device mounted on a device having a mounting mechanism capable of attaching and detaching at least a device capable of storing captured images.
The device has a processing means capable of executing a predetermined process on image data based on an image captured by the image pickup device when the device is mounted on the image pickup device having the mounting mechanism.
The method is
Determining whether or not the image pickup apparatus has a function capable of causing the processing means to perform the predetermined processing.
When it is determined that the imaging device does not have the function, predetermined data is stored and
A method characterized by including. A method performed by an imaging device having a mounting mechanism capable of attaching and detaching at least a device capable of storing captured images, and a communication device capable of accessing the device mounted on the mounting mechanism.
Refer to the predetermined data stored in the device when the image pickup device does not have a function of causing the device mounted on the image pickup device to perform a predetermined process on the image data based on the image captured by the image pickup device. A method comprising identifying that the device capable of performing the predetermined process is attached to the image pickup apparatus having no function. A program for causing a computer to function as the device according to any one of claims 1 to 11 or the communication device according to any one of claims 12 to 14.

Images