JP6519805B2 - Camera system and camera device - Google Patents

Description

  The present invention relates to a camera system and a camera apparatus for imaging a desired subject and detecting an object in a predetermined area in a captured image.

  Camera systems are used in a variety of applications, such as surveillance applications that monitor the conditions of a particular location. In a camera system, a camera apparatus for imaging a subject adjusts an angle of view of an imaging lens to set an imaging area, or uses a PTZ (pan-zoom-tilt) mechanism, in order to image a desired area. It has functions such as displacing the imaging lens to make the imaging area adjustable arbitrarily.

  In this type of camera device, when a specific subject is continuously imaged, the direction and zoom amount of the imaging lens are determined so as to be an appropriate imaging area including the subject to be imaged, and the imaging area is fixed. In this state, the position of the imaging lens is fixed. However, there is a possibility that the image position of the captured image captured by the camera device may change due to secular change, etc., and due to the positional deviation of the captured image, the target subject can not be captured properly and monitoring of the target is normal. Challenges such as inability arise.

  For example, Patent Document 1 discloses an image misalignment detection device capable of detecting the occurrence of misalignment of a captured image. In the image positional deviation detection apparatus of this conventional example, a correction image obtained by performing tone correction processing on a time accumulation image of an input image is created, a correlation value between the correction image and the reference image is calculated, and the correlation value is calculated. Present the misalignment based on the As a result, even if a large change occurs in the subject to be photographed by the camera and the surrounding environment due to the secular change, the positional deviation can be detected.

Unexamined-Japanese-Patent No. 2007-208513

  In the camera system, when detecting an object in a specific area in the imaging area, for example, the detection area is set in the captured image, and the object appearing in the set detection area is detected. In this case, positional deviation of the captured image occurs due to aging or the like, and if the position of the set detection area changes, detection omission or false detection may occur with respect to the assumed event. In addition, when an environmental change such as layout change, movement or increase / decrease of the object occurs in the imaging area, the positional relationship between the detection area and the object initially set may be deviated, which may cause detection omission or erroneous detection. There is. It is difficult to detect an object by adapting to such various changes in the captured image, and it was not possible to easily set an appropriate detection region.

  In order to solve the above-described conventional problems, the present invention provides a camera system and a camera apparatus capable of detecting an object with high accuracy by performing appropriate area setting even when a change occurs in a captured image. Intended to be provided.

The present invention is a camera system in which a camera device and a server device are connected to each other, and the camera device captures an image of a target area, and performs detection of an object in the captured image of the target area. Setting an area including the setting of the event information, and when a predetermined detection event occurs in the detection area in the captured image, determine event occurrence and transmit event detection information to the server apparatus; wherein determining the presence or absence of a change in the area setting, you send area change detection information to the server device when the change in the area setting has occurred, to provide a camera system.

Further, the present invention is a camera device for imaging a target area, and in the captured image of the target area, an area setting unit that performs area setting including setting of a detection area for executing detection of an object; When a predetermined detection event occurs in a detection area in an image, the first detection unit that determines event occurrence and generates event detection information, and determines whether there is a change in the area setting at a predetermined determination timing and, and a second detection portion that forms the raw area change detection information when the change in the area setting has occurred, to provide a camera device.

  According to the present invention, even when a change occurs in a captured image, an object can be detected with high accuracy by performing appropriate area setting.

Block diagram showing system configuration of camera system of the present embodiment (A), (B) A diagram showing an example of a setting screen for setting an area in a captured image Block diagram showing the functional internal configuration of the camera device and the server device of the first embodiment A flowchart showing a first example of the process of detecting deviation of area setting in the camera device of the first embodiment A flowchart showing a second example of the process of detecting deviation of area setting in the camera device of the first embodiment (A), (B) A diagram showing an example of a notification screen in the case of detecting an area setting deviation Block diagram showing the functional internal configuration of the camera device and the server device of the second embodiment A flowchart showing a first example of the process of detecting deviation of area setting in the camera device of the second embodiment A flowchart showing a second example of the process of detecting deviation of area setting in the camera device of the second embodiment

  Hereinafter, an embodiment (hereinafter, referred to as “the present embodiment”) which specifically discloses a camera system and a camera apparatus according to the present invention will be described in detail with reference to the drawings as appropriate. However, the detailed description may be omitted if necessary. For example, detailed description of already well-known matters and redundant description of substantially the same configuration may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. It is to be understood that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and they are not intended to limit the claimed subject matter.

  In the present embodiment described below, for example, as shown in FIG. 1, a plurality of sites (site A (100A), site B (100B),...) And an operating headquarters 200 that manages the camera system The explanation will be given assuming the connected system form. However, embodiments of the camera system and the camera apparatus according to the present invention are not limited to the contents of the present embodiment described later.

(System configuration)
FIG. 1 is a block diagram showing a system configuration of a camera system of the present embodiment. In the camera system, a plurality of sites 100A, 100B,... And an operation headquarters 200 that manages the camera system are mutually connected via the network 300, and transmission and reception of image data etc. taken at each site is not possible. It will be. A plurality of camera devices 10A, 10B,... 10N, a server device 20, a recorder 30, an input device 41, and a monitor 42 are provided in the site A (100A), and these devices are connected via the network 70. Connected to each other. The networks 300, 70 are wireless networks or wired networks. The wireless network is, for example, a wireless local area network (LAN), a wide area network (WAN), 3G, long term evolution (LTE), or wireless gigabit (WiGig). The wired network is, for example, an intranet or the Internet. Although not shown, a camera apparatus, a server apparatus, a recorder, an input device, and a monitor are provided at site B (100B) as well as site A (100A).

  Camera devices 10 (hereinafter, representative of camera devices 10A, 10B,... 10N) are respectively installed at desired positions in site 100 (hereinafter, representative of site A (100A), site B (100B),. The subject in the imaging area set at each installation position is imaged. The detailed configuration of the camera device 10 will be described later. The server device 20 is configured using, for example, a PC (Personal Computer), and performs various settings, control, and the like of the camera device 10 according to an input operation of a user using the input device 41. At this time, the server device 20 displays on the monitor 42 a menu screen, a setting screen, a control screen and the like regarding management of the camera device 10. Further, the server device 20 acquires image data of an image obtained by the imaging of each camera device 10 (hereinafter, an image obtained by the imaging of the camera device is referred to as “captured image”), and the monitor 42 obtains the captured image. Display. In addition, the server device 20 displays the detection notification screen on the monitor 42, transmits the detection notification to the management headquarters 200, and the like according to the detection result of the object by the camera device 10.

  The recorder 30 is configured using, for example, a semiconductor memory or a hard disk drive, and stores image data of a captured image captured by each camera device 10 installed in the site 100. The image data of the captured image stored in the recorder 30 is provided for monitoring work such as crime prevention, security, and material management, for example.

  The input device 41 is configured using, for example, a mouse, a keyboard, a touch panel, or a touch pad, and outputs a signal corresponding to an input operation of the user to the camera device 10 or the server device 20. Although FIG. 1 shows the configuration in which the input device 41 is connected to the server device 20, the input device 41 may be connected to the camera device 10.

  The monitor 42 is configured using, for example, an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence), and displays a captured image captured by the camera device 10 and various screens generated by the server device 20. Although the monitor 42 is provided as an external device different from the server device 20, the monitor 42 may be included in the server device 20.

  The operation headquarters 200 is provided with a server device 210, an input device 221, and a monitor 222, and browsing processing of various information related to captured images such as browsing of captured images captured at each site 100, confirmation of detection results of objects, etc. To be done. The server device 210 causes the monitor 222 to display a display screen of various information such as a captured image and a detection result in response to an input operation by the user using the input device 221. Further, the server device 210 may have the same function as the server device 20 of each site 100, and may be capable of processing various settings, control, and the like of the camera device 10.

(Area setting)
When detecting an object in a specific area in a camera system, the user sets a detection area for performing detection of an object and a mask area excluding detection determination in a captured image captured by the camera device 10. The setting of the area including the detection area and the mask area is referred to as “area setting”, and the area including the detection area and the mask area set by the area setting is referred to as “setting area”. The camera device 10 determines that a predetermined detection event has occurred when the presence, movement, or other events of a specific object occur in a detection area in a captured image, and records the event occurrence, the server device 20 An action at the time of occurrence of an event such as transmission of event detection information to it is executed. Also, even if an event occurs in the mask area, the camera device 10 determines that the event is not a predetermined detection event, and does not execute the event occurrence operation. When receiving the event detection information from the camera device 10, the server device 20 displays a notification screen notifying the occurrence of the detection event on the monitor 42, and notifies the user that a predetermined event has been detected.

  FIG. 2 is a diagram showing an example of a setting screen for setting an area in a captured image. The setting screen, an example of which is shown in FIG. 2, is displayed on the monitor 42 provided in the site 100 or the monitor 222 provided in the operation headquarters 200, and area setting processing is executed by the operation input of the input devices 41 and 221. . Hereinafter, the case where the area setting process is performed by the server device 20, the monitor 42, and the input device 41 in the site 100 will be described.

  FIG. 2A shows a first example of an area setting method for setting a detection area and a mask area by specifying an area. The camera device 10 transmits a captured image to the server device 20. The server device 20 displays the captured image 50 acquired by the camera device 10 on the monitor 42, as in the setting screen on the left side in the drawing. The camera device 10 recognizes boundaries, contours, specific objects, and the like based on values of color, brightness, and the like in the captured image 50, and changes, and performs area division of the captured image. Then, the server device 20 displays an area frame with outlines indicating the respective areas 51a, 51b, 51c, and 51d divided into areas on the captured image, an area surface with a filled or shaded plane, and the like. In the setting screen, first, the user designates the “detection area” or the “mask area” in the lower left portion of the setting screen by the input device 41, and selects the area type to be set. Then, the user designates an area desired to be set as a detection area or a mask area by the input device 41 from each area 51a, 51b,... Divided into areas, selects a target area, and designates the setting button 52 in the lower right part of the setting screen To set a detection area or a mask area. The setting instruction is sent to the camera device 10, and setting of the detection region and the mask region is performed in the camera device 10. As a result, the set detection area 55 and the mask area 56 are displayed as in the setting screen after the area setting on the right side in the drawing. By instructing the OK button 57 on the lower right portion of the setting screen after area setting, the area setting process is completed, and the area setting information by the user operation is stored in the camera device 10.

  FIG. 2B shows a second example of the area setting method for setting the detection area and the mask area by selecting the feature points. The server device 20 displays the captured image 50 acquired by the camera device 10 on the monitor 42, as in the setting screen on the left side in the drawing. The camera device 10 recognizes boundaries, contours, specific objects, and the like based on values of color, brightness, and the like in the captured image 50, and changes, and extracts feature points in the captured image. Then, the server device 20 displays the feature points 53a, 53b, 53c, ... 53k, ... 53n extracted on the captured image. In the setting screen, first, the user designates the “detection area” or the “mask area” in the lower left portion of the setting screen by the input device 41, and selects the area type to be set. Then, from the extracted feature points 53a to 53n, the target device is determined by designating a plurality of feature points corresponding to the area that the user wants to set in the detection area or the mask area by the input device 41, and setting the lower right part of the setting screen The button 52 is instructed to set a detection area or a mask area. The setting instruction is sent to the camera device 10, and setting of the detection region and the mask region is performed in the camera device 10. As a result, the set detection area 55 and the mask area 56 are displayed as in the setting screen after the area setting on the right side in the drawing. By instructing the OK button 57 on the lower right portion of the setting screen after area setting, the area setting process is completed, and the area setting information by the user operation is stored in the camera device 10.

First Embodiment
<Configuration of Camera Device>
FIG. 3 is a block diagram showing a functional internal configuration of the camera device and the server device of the first embodiment. The camera device 10 includes an imaging unit 101, an image input unit 102, a feature extraction unit 103, an area setting unit 104, an area setting information storage unit 105, a shift / change detection unit 106, a transmission unit 107, and a reception unit 108. . The feature extraction unit 103 is configured to include a region division unit 131 and a feature point extraction unit 132. The shift / change detection unit 106 includes a shift determination unit 161 and a change extraction unit 162.

  The imaging unit 101 at least includes a lens and an image sensor. The lens condenses light (light beam) incident from the outside of the camera device 10 and forms an image on a predetermined imaging surface of the image sensor. As the lens, a fisheye lens or a wide-angle lens capable of obtaining an angle of view of, for example, 140 degrees or more is used. The image sensor is, for example, a solid-state imaging device such as a CCD (Charged-Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and converts an optical image formed on an imaging surface into an electrical signal. Note that the imaging unit 101 is any imaging optical system such as an omnidirectional camera device, a fixed camera device with a fixed angle of view, or a PTZ camera device having a PTZ mechanism having a pan direction or tilt direction and a zoom function. It may have the

  The image input unit 102 is configured using, for example, a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP), and performs predetermined signal processing using an electrical signal from the imaging unit 101. Then, data (frames) of a captured image of a predetermined frame rate (for example, 30 fps (frames per second)) defined by human-recognizable RGB (Red Green Blue) or YUV (luminance / color difference) or the like is generated. The image input unit 102 outputs the data of the generated captured image to the feature extraction unit 103.

  The feature extraction unit 103 is configured using, for example, a CPU, an MPU or a DSP, and performs predetermined image processing (for example, area division) for each data of a predetermined time interval using data of a captured image output from the image input unit 102 Processing and feature point extraction processing are performed to extract feature information based on regions, feature points, and the like in the captured image. The area dividing unit 131 recognizes a feature such as a boundary, an outline, or a specific object based on values of color, brightness, and the like in a captured image and a feature such as a specific object, and performs area division processing for dividing the captured image into a plurality of areas. As a processing result of the area division processing, division area information including information such as the number of each divided area, the position (coordinates of the boundary and the like), and the area is acquired. The feature point extraction unit 132 recognizes a boundary, an outline, a specific object, and the like based on values of color, brightness, and the like in the captured image, and a change amount, and performs feature point extraction processing for extracting a plurality of feature points in the captured image. As processing results of the feature point extraction processing, feature point information including information such as the number of extracted feature points, positions (coordinates of feature points), and feature vectors is acquired.

  The area setting unit 104 excludes the detection area for performing the detection of the object and the detection determination according to the procedure using FIG. 2 described above using the divided area information or the feature point information extracted by the feature extraction unit 103. Set the area with the mask area to be set. The area setting unit 104 outputs area setting information on the detection area and the mask area to the area setting information storage unit 105. The area setting information storage unit 105 is configured using, for example, a semiconductor memory or a hard disk drive, and stores area setting information of the detection area and the mask area set by the area setting unit 104.

  The shift / change detection unit 106 is configured using, for example, a CPU, an MPU, or a DSP, and division area information or feature point information generated by the feature extraction unit 103, and area setting stored in the area setting information accumulation unit 105. The information is used to detect the occurrence of misalignment or change in relation to the setting of the detection area and the mask area. As a method of detecting the displacement of the setting area, a known motion detection method in an image may be used. For example, a pixel without motion in a predetermined period (long term or short term) is regarded as an immobile area with no feature change (no motion), and features by edge, color, luminance, etc. of this immobile area are extracted and extracted A method or the like is used to calculate the deviation by finding the amount of change of the feature portion by pattern matching or point matching of the feature. As the deviation of the setting area, expansion or reduction of the area of each area, movement of each area (increase in positional error of feature points) and the like are detected. Here, the following detection method is used.

  The shift determination unit 161 compares the area setting information with the divided area information or feature point information, and the difference between the position of the setting area set previously and the position of the area division result or feature point in the current captured image is It is determined whether or not the difference is equal to or more than a predetermined value, and when the difference in position is equal to or more than the predetermined value, it is determined that the positional deviation has occurred. The change extraction unit 162 compares the area setting information with the divided area information or the feature point information, and determines the position between the previously set setting area and the area division result in the current captured image or the extraction area according to the feature points. Alternatively, it is determined whether or not the difference in area is equal to or greater than a predetermined value, and it is determined that there is a region change when the difference in position or area is equal to or greater than the predetermined value. The shift / change detection unit 106 outputs area change detection information when it detects a positional shift or change relating to area setting as a processing result of the shift detection process.

  The transmitting unit 107 includes a communication circuit for performing wired communication or wireless communication, and divided area information or feature point information generated by the feature extracting unit 103, and area setting information generated by the area setting unit 104. The area change detection information generated by the deviation / change detection unit 106 is transmitted to the server device 20. The receiving unit 108 includes a communication circuit that performs wired communication or wireless communication, and receives input information transmitted from the server device 20. Although the transmission unit 107 and the reception unit 108 are separately illustrated in the illustrated example, a configuration including a communication unit including the transmission unit 107 and the reception unit 108 may be employed.

  The area setting unit 104 designates an area or feature point related to the detection area and the mask area with respect to the divided area information or feature point information generated by the feature extraction unit 103 according to the input information from the server device 20 based on the user operation. Input and execute area setting of detection area and mask area.

  The server device 20 is configured to include an input information receiving unit 201, a display image generating unit 202, a receiving unit 203, and a transmitting unit 204. The input information receiving unit 201 is configured to have an interface circuit such as USB (Universal Serial Bus), for example, receives input information on a region setting instruction output from the input device 41 based on a user operation, and generates a display image The signal is output to the unit 202 and the transmitting unit 204. The display image generation unit 202 is configured to have an interface circuit such as High-Definition Multimedia Interface (HDMI) (registered trademark), and the reception information from the camera device 10 received by the reception unit 203, and the input information reception unit A display image signal is generated using the input information from 201 and output to the monitor 42. The display image generation unit 202 notifies of the detection of the setting screen regarding the setting of the captured image and the area, and the setting of the area based on the captured image data, the divided area information or the feature point information, the area setting information, and the area change detection information. The display image such as the notification screen is generated and displayed on the monitor 42.

<Displacement detection processing of area setting>
In the first embodiment, a configuration example will be described in which a shift of a setting area caused by a change in a captured image due to a position shift of a lens of a camera device or a change in angle of view is detected.

  FIG. 4 is a flowchart showing a first example of the process of detecting the deviation of the area setting in the camera device of the first embodiment. The first example shows a process of detecting deviation of area setting in the case of performing area setting by area specification shown in FIG. 2A.

  The camera device 10 determines whether it is time to determine the occurrence of the deviation of the area setting in the deviation / change detection unit 106 (S11). The determination timing of the area setting deviation is, for example, when the predetermined time such as one hour has elapsed or when the imaging angle of view is changed by the PTZ mechanism, the deviation determination every predetermined time or the position change of the imaged image It is possible to determine the time shift. At this time, in the camera device 10, it is assumed that the area setting of the detection area and the mask area by area division has already been made.

  When the determination timing of the area setting deviation comes, the camera device 10 causes the image input unit 102 to acquire a captured image captured by the imaging unit 101 (S12). Subsequently, the camera device 10 performs area division processing in the area division unit 131 of the feature extraction unit 103, and performs area division of the currently acquired captured image (S13). Then, the camera device 10 executes the change determination processing of determining the presence / absence of the change of the area setting in the shift determination unit 161 and the change extraction unit 162 of the shift / change detection unit 106 (S14).

  The shift determination unit 161 and the change extraction unit 162 first obtain area setting information stored in the area setting information storage unit 105 as the change determination process (S141). Then, the divided area information generated by the area dividing unit 131 is acquired, and the position setting area of each area divided into areas is compared by comparing the previously set area setting information with the current divided area information. (S142). Next, the shift determination unit 161 and the change extraction unit 162 determine whether there is a difference between the position and area of each of the divided regions (S143). Here, the shift determination unit 161 and the change extraction unit 162 determine that there is a change in each of the divided regions if there is a difference greater than or equal to a predetermined value (S144). On the other hand, when there is no difference greater than or equal to the predetermined value, the deviation determination unit 161 and the change extraction unit 162 determine that there is no change in each of the divided regions (S145). With this change determination processing, it is determined whether there is a change in the area regarding the area setting of the detection area and the mask area by area division.

  The camera device 10 determines in the shift / change detection unit 106 whether a shift or change in the detection area and the mask area is detected based on the processing result of the change determination process (S15). When a shift or change in the set area is detected, the camera apparatus 10 updates the area setting information in the area setting unit 104 according to the change in the area in the captured image, and adds new area setting information to the area setting information storage unit 105. (S16). Then, the camera device 10 transmits the area change detection information and the area setting information from the transmitting unit 107 to the server device 20, causes the monitor 42 to display the updated information of the set area, and prompts the confirmation of the update of the setting area S17).

  FIG. 5 is a flowchart showing a second example of the process of detecting the deviation of the area setting in the camera device of the first embodiment. The second example shows a process of detecting deviation of area setting in the case of performing area setting by feature point selection shown in FIG. 2 (B).

  Similar to the first example of FIG. 4, the camera device 10 determines whether it is time to determine the occurrence of the deviation of the area setting in the deviation / change detection unit 106 (S21). At this time, in the camera device 10, it is assumed that the area setting of the detection area and the mask area has already been performed by the feature point selection. When the determination timing of the area setting deviation comes, the camera device 10 causes the image input unit 102 to acquire a captured image captured by the imaging unit 101 (S22). Subsequently, the camera device 10 performs feature point extraction processing in the feature point extraction unit 132 of the feature extraction unit 103, and performs extraction of feature points in the currently acquired captured image (S23). Then, in the camera device 10, the shift determination unit 161 and the change extraction unit 162 of the shift / change detection unit 106 execute a change determination process of determining the presence / absence of a change in the area setting (S24).

  The shift determination unit 161 and the change extraction unit 162 first obtain the area setting information accumulated in the area setting information accumulation unit 105 and the feature point information extracted by the feature point extraction unit 132 as the change determination process, and The selected points of the feature points of the area set in are matched with the currently extracted feature points (S241). Then, an error in the position between the corresponding selected point and the feature point where the two features match is calculated (S 242). Next, the deviation determination unit 161 and the change extraction unit 162 determine whether the error between the corresponding selection point and the feature point is equal to or more than a predetermined threshold (S243). Here, when the error is equal to or greater than the threshold, the deviation determination unit 161 and the change extraction unit 162 determine that there is a change in each feature point related to the area setting (S244). On the other hand, when the error is less than the threshold, the deviation determination unit 161 and the change extraction unit 162 determine that there is no change in each feature point related to the area setting (S245). With this change determination process, it is determined whether there is a change in the area regarding the area setting of the detection area and the mask area by the feature point selection.

  The camera device 10 determines whether the shift or change in the detection area and the mask area has been detected in the shift / change detection unit 106 based on the processing result of the change determination process (S25). When a shift or change in the set area is detected, the camera apparatus 10 updates the area setting information in the area setting unit 104 according to the change in the area in the captured image, and stores the new area setting information in the area setting information. The information is stored in the unit 105 (S26). Then, the camera device 10 transmits the area change detection information and the area setting information from the transmitting unit 107 to the server device 20, causes the monitor 42 to display the updated information of the set area, and prompts the confirmation of the update of the setting area S27).

  In the procedures of the flowcharts of FIGS. 4 and 5, when a shift or a change in the set area is detected, the area setting information is updated to notify the user of a new detection area and a mask area. Alternatively, the user may be notified of detection of a shift or change of the set area, and the user may be prompted to change the area setting.

  FIG. 6 is a diagram showing an example of a notification screen when the area setting deviation is detected. FIG. 6A shows a notification screen 61 of a first example in which the user is notified of the detection of the deviation of the setting area and the user is allowed to receive the change operation of the area setting. In this case, on the notification screen 61, the server device 20 displays on the monitor 42 the captured image and setting area 65 at the time of the previous area setting and the captured image and setting area 66 in the state where the current area change occurs. . Then, when the area change button 68 in the lower left portion of the notification screen is instructed, an instruction input is transmitted to the camera device 10, and the camera device 10 executes the area setting process again to update the area setting information.

  FIG. 6B shows a second example of notifying the user of detection of the deviation of the setting area, presenting the user with the setting area proposal after the change, and accepting the permission operation of the area setting change from the user. It shows a screen 62. In this case, the camera device 10 performs processing such as feature point extraction and area division on the current captured image, and changes the position and area of the setting area so that the target area in which the area has changed is in an appropriate state. By this, a setting area proposal is generated. In the notification screen 61, the server device 20 displays on the monitor 42 the captured image and the setting area 66 in the state where the current area change has occurred, and the setting area plan 67 in which the target area where the area change occurred is appropriately changed. Do. Then, when the area change button 69 in the setting plan on the lower right part of the notification screen is instructed, an instruction input is transmitted to the camera device 10, and the camera device 10 executes the area setting processing to adopt the setting area plan. , Update area setting information. When the area change button 68 in the lower left portion of the notification screen is instructed, the server device 20 transmits an instruction input to the camera device 10 as in the first example, and the camera device 10 executes the area setting process again. And update the area setting information.

  For example, in the example illustrated in FIG. 6A, in the current captured image on the right side, the angle of view of the camera device 10 moves downward from before, and the captured image is shifted upward. In this case, the same feature point as the previous captured image is moved upward, and the detection area and the mask area are shifted downward with respect to the captured image. For this reason, a part of the part which should be a detection area becomes a mask area, and there is a possibility that a detection omission may occur. In the present embodiment, the area setting deviation can be detected by determining the change of the setting area or the feature point, and the setting change of the detection area and the mask area appropriate for the current captured image can be made.

  As described above, in the present embodiment, the camera apparatus operates in a state in which the detection operation is performed by performing area setting using area division of the captured image or area setting using feature point detection in the captured image. When a shift in the set area occurs due to a change in the captured image due to a lens shift or a change in the angle of view, a change in the area setting of the detection area and the mask area is detected It is possible to change the area setting according to the amount. Further, by extracting and using feature points in area setting and area setting change, area setting of a new detection area and mask area can be easily performed even if positional deviation of a captured image occurs. At this time, not only positional deviation of parallel movement but also various positional changes such as rotation and scaling can be coped with for each area of the detection area and the mask area. Therefore, even when a change occurs in the captured image due to a change in the angle of view of the imaging lens of the camera device, a change in the imaging region, or the like, the object can be detected with high accuracy by setting the appropriate region.

Second Embodiment
<Configuration of Camera Device>
FIG. 7 is a block diagram showing a functional internal configuration of the camera device and the server device of the second embodiment. The second embodiment is an example in which the configuration of the camera apparatus is partially changed. The camera apparatus 10-1 includes an imaging unit 101, an image input unit 102, a feature extraction unit 103-1, an area setting unit 104, an area setting information storage unit 105, a shift / change detection unit 106, a transmission unit 107, a reception unit 108, The configuration includes a detection history information storage unit 109. That is, in addition to the configuration of the first embodiment shown in FIG. 3, the camera apparatus 10-1 includes a detection history information storage unit 109. The feature extraction unit 103-1 is configured to include a region division unit 131, a feature point extraction unit 132, and a detection history feature extraction unit 133. In the description of each part of the camera device 10-1 shown in FIG. 7, the same components and operations as those of the camera device 10 shown in FIG. 3 are assigned the same reference numerals and explanations thereof will be omitted, and different contents will be described.

  The detection history information storage unit 109 is configured using, for example, a semiconductor memory or a hard disk device, and stores statistical information of the history detected in the captured image at a predetermined time in the past as detection history information. As statistical information of detection history, for example, flow line information (for example, “target position information”, “flow line information”, and “passing status or” about the staying position or passing position of the moving object included in the captured image of the target area) Information on the status of residence ”is extracted. Flow line analysis information such as the number of detected moving objects, residence time, heat map (flow line analysis image), etc. acquired by this flow line analysis process It may be used.

<Displacement detection processing of area setting>
In the second embodiment, a configuration example will be described in which a shift of a setting area caused by a change in a captured image due to a change in an imaging target is detected. In this case, depending on the amount of detection events in the captured image, processing suitable for each case is performed.

  FIG. 8 is a flowchart showing a first example of the process of detecting the deviation of the area setting in the camera device of the second embodiment. The first example shows the process of detecting the deviation of the area setting when the number of detected events in the captured image is relatively large. In a camera system that counts the number of people and detects staying at a site where a person appears steadily, such as a store, relatively many detection events occur when person detection is a detection event. The first example is a processing example suitable for such a case.

  The camera device 10-1 determines whether it is the timing to determine the occurrence of the deviation of the area setting in the deviation / change detection unit 106 (S31). The determination timing of the area setting deviation is, for example, when the predetermined time such as one hour has elapsed or when the imaging angle of view is changed by the PTZ mechanism, the deviation determination every predetermined time or the position change of the imaged image It is possible to determine the time shift. At this time, in the camera device 10-1, it is assumed that the areas of the detection area and the mask area have already been set.

  When the determination timing of the area setting deviation comes, the camera device 10-1 detects the past predetermined period stored in the detection history information storage unit 109 in the detection history feature extraction unit 133 of the feature extraction unit 103-1. History information is acquired (S32). Subsequently, the detection history feature extraction unit 133 acquires current detection history information (S33). Next, the camera device 10-1 compares the past and present detection history information in the change extraction unit 162 of the shift / change detection unit 106 (S34).

  The camera device 10-1 determines whether or not there is a change in the detection history in the shift / change detection unit 106 (S35), and determines that there is a change in the area setting when a change of a predetermined amount or more is detected in the detection history. Do. In this case, the region setting unit 104 performs region setting processing using the current captured image, and stores new region setting information in the region setting information storage unit 105 (S36). Then, the camera device 10-1 transmits the area change detection information and the area setting information from the transmitting unit 107 to the server device 20, causes the monitor 42 to display the changed information of the set area, and presents the area setting plan to the user. (S37). The camera device 10-1 receives an instruction input from the server device 20, and performs processing for changing or maintaining the area setting (S38).

  FIG. 9 is a flowchart showing a second example of the process of detecting the deviation of the area setting in the camera device of the second embodiment. The second example shows the process of detecting the deviation of the area setting when the number of detection events in the captured image is relatively small. In a camera system that monitors an intruder or a suspicious person, the frequency of occurrence of person detection or the like is low in a detection area where no person normally exists, and the occurrence of detection events is relatively small. The second example is a processing example suitable for such a case.

  Similar to the first example of FIG. 8, the camera device 10-1 determines whether it is time to determine the occurrence of an area setting deviation in the deviation / change detection unit 106 (S <b> 41). At this time, in the camera device 10-1, it is assumed that the areas of the detection area and the mask area have already been set. When it comes to the determination timing of the region setting deviation, the camera device 10-1 causes the feature point extraction unit 132 of the feature extraction unit 103-1 to use a short-term average image or a long-term average image as an average image for a predetermined period regarding the captured image. An average image is acquired (S42). The time interval for generating the average image may be set as appropriate. For example, an average image every 10 minutes in the short term and an average image every two hours in the long term are created and used. Subsequently, the feature point extraction unit 132 extracts features such as edges, colors, and luminance with respect to the short-term average image or the long-term average image (S43).

  Next, the camera device 10-1 causes the feature point extraction unit 132 to acquire the current captured image (S44), and extract features such as edge, color, and luminance with respect to the current captured image (S45). Then, the camera device 10-1 compares the feature of the short-term average image or the long-term average image with the current captured image in the change extraction unit 162 of the shift / change detection unit 106 (S46).

  The camera apparatus 10-1 determines whether the image feature has a change in the shift / change detection unit 106 (S47), and if a change of a predetermined amount or more is detected in the image feature, there is a change in the area setting. It is determined that In this case, the region setting unit 104 performs region setting processing using the current captured image, and stores new region setting information in the region setting information storage unit 105 (S48). Then, the camera device 10-1 transmits the area change detection information and the area setting information from the transmitting unit 107 to the server device 20, causes the monitor 42 to display the changed information of the set area, and presents the area setting plan to the user. (S49). The camera device 10-1 receives an instruction input from the server device 20, and performs processing for changing or maintaining the area setting (S50).

  When an environmental change in the imaging area occurs, such as increase or decrease of a structure in the imaging area, movement of an installation, or the like, detection omission and false detection may occur in the detection area and the mask area set initially. For example, if there is no structure that was in the mask area, and there is a possibility that a person may exist or pass through, the mask area is shrunk and the detection area is enlarged in order to prevent detection omission. There is a need. As in the first example described above, when a relatively large number of detection events occur, when the detection tendency such as the number of detections, residence time, heat map, etc. has changed significantly from before by comparing the detection history information. An area setting deviation can be detected, and settings of the detection area and the mask area can be changed appropriately according to the current captured image. Also, as in the second example described above, when the occurrence of the detection event is relatively small, the position of the feature point or the area division is obtained by comparing the short-term or long-term average image with the current captured image. When it changes, the area setting deviation can be detected, and the setting and changing of the detection area and the mask area can be appropriately performed in accordance with the current captured image.

  As described above, in the present embodiment, when a shift in the set area occurs due to a change in the captured image due to a change in the imaging target, the area setting of the detection area and the mask area is performed by comparison with the detection history information or the average image. It is possible to detect a change, to notify the user of the area setting deviation, and to change the area setting according to the deviation amount. Therefore, even when a change occurs in the captured image due to an environmental change such as a layout change, movement, increase or decrease, etc. of the object, it is possible to set an appropriate region and detect the object with high accuracy.

  As a site using the camera system of the present embodiment, a store, a factory, a construction site, a care site, etc. are assumed. For example, in a care site, in a system that monitors an accident such as a subject falling from a bed, a change in a set area is detected in response to positional deviation or arrangement change of a bed at an imaging target location, and area setting It is sufficient to notify occurrence of deviation. Thereby, in the monitoring target area of the care site, it is possible to determine the change in the area where the event should be detected, and it becomes possible to take measures such as resetting of the appropriate detection area in accordance with the current situation.

  In the camera system according to the present embodiment, the camera device 10 captures an image of a target area, performs area setting including setting of a detection area for performing detection of an object in a captured image of the target area, and detects a detection area in the captured image. When a predetermined detection event occurs in (4), event occurrence is determined and event detection information is transmitted to the server device 20. In addition, the camera device 10 determines the presence / absence of a change in the area setting at a predetermined determination timing, and transmits the area change detection information to the server device 20 when the change in the area setting occurs. The server device 20 acquires event detection information or area change detection information transmitted from the camera device 10, and notifies the monitor 42 connected to the server device 20 of occurrence of a detection event or change in area setting. Display

  Thereby, when a change occurs in the area setting such as the detection area and the mask area while the detection operation for detecting the occurrence of the predetermined detection event is operated based on the captured image of the camera device, this area setting By detecting and notifying the user of the change in area, it is possible to notify the user of the area setting deviation, to change the area setting according to the deviation amount, and the like. The user recognizes the change in the area setting, and manually resets the area such as the detection area according to the amount of deviation so as to be compatible with the current captured image, or the area setting draft generated by the camera device It is possible to update the area settings by

  Further, in the camera system of the present embodiment, the camera device 10 performs area division of the current captured image and compares the positions or areas of the respective areas with each area of the captured image divided in the previous area setting. When the difference is equal to or more than a predetermined value, it is determined that there is a change in the area setting, and area change detection information is transmitted to the server device 20.

  Thereby, in the state where the area setting using the area division of the captured image is performed and the detection operation is operated, the shift of the setting area caused by the change of the captured image due to the lens position shift of the camera device or the angle change When this occurs, it is possible to detect a change in the area setting such as the detection area and the mask area, and to notify the user of the area setting deviation.

  Further, in the camera system according to the present embodiment, the camera device 10 extracts feature points in the current captured image, and compares the positions of corresponding feature points with the feature points of the captured image extracted in the previous area setting. If the error of is greater than or equal to the threshold value, it is determined that the area setting has changed, and area change detection information is transmitted to the server device 20.

  Thereby, in the state where the area setting using the feature point detection in the captured image is performed and the detection operation is operated, the setting area of the camera apparatus due to the displacement of the lens of the camera device or the change of the captured image When a deviation occurs, it is possible to detect a change in area setting such as a detection area and a mask area, and to notify the user of the area setting deviation.

  Further, in the camera system of the present embodiment, the camera device 10 acquires detection history information including statistical information of the history detected in the captured image, and compares the previous detection history information with the current detection history information. When there is a change of a predetermined amount or more in the detection history, it is determined that the area setting has changed, and the area change detection information is transmitted to the server device 20.

  As a result, the detection history information in the case where the detection tendency of an object such as the number of detections, the residence time, and the heat map largely changes from before with a change in the captured image due to a change in the imaging target. The change of the area setting can be detected by the comparison of and the user can be notified of the area setting deviation.

  Further, in the camera system according to the present embodiment, the camera device 10 obtains an average image of a predetermined period related to a captured image, compares the average image with the current captured image, and changes the image features by a predetermined amount or more. In this case, it is determined that there is a change in the area setting, and the area change detection information is transmitted to the server device 20.

  Thereby, when the position of the feature point or the area division changes with the change of the captured image due to the change of the imaging target, and the shift of the setting area occurs, the change of the area setting is detected by comparison with the average image, It is possible to notify the user of the area setting deviation.

  Further, in the camera system of the present embodiment, when a change in the area setting occurs, the camera apparatus 10 performs the area setting based on the current captured image to update the area setting information, and the updated area setting information. Is transmitted to the server device 20, and the server device 20 acquires the area change detection information and the area setting information transmitted from the camera device 10, notifies the monitor 42 of the occurrence of the change in the area setting, and the area after the update. A notification screen including a region setting proposal based on setting information is displayed.

  Thereby, the user can recognize the occurrence of the change in the area setting, and change the area setting so as to be adapted to the current captured image, using the area setting plan updated in the camera device.

  In the camera device 10 of the present embodiment, the target area is imaged, and in the captured image of the target area, area setting including setting of a detection area for executing detection of the target is performed, and predetermined detection is performed in the detection area in the captured image. When an event occurs, event occurrence is determined and event detection information is generated, and at a predetermined determination timing, it is determined whether or not there is a change in the area setting, and the area change detection information is Based on the event detection information or the area change detection information, the monitor 42 connected to the camera device 10 displays a notification screen for notifying the occurrence of the detection event or the change of the area setting. As a result, the user can be notified of the occurrence of a detection event, and if a change occurs in the area setting, the occurrence of the area setting deviation can be notified, and it is possible to take measures such as changing the area setting according to the deviation amount. Become.

  Further, in the camera device 10 according to the present embodiment, when a change in the area setting occurs, the area setting is performed based on the current captured image to update the area setting information, and the area change detection information and the area setting after the update Based on the information, the monitor 42 displays a notification screen including a notification of the occurrence of a change in the area setting and a proposed area setting according to the updated area setting information. Thereby, the user can recognize the occurrence of the change in the area setting, and change the area setting so as to be adapted to the current captured image, using the area setting plan updated in the camera device.

  The present invention is defined as a detection area setting method including any of area setting for detecting a target object by the camera system or the camera apparatus according to the present embodiment, and operation (step) of detecting deviation of area setting. You may

  Although various embodiments have been described above with reference to the drawings, it goes without saying that the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the appended claims, and of course these also fall within the technical scope of the present invention. It is understood.

  The present invention is useful as a camera system and a camera apparatus capable of detecting an object with high accuracy by setting an appropriate area even when a change occurs in a captured image.

10, 10A, 10B, 10N, 10-1 Camera device 20, 210 Server device 30 Recorder 41, 221 Input device 42, 222 Monitor 70, 300 Network 100, 100A, 100B Field 101 Imaging unit 102 Image input unit 103, 103- Reference Signs List 1 feature extraction unit 104 area setting unit 105 area setting information storage unit 106 shift / change detection unit 107 transmission unit 108 reception unit 131 area division unit 132 feature point extraction unit 133 detection history feature extraction unit 161 shift determination unit 162 change extraction unit 200 Administration headquarters

Claims (12)

A camera system in which a camera device and a server device are mutually connected,
The camera device is
Capture the target area
In the captured image of the target area, area setting including setting of a detection area for executing detection of the target is performed,
When a predetermined detection event occurs in a detection area in the captured image, event occurrence is determined and event detection information is transmitted to the server device;
In certain determination timing, and determines the presence or absence of change in the area setting, you send area change detection information to the server device when the change in the area setting has occurred,
Camera system. The camera system according to claim 1,
The camera device is
If the difference between the position or area of each of the areas is greater than or equal to a predetermined value as compared with each area of the captured image divided in the previous area setting, the area of the current captured image is divided. It is determined that there is a change, and the area change detection information is transmitted to the server device.
Camera system. The camera system according to claim 1,
The camera device is
When the error of the position of the corresponding feature point is extracted with the threshold value or more by extracting the feature point in the current captured image and comparing with the feature point of the captured image extracted in the previous area setting, the change of the area setting It determines that there is, and transmits the area change detection information to the server device.
Camera system. The camera system according to claim 1,
The camera device is
The detection history information including statistical information of the history detected in the captured image is acquired, the previous detection history information is compared with the current detection history information, and the detection history changes by a predetermined amount or more. It is determined that there is a change in area setting, and the area change detection information is transmitted to the server device.
Camera system. The camera system according to claim 1,
The camera device is
An average image of a predetermined period related to the captured image is acquired, and the average image and the current captured image are compared, and if there is a change of a predetermined amount or more in the image feature, it is determined that the area setting is changed. Transmitting the area change detection information to the server device;
Camera system. The camera system according to any one of claims 1 to 5, wherein
The camera device is
If the change in the area setting has occurred, updated the region setting information by performing the area set based on the current captured image, you send area setting information after update to the server apparatus,
Camera system. The camera system according to any one of claims 1 to 6,
The server device is
A notification screen for acquiring the event detection information or the area change detection information transmitted from the camera device, and notifying a monitor connected to the server apparatus of the occurrence of the detection event or a change in the area setting indicate,
Camera system. The camera system according to any one of claims 1 to 6,
The server device is
The area change detection information and the area setting information transmitted from the camera device are acquired, and the monitor connected to the server apparatus is notified of the occurrence of the change in the area setting, and the area setting by the area setting information after the update Display a notification screen including a draft,
Camera system. A camera device for imaging a target area,
In the captured image of the target area, and the row Cormorant area setting unit area settings, including detection area for performing a detection of the object,
A first detection unit that determines event occurrence and generates event detection information when a predetermined detection event occurs in a detection area in the captured image;
In certain determination timing, and it determines the presence or absence of change in the area setting, and a second detection portion that forms the raw area change detection information when the change in the area setting has occurred,
Camera device. The camera apparatus according to claim 9 ,
The region setting unit, when the change in the area setting has occurred, new to Ru further the region setting information by performing the area set based on the current captured image,
Camera device. The camera apparatus according to claim 9,
The transmission unit further displays a notification screen for notifying the occurrence of the detection event or the change of the area setting on a monitor connected to the camera device based on the event detection information or the area change detection information. ,
Camera device. A camera apparatus according to claim 10, wherein
Based on the area change detection information and the updated area setting information, the monitor connected to the camera apparatus includes a notification of occurrence of the change in the area setting and an area setting plan based on the updated area setting information. The transmission unit further displaying a notification screen,
Camera device.

Images