JP7419095B2 - Image sensors, computing devices, and image sensor systems - Google Patents

Description

Embodiments of the invention relate to image sensors, computing devices, and image sensor systems.

Recent image sensors are equipped with a CPU (Central Processing Unit) and memory, and can be said to be a built-in computer with a lens. It also has advanced image processing functions, and can analyze captured image data to calculate, for example, the presence or absence of people or the number of people. This type of image sensor is being used in combination with building management systems to improve comfort, livability, and promote energy conservation.

Patent No. 4667508 specification Patent No. 5739722 specification

Existing surveillance cameras used in security systems and the like detect a person in an abnormal state (such as someone falling down) using a telephoto camera installed with a diagonal downward view. However, images taken at an angle of view looking down diagonally lack versatility and cannot be used for other purposes. It is also desired that an image sensor that obtains various sensing data by analyzing image data acquired under a direct-down viewing angle be capable of detecting, for example, a person in an abnormal state.

Therefore, an object of the present invention is to provide an image sensor, a calculation device, and an image sensor system that can accurately detect sensing data regardless of the angle of view.

According to the embodiment, the image sensor includes an imaging section, a storage section, a conversion section, and an image processing section. The imaging unit images the area and obtains image data. The storage unit stores dictionary data created in advance for the detection target. The converter converts the acquired image data or the feature amount extracted from the image data according to the dictionary data to create converted data. The image processing unit processes the conversion data using the dictionary data to generate sensing data in the area.

FIG. 1 is a schematic diagram showing an example of an image sensor system including an image sensor according to an embodiment. FIG. 2 is a diagram showing an example of the interior of a building floor. FIG. 3 is a block diagram showing an example of the image sensor system shown in FIG. FIG. 4 is a block diagram illustrating an example of an image sensor according to the embodiment. FIG. 5 is a flowchart showing the basic processing procedure of the image sensor 3. FIG. 6 is a diagram showing an example of the flow of data in the image sensor 3. FIG. 7 is a block diagram showing an example of the gateway device 7 according to the embodiment. FIG. 8 is a diagram showing an example of image data acquired by the image sensor 3. FIG. 9 is a diagram showing an example of a learning image obtained by panoramicizing the field of view of image data acquired by the image sensor 3. FIG. 10 is a diagram showing an example of a learning image obtained by rotating the image data acquired by the image sensor 3. FIG. 11 is a diagram showing an example of a learning image obtained by reducing the image data acquired by the image sensor 3. FIG. 12 is a diagram showing an example of a learning image obtained by enlarging the image data acquired by the image sensor 3. FIG. 13 is a diagram showing an example of a learning image obtained by correcting distortion of the field of view of the image sensor 3. FIG. 14 is a diagram illustrating an example of performing conversion processing on feature amounts. FIG. 15 is a diagram illustrating an example of learning images in different orientations included in the image data. FIG. 16 is a diagram showing an example of learning images of different sizes included in the image data. FIG. 17 is a diagram illustrating an example of the fact that learning data can be created for each different posture and orientation of a person. FIG. 18 is a diagram for explaining that learning data can be created for each region of divided image data. FIG. 19 is a diagram for explaining that learning data can be created for each region of a divided panoramic image. FIG. 20 is a diagram for explaining that the number of learning images can be increased by rotation. FIG. 21 is a diagram for explaining that the number of learning images can be increased by slightly shifting the positions.

Image sensors can acquire a variety of information compared to human sensors, light sensors, infrared sensors, and the like. By using a fisheye lens or an ultra-wide-angle lens, the area that can be captured by a single image sensor can be expanded, and image distortion can be corrected through calculation processing. Image sensors are also known that have a function to mask areas within the field of view that are not desired to be sensed, and a learning function.

[composition]
FIG. 1 is a schematic diagram showing an example of an image sensor system including an image sensor according to an embodiment. In FIG. 1, lighting equipment 1, lighting controller 4, and image sensors 3 (3-1 to 3-n) are provided, for example, on each floor of building 400, and are communicably connected to gateway device 7. The gateway device 7 on each floor is communicably connected to, for example, a BEMS (Building Energy Management System) server 5 of a building management center via an in-building network 8.

The BEMS server 5 can be connected to a cloud computing system (cloud) 100 via a communication network 10 based on TCP/IP (Transmission Control Protocol/Internet Protocol), for example. The cloud 100 includes a server 200 and a database 300, and provides services related to building management and the like.

As shown in FIG. 2, the lighting equipment 1, the air outlet of the air conditioner 2, and the image sensor 3 are arranged, for example, on the ceiling of each floor. Each image sensor 3 is assigned an area to be sensed. In FIG. 2, it is shown that areas A1 and A2 each cover approximately half of the floor. All areas combined can cover the entire floor of the target space. As shown by hatching in the figure, the allocated areas of different image sensors 3 may partially overlap (overlapping portion).

The image sensor 3 looks down on the assigned area, captures it within its field of view at an angle of view, photographs an image within its field of view, and acquires image data. This image data is processed by the image sensor 3, and human information, environmental information, etc. are calculated. That is, the image sensor 3 detects (senses) an object in the object space and calculates sensing data related to this object.

The environmental information is information regarding the environment of the target space (zone), such as the illuminance and temperature of the floor. The person information is information about people in the target space. For example, person information includes the number of people in an area, people's actions, amount of people's activity, presence or absence indicating a person's presence or absence, or walking retention indicating whether a person is walking or staying in one place. This is an example.

In recent years, studies have been conducted to ensure human comfort and safety by controlling lighting equipment and air conditioning equipment based on this information in human living environments. It is also possible to calculate the environmental information and person information for each small region obtained by dividing the zone into a plurality of regions. This small area may be associated with an allocated area for each image sensor 3.

FIG. 3 is a block diagram showing an example of the image sensor system shown in FIG. In FIG. 3, the gateway device 7 accommodates a plurality of image sensors 3 (3-1 to 3-n) as subordinates. The image sensors 3-1 to 3-n are connected in a single-stroke pattern by the sensor network 9 (crossover wiring). The sensor network 9 communicably connects the image sensors 3-1 to 3-n and the gateway device 7. As a protocol for the sensor network 9, for example, EtherCAT (registered trademark) is known.

On the other hand, the gateway device 7 is connected to the in-building network 8 . A management terminal 20, a BEMS server 5, and a lighting controller 4 are connected to the in-building network 8. Gateway device 7 converts the communication protocol between sensor network 9 and in-building network 8 . Thereby, the BEMS server 5 and the management terminal 20 can also communicate with the image sensors 3-1 to 3-n via the gateway device 7.

A typical communication protocol for the in-building network 8 is Building Automation and Control Networking protocol (BACnet (registered trademark)). Other protocols such as DALI, ZigBee (registered trademark), and ECHONET Lite (registered trademark) are also known.

FIG. 4 is a block diagram showing an example of the image sensor 3 according to the embodiment. The image sensor 3 includes an imaging section 31, a memory 32, a processor 33, and a communication section 34. These are connected to each other via an internal bus 35. Image sensor 3 including memory 32 and processor 33 functions as a computer.

The imaging unit 31 includes a fisheye lens 31a, an aperture mechanism 31b, an image sensor 31c, and a register 30. The fisheye lens 31a captures the area in its field of view looking down from the ceiling, and forms an image on the image sensor 31c. The amount of light from the fisheye lens 31a is adjusted by an aperture mechanism 31b. The image sensor 31c is, for example, a CMOS (complementary metal oxide semiconductor) sensor, and generates a video signal at a frame rate of, for example, 30 frames per second. This video signal is digitally encoded and output as image data. The image data is stored in the memory 32 (image data 32b).

The register 30 stores camera information 30a. The camera information 30a is information regarding the imaging unit 31, such as the state of the auto gain control function, gain value, and exposure time, or information regarding the image sensor 3 itself. For example, parameters that affect the sensitivity and accuracy of sensing, such as the frame rate, the gain value of the image sensor 31c, and the exposure time, are controlled by the processor 33.

The memory 32 is a semiconductor memory such as SDRAM (Synchronous Dynamic RAM), or a nonvolatile memory such as NAND flash memory or EPROM (Erasable Programmable ROM). The memory 32 stores a program 32a for operating the processor 33, image data 32b, conversion data 32c, and dictionary data 32d created in advance for data to be detected.

The dictionary data 32d is data used to generate sensing data from image data, and can be generated, for example, by a method using a machine-learning framework such as SVM (Support Vector Machine). It is. Dictionary data used by classifiers in machine learning are disclosed in numerous documents. For example, by pattern matching processing using the dictionary data 32d, it is possible to identify a detection target (person, etc.) in the area. Note that a plurality of dictionary data 32d may be used for one sensing item. Further, one dictionary data 32d may be used in common for a plurality of sensing items.

Furthermore, the memory 32 stores sensing data 32e and feature quantities 32f. In addition, mask setting data and the like may be stored in the memory 32. The mask setting data is used to distinguish between an area to be image-processed (an image-processing target area) and an area not to be image-processed (an image-processing non-target area) out of the field of view captured by the imaging unit 31.

The processor 33 loads and executes programs stored in the memory 32, thereby realizing various functions described in the embodiments. The processor 33 is, for example, an LSI (Large Scale Integration) that includes a multi-core CPU (Central Processing Unit) and is tuned to perform image processing at high speed. The processor 15 can also be configured with an MPU (Micro Processing Unit), an FPGA (Field Programmable Gate Array), or the like.

The communication unit 34 is connectable to the sensor network 9 and mediates data exchange with a communication partner (gateway device 7, management terminal 20, other image sensor 3, BEMS server 5, etc.). The communication interface may be wired or wireless. Any topology such as a star type or a ring type can be applied to the communication network. The communication protocol may be a general purpose protocol or an industrial protocol. A single communication method may be used, or a plurality of communication methods may be combined.

In particular, the communication unit 34 receives sensing data from the image sensor 3, processing results from the processor 33, processed data, parameters, image data, motion vector data, accumulated past motion vector data, processed motion vector data, and statistics. Analysis data, output data, parameters, dictionary data, firmware, etc. are transmitted and received via the sensor network 9 as a communication network. Thereby, the above data and information can be shared with the gateway device 7, other image sensors 3, BEMS server 5, management terminal 20, etc. via the in-building network 8, etc.

By the way, the processor 33 includes an image processing section 33a, a conversion section 33b, an extraction section 33c, and an integration processing section 33d as processing functions according to the embodiment. The image processing section 33a, the conversion section 33b, the extraction section 33c, and the integrated processing section 33d load the program 32a stored in the memory 32 into the register of the processor 33, and the processor 33 performs arithmetic processing as the program progresses. It can be understood as a process generated by execution.

Of these, the extraction unit 33c extracts the feature amount of the image data 32b from the image data 32b. The extracted feature amount is stored in the memory 32 (feature amount 32f).
The converter 33b converts the image data 32b according to the dictionary data 32d to create converted data 32c. Further, the converting unit 33b converts the feature amount 32f according to the dictionary data 32d to create converted data 32c. The created conversion data is stored in the memory 32 (conversion data 32c).

The image processing unit 33a processes the conversion data 32c by applying, for example, a machine learning classifier algorithm using the dictionary data 32d, and generates sensing data in the area assigned to its own sensor. The generated sensing data is stored in the memory 32 (sensing data 32e).

Note that the image processing unit 33a can also generate sensing data by processing the image data 32b. However, since the converted data 32c is generated to match the dictionary data 32d, it is more advantageous to generate sensing data using the converted data 32c.

In the embodiment, (person sensing data in a normal state), (environmental information), and (person sensing data in an abnormal state) are taken as sensing data.
(Normal state human sensing data) includes presence/absence, estimated number of people, walking stagnation, person recognition, motion vector, etc.
(Environmental information) includes estimated illuminance, amount of activity, degree of crowding, etc.
(Sensing data on people in abnormal conditions) includes people who have collapsed, elderly people, people in wheelchairs, etc.

The image processing unit 33a senses these items from the converted data 32d (or image data 32b). All items may be sensed simultaneously all the time, or only selected items may be sensed as needed. Items to be sensed can also be selected according to time, calendar, brightness, external instructions, etc. The processing cycle may be the same for all items, or may be processed at different cycles for each sensing item.

The integrated processing unit 33d refers to either or both of (environmental information) and (normal state human sensing data) when generating (human sensing data in an abnormal state). In other words, the integrated processing unit 33d, based on the result of integrated determination of (human sensing data in abnormal state) itself, at least one of (environmental information), and (human sensing data in normal state), ( Generate and output abnormal human sensing data).

For example, even if a collapsed person (abnormal human sensing data) is detected in a crowded environment (environmental information), it can be said that there is not that much urgency. On the other hand, if a person is detected in the absence of a person (person sensing data in an abnormal state), the situation is highly urgent and requires immediate notification to system personnel. The integrated processing unit 33d thus generates (person sensing data in an abnormal state) based on the result of integrated determination of a plurality of pieces of information.

FIG. 5 is a flowchart showing the basic processing procedure of the image processing section 33a shown in FIG. In FIG. 5, image data 32b acquired by the imaging section 31 (step S1) is temporarily stored in the memory 32 (step S2), and then sent to the image processing section 33a. The image processing unit 33a performs image processing on the image data 32b to detect (sensing) a person in a normal state (step S3), estimate environmental information (step S4), and detect a person in an abnormal state (step S5). Each process is executed to sense these items from the image data 32b.

All of these sensing items may be sensed simultaneously at all times. Alternatively, sensing may be performed individually as necessary. The processing cycle for each item may be the same for all items, for example in synchronization with the frame cycle, or may be different for each item. The communication unit 34 communicates these sensing data and various parameters related to calculation of items to the gateway device 7 via the sensor network 9 (step S6).

FIG. 6 is a diagram showing an example of the flow of data in the image sensor. In FIG. 6, image data 32b acquired by the imaging section 31 is temporarily stored in the memory 32, and then sent to the image processing section 33a of the processor 33. The image processing unit 33a performs image processing on the image data 32b using the dictionary data 32d stored in the memory 32, and generates sensing data. The communication unit 34 transmits various data including the generated sensing data to the gateway device 7 via the sensor network 9.

Based on the flow shown in FIG. 6, the embodiment will describe a technique for generating sensing data by performing image processing based on converted data 32c obtained by converting image data 32b according to dictionary data 32c.

FIG. 7 is a block diagram showing an example of the gateway device 7 according to the embodiment. The gateway device 7 as an example of a computing device is a computer that includes a processor 75 such as a CPU or an MPU (Micro Processing Unit), a ROM (Read Only Memory) 72, and a RAM (Random Access Memory) 73. The gateway device 7 further includes a storage section 74 such as a hard disk drive (HDD), an optical media drive 76, and a communication section 77.

The ROM 72 stores basic programs such as BIOS (Basic Input Output System) and UEFI (Unified Extensible Firmware Interface), various setting data, and the like. The RAM 73 temporarily stores programs and data loaded from the storage section 74.

The optical media drive 76 reads digital data recorded on a recording medium such as a CD-ROM 78. Various programs executed by the gateway device 7 are recorded on, for example, a CD-ROM 78 and distributed. The program stored in this CD-ROM 78 is read by the optical media drive 76 and installed in the storage section 74.

The communication unit 77 has a function for communicating with the image sensor 3, the BEMS server 5, etc. via the in-building network 8. Various programs to be executed by the gateway device 7 can also be downloaded from a server via the communication unit 77 and installed in the storage unit 74, for example. It is also possible to download the latest program from the cloud 100 (FIG. 1) via the communication unit 77 and update the installed program. Further, the gateway device 7 can download and transmit dictionary data to the image sensor 3 via the sensor network 9.

The processor 75 executes an OS (Operating System) and various programs. Furthermore, the processor 75 includes an acquisition section 75a, a creation section 75b, and a transmission section 75c as processing functions according to the embodiment.

These functional blocks can be understood as processes that are generated in the process of loading the program 74a stored in the storage unit 74 into the RAM 73 and executing the program. That is, the program 74a causes the gateway device 7, which is a computer, to operate as an acquisition section 75a, a creation section 75b, and a transmission section 75c.

The acquisition unit 75a acquires image data generated by each of the image sensors 3-1 to 3-n. The acquired image data is stored in the storage unit 74 (image data 74b). Further, the acquisition unit 75a acquires the feature amounts extracted from the image data in the image sensors 3-1 to 3-n from the image sensors 3-1 to 3-n. The acquired feature amount is stored in the storage unit 74 (feature amount 74c).

The creation unit 75b creates dictionary data used by the image sensors 3-1 to 3-n based on the image data 74b. The created dictionary data is stored in the storage unit 74 (dictionary data 74d).

The transmitter 75c transmits the created dictionary data 74d to the destination image sensors 3-1 to 3-n. The transmitted dictionary data is stored in each of the image sensors 3-1 to 3-n and used to generate sensing data.

In addition to the program 74a executed by the processor 75, the storage unit 74 stores image data 74b, feature amounts 74c, and dictionary data 74d.

The image data 74b is image data acquired from the image sensors 3-1 to 3-n and 3-1 to 3-m. The feature quantity 74c is information extracted from the image data 74b and indicating the characteristics of this image data 74b. Features include histograms of oriented gradients (HOG), contrast, resolution, S/N ratio, and color tone. A brightness gradient direction co-occurrence histogram (Co-HOG) feature, a Haar-Like feature, and the like are also known as features.

[Effect]
Next, the operation of the above configuration will be explained.
FIG. 8 is a diagram showing an example of image data acquired by the image sensor 3. It is assumed that this image data includes, for example, a person lying on the floor (the area surrounded by a frame). The use of this learning image as learning data for the classifier of the image processing unit 33a will be described in detail below.

The conversion unit 33b of the image sensor 3 processes the image data acquired by itself into a form suitable for using the dictionary data 32d.
As shown in FIG. 9, for example, by panoramicizing the field of view of the image data acquired by the image sensor 3, the orientation of the images can be aligned.

As shown in FIG. 10, the orientation of the images can also be aligned by rotating the image data acquired by the image sensor 3.
As shown in FIG. 11, data suitable for image processing can also be obtained by reducing the image data acquired by the image sensor 3.
As shown in FIG. 12, data suitable for image processing can also be obtained by enlarging the image data acquired by the image sensor 3. This type of scaling process is also known as a pyramid image.
As shown in FIG. 13, data suitable for image processing can also be obtained by correcting the distortion of the field of view of the image sensor 3.
FIG. 14 is a diagram illustrating an example of performing conversion processing on feature amounts. For example, performing rotation processing on CoHOG, which is one of the feature amounts, is the same as sequentially replacing bins in the tilt direction.

The converted data 32c obtained by converting the image data 32b can also be used as learning data (learning data) for generating and updating the dictionary data 32d.
As shown in FIG. 15, learning images with different orientations within the image data can be individually employed as learning data. That is, dictionary data can be created in advance by learning for each different condition. The image processing unit 33a processes the conversion data 32c (image data 32b) using dictionary data 32d for each different condition to generate sensing data 32e.

As shown in FIG. 16, learning images of different sizes within the image data can also be individually employed as learning data.
As shown in FIG. 17, learning data can also be created for each posture or orientation of a person. Dictionary data is created using image data of multiple postures as learning data across multiple categories such as (a) supine, (b) prone, (c) right facing, (d) left facing, and (e) free posture. be able to.

As shown in FIG. 18(a), image data can be divided into concentric areas, and learning data can be created for each image in each area. Alternatively, as shown in FIG. 18(b), learning data can be created for each area divided radially from the center. Furthermore, as shown in FIG. 19(a), a panoramic image can be divided into multiple regions based on distance, or as shown in FIG. 19(b), a panoramic image can be divided into multiple regions based on angle. You can also do it.

Furthermore, it is also possible to convert the image data 32b by the converter 33b and create a larger number of converted data than the acquired image data.
As shown in FIG. 20, one learning image can be rotated to different angles (affine transformation) to pad the data. The larger the amount of learning data, the more reliable the dictionary data generated through learning becomes, and the more accurate sensing can be performed.
As shown in FIG. 21, learning images at peripheral positions (1) to (8) can also be generated by shifting the learning image at the central teaching position little by little. Note that the learning data may be padded using an image that has been subjected to image processing such as blurring or noise load. Furthermore, the learning image can be padded using image data using CG (Computer Graphics) technology instead of the actual image.

As described above, the converter 33b of the image sensor 3 converts the acquired image data 32b according to the dictionary data 32d to create converted data 32c. Then, based on the converted data 32c, the image processing unit 33a creates sensing data 32e by identification using, for example, SVM.

[effect]
As explained above, in this embodiment, the image sensors 3-1 to 3-n convert the acquired image data into a format suitable for their own dictionary data 32d, perform image processing on the converted data, and convert the sensing data into I tried to get it.

By doing this, it is possible to normalize raw image data and perform image processing based on the normalized data. That is, the image data itself or the feature amount is converted (preprocessed) in accordance with the dictionary data used for each of the image sensors 3-1 to 3-n. By doing so, normalized image data can be handled regardless of the mounting position or viewing angle of the image sensor, and useful learning data can be accumulated in any environment.

For these reasons, according to the embodiment, it is possible to provide an image sensor, a calculation device, and an image sensor system that can accurately detect sensing data regardless of the angle of view. As a result, it becomes possible to accurately detect a person in an abnormal state regardless of the angle of view.

Note that this invention is not limited to the above embodiments. For example, various functions implemented in the gateway device 7 as a computing device may be implemented in the management terminal 20 or the BEMS server 5.

Alternatively, some of the functions of the image sensor 3 may be implemented in the gateway device 7. For example, a converter is installed in the gateway device 7, and the converter converts image data acquired from an image sensor to create converted data. Then, the creation unit 75b creates dictionary data for each image sensor 3 based on this converted data. Furthermore, the created dictionary data may be transmitted to the image sensor 3 by the transmitter 75d.

Alternatively, the image data obtained by each image sensor 3 may be collected/accumulated in the gateway device 7, and the gateway device 7 may generate dictionary data through learning. Alternatively, the feature quantities extracted by the image sensors 3-1 to 3-n are collected/stored in the gateway device 7, and the feature quantities are converted into a form suitable for the dictionary data used by the image sensors 3-1 to 3-n. , and then dictionary data may be generated by learning.

Although an embodiment has been described, this embodiment is presented as an example and is not intended to limit the scope of the invention. This novel embodiment can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. This embodiment and its modifications are included within the scope and gist of the invention, as well as within the scope of the invention described in the claims and its equivalents.
Below, the invention described in the original claims of the present application will be added.
[1]
an imaging unit that images an area and obtains image data;
a storage unit that stores dictionary data created in advance regarding the detection target;
a conversion unit that converts the acquired image data or the feature amount extracted from the image data according to the dictionary data to create converted data;
an image processing unit that processes the conversion data using the dictionary data to generate sensing data in the area.
[2]
The imaging unit includes:
a fisheye lens that looks down on the area and captures it within the field of view;
The image sensor according to [1], further comprising an image sensor that captures an image formed from the fisheye lens and generates the image data.
[3]
The storage unit stores a plurality of dictionary data created in advance for each different condition,
The conversion unit converts the acquired image data according to each of the plurality of dictionary data to create converted data,
The image sensor according to [1], wherein the image processing unit processes the conversion data using dictionary data for each of the different conditions to generate the sensing data.
[4]
The image sensor according to [1], wherein the conversion unit converts the acquired image data to create a larger number of converted data than the acquired image data.
[5]
The image sensor according to [1], further comprising a creation unit that creates the dictionary data by learning based on the conversion data.
[6]
The sensing data is
The image sensor according to [1], including sensing data of a person in an abnormal state in the area.
[7]
The sensing data is
The image sensor according to [6], further including at least one of sensing data of a person in a normal state in the area and environmental information of the area.
[8]
Based on the result of integrated determination of the sensing data of the person in the abnormal state, the sensing data of the person in the normal state in the area, and at least one of the environmental information of the area, The image sensor according to [7], further comprising an integrated processing unit that generates sensing data.
[9]
an acquisition unit that acquires the image data from an image sensor that images an area and acquires the image data;
a conversion unit that converts the acquired image data or the feature amount extracted from the image data according to dictionary data used by the image sensor to create converted data;
a creation unit that creates dictionary data to be used in the image sensor based on the conversion data;
A computing device comprising: a transmitting unit that transmits the created dictionary data to the image sensor.
[10]
multiple image sensors that sense the target;
comprising a computing device capable of communicating with the image sensor,
The image sensor includes:
an imaging unit that images an area and obtains image data;
a storage unit that stores dictionary data created in advance regarding the detection target;
a conversion unit that converts the acquired image data or the feature amount extracted from the image data according to the dictionary data to create converted data;
an image processing unit that processes the conversion data using the dictionary data to generate sensing data in the area,
The computing device includes:
an acquisition unit that acquires the image data from the image sensor;
a creation unit that creates dictionary data to be used in the image sensor based on the image data;
An image sensor system comprising: a transmitting section that transmits the created dictionary data to the image sensor.
[11]
The imaging unit includes:
a fisheye lens that looks down on the area and captures it within the field of view;
The image sensor system according to [10], further comprising an image sensor that captures an image formed from the fisheye lens and generates the image data.
[12]
The storage unit stores a plurality of dictionary data created in advance for each different condition,
The conversion unit converts the acquired image data according to each of the plurality of dictionary data to create converted data,
The image sensor system according to [10], wherein the image processing unit processes the conversion data using dictionary data for each of the different conditions to generate the sensing data.
[13]
The image sensor system according to [10], wherein the conversion unit converts the acquired image data to create a larger number of converted data than the acquired image data.
[14]
The image sensor system according to [10], wherein the creation unit creates dictionary data used by the image sensor by learning based on the conversion data.

1... Lighting equipment, 2... Air conditioning equipment, 3 (3-1 to 3-n)... Image sensor, 4... Lighting controller, 5... BEMS server, 7... Gateway device, 8... In-building network, 9... Sensor network, DESCRIPTION OF SYMBOLS 10... Communication network, 15... Processor, 20... Management terminal, 30... Register, 30a... Camera information, 31... Imaging unit, 31a... Fisheye lens, 31b... Aperture mechanism, 31c... Image sensor, 32... Memory, 32a... Program, 32b...Image data, 32c...Conversion data, 32d...Dictionary data, 32e...Sensing data, 32f...Feature amount, 33...Processor, 33a...Image processing section, 33b...Conversion section, 33c...Extraction section, 33d...Integration processing section , 34...Communication unit, 35...Internal bus, 72...ROM, 73...RAM, 74...Storage unit, 74a...Program, 74b...Image data, 74c...Feature amount, 74d...Dictionary data, 75...Processor, 75a...Acquisition Department, 75b... Creation Department, 75c... Transmission Department, 76... Optical Media Drive, 77... Communication Department, 100... Cloud, 200... Server, 300... Database, 400... Building, A1, A2... Area.

Claims (12)

an imaging unit that images an area and obtains image data;
a conversion unit that performs rotation processing on the CoHOG feature amount of the acquired image data to normalize it and create converted data;
a creation unit that creates dictionary data by machine learning based on the converted data;
an image processing unit that processes the conversion data using the dictionary data to generate sensing data in the area. The imaging unit includes:
a fisheye lens that looks down on the area and captures it within the field of view;
The image sensor according to claim 1, further comprising an image sensor that captures an image formed from the fisheye lens and generates the image data. The conversion unit further includes a storage unit that stores a plurality of dictionary data created in advance for each condition that the orientation is different within the image data or for each condition that the size is different within the image data, and the conversion unit is configured to perform the acquisition. converting the image data according to each of the plurality of dictionary data to create converted data;
The image sensor according to claim 1, wherein the image processing unit processes the conversion data using dictionary data for each of the different conditions to generate the sensing data. The image sensor according to claim 1, wherein the converter converts the acquired image data to create a larger number of converted data than the acquired image data. The sensing data is
The image sensor according to claim 1, comprising sensing data of a person in an abnormal state in the area. The sensing data is
The image sensor according to claim 5, further comprising at least one of sensing data of a person in a normal state in the area and environmental information of the area. Based on the result of integrated determination of the sensing data of the person in the abnormal state, the sensing data of the person in the normal state in the area, and at least one of the environmental information of the area, The image sensor according to claim 6, further comprising an integrated processing unit that generates sensing data. an acquisition unit that acquires the image data from an image sensor that images an area and acquires the image data;
a conversion unit that performs rotation processing on the CoHOG feature amount of the acquired image data to normalize it and create converted data;
a creation unit that creates dictionary data by machine learning based on the converted data;
A computing device comprising: a transmitting unit that transmits the created dictionary data to the image sensor. multiple image sensors that sense the target;
comprising a computing device capable of communicating with the image sensor,
The image sensor includes:
an imaging unit that images an area and obtains image data;
a conversion unit that performs rotation processing on the CoHOG feature amount of the acquired image data to normalize it and create converted data;
a storage unit that stores dictionary data created in advance regarding the detection target;
an image processing unit that processes the conversion data using the dictionary data to generate sensing data in the area,
The computing device includes:
an acquisition unit that acquires the conversion data from the image sensor;
a creation unit that creates dictionary data by machine learning based on the converted data;
An image sensor system comprising: a transmitting section that transmits the created dictionary data to the image sensor. The imaging unit includes:
a fisheye lens that looks down on the area and captures it within the field of view;
The image sensor system according to claim 9, further comprising an image sensor that captures an image formed from the fisheye lens and generates the image data. The storage unit stores a plurality of dictionary data created in advance for each condition that the orientation is different within the image data or for each condition that the size is different within the image data,
The conversion unit converts the acquired image data according to each of the plurality of dictionary data to create converted data,
The image sensor system according to claim 9, wherein the image processing unit processes the conversion data using dictionary data for each of the different conditions to generate the sensing data. The image sensor system according to claim 9, wherein the conversion unit converts the acquired image data to create a larger number of converted data than the acquired image data.