JP6982875B2 - Information provision system - Google Patents

Description

The present invention relates to an information providing system.

Patent Document 1 proposes a system in which information is added to an image based on image analysis without increasing the processing load of the server, and a reward is given according to the processing.

Japanese Unexamined Patent Publication No. 2003-16187

Similar to Patent Document 1, there is known a service of receiving image data or video data from a camera or the like installed by a user and giving a considerable reward to the user. Such services are, for example, for crime prevention purposes, environmental surveys, and the like.

Specifically, there are those that constantly send images (videos) from the camera installed by the user and give a reward for each transmission time, and those that give a reward for each number of images acquired by the camera.

However, when the image (raw data) acquired by the camera is transmitted to the service provider in real time, there is a risk of disclosure of leakage risk from the viewpoint of personal information and security.

Therefore, it is an object of the present invention to provide a technique for effectively utilizing images obtained from users while reducing privacy and security risks in particular.

According to the present invention, it is an information providing system including a management server and a plurality of information acquisition devices connected to the management server via a network.
The information acquisition device is:
Receives mission information including judgment condition information and processing processing information;
The first external world information is acquired based on the determination condition information, and it is determined whether or not the first external world information satisfies the determination condition;
When the first outside world information satisfies the determination condition, the second outside world information is acquired, and the second outside world information is processed based on the processing processing information to generate processing information;
Sending the processing information to the management server,
An information provision system can be obtained.

According to the present invention, by providing a determination unit and a processing unit for detecting and processing an object contained in external information on the information acquisition device side, it is not necessary to transmit so-called raw data to a management server, and security and personal information can be obtained. Information leakage and unnecessary disclosure can be prevented.

In other words, by using the present invention, the user can provide only necessary information to the management server while constantly activating the acquisition unit such as a camera.

Further, according to the present invention, the user can obtain a plurality of rewards by receiving a plurality of mission information as long as he / she has his / her own terminal, and can encourage active participation in the system. ..

Further, according to the present invention, since the user can provide the management server with information within the range that he / she can tolerate, he / she can control the security by his / her own will.

It is a block diagram of the information provision system by embodiment of this invention. It is a figure which shows the functional block diagram of the management server of FIG. It is a figure which shows the functional block diagram of the information acquisition apparatus of FIG. It is a figure explaining the flow of information processing on the information acquisition apparatus side by this invention. It is a figure explaining the flow of information processing on the management server side by this invention. It is a figure which shows the area acquired by a camera. This is a display example displayed on the user's HMD. It is a figure explaining the kind of information transmitted from an information acquisition apparatus to a management server.

The contents of the embodiments of the present invention will be described in a list. The information providing system according to the embodiment of the present invention has the following configurations.
[Item 1]
An information providing system including a management server and a plurality of information acquisition devices connected to the management server via a network.
The information acquisition device is:
Receive mission information including judgment condition information;
The first external world information is acquired based on the determination condition information, and it is determined whether or not the first external world information satisfies the determination condition;
Judgment result information based on the above judgment is generated;
The determination result information is transmitted to the management server.
Information provision system.
[Item 2]
The information provision system described in item 1.
The information acquisition device transmits the determination result information to the management server according to a predetermined transmission condition.
Information provision system.
[Item 3]
The information providing system according to claim 2.
The information acquisition device transmits the determination result information to the management server at predetermined time intervals.
Information provision system.
[Item 4]
The information providing system according to any one of claims 2 to 4.
The information acquisition device transmits the determination result information to the management server according to the transmission instruction.
Information provision system.
[Item 5]
An information providing system including a management server and a plurality of information acquisition devices connected to the management server via a network.
The information acquisition device is:
Judgment condition information and processing processing information are generated with reference to the mission information;
The first external world information is acquired based on the determination condition information, and it is determined whether or not the first external world information satisfies the determination condition;
When the first outside world information satisfies the determination condition, the second outside world information is acquired, and the second outside world information is processed based on the processing processing information to generate processing information;
Sending the processing information to the management server,
Information provision system.

(First Embodiment)
<Structure>
FIG. 1 is a schematic diagram showing elements constituting an information providing system according to the present embodiment. The structure of the figure is an example, and elements other than these may be included.

The information providing system according to the present invention has a management server 1, a plurality of users P1 (information acquisition device 2: not shown), and a plurality of companies (corporate terminal: not shown). These are connected to each other via a network so as to be able to communicate with each other starting from the management server 1.

In the information providing system according to the present embodiment, the information acquisition device 2 receives the mission information transmitted from the corporate terminal via the management server 1, and the acquisition unit (GPS, camera, microphone, etc.) of the information acquisition device 2 is input. From the information obtained in the section), only the information satisfying a predetermined condition is cut out (or processed) and transmitted to the management server 1 to obtain a reward.

<Hardware configuration>
The hardware configuration of the information providing system will be described with reference to FIGS. 2 and 3. The management server 1 and the information acquisition device 2 according to the present embodiment have the following hardware configurations. The following configuration is an example, and may have other configurations. It is also possible to appropriately adopt other means having the same function.

<Management server 1>
The management server 1 constitutes a part of the information providing system by executing information processing via communication between the information acquisition device 2 and the corporate terminal. The management server 1 has a function related to processing of the entire information providing system.

FIG. 2 is a diagram showing an example of a functional block diagram of the management server 1. The configuration shown is an example, and functions other than these may be added.

The management server 1 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

The management server 1 includes at least a processor 10, a memory 11, a storage 12, a transmission / reception unit 13, an input / output unit 14, and the like, and these are electrically connected to each other through a bus 15.

The processor 10 is an arithmetic unit that controls the operation of the entire management server 1, controls the transmission and reception of data between each element, and performs information processing necessary for executing an application. For example, the processor 10 is a CPU (Central Processing Unit), and executes each information processing by executing a program or the like stored in the storage 12 and expanded in the memory 11.

The memory 11 includes a main storage configured by a volatile storage device such as a DRAM (Dynamic Random Access Memory) and an auxiliary storage configured by a non-volatile storage device such as a flash memory or an HDD (Hard Disk Drive). .. The memory 11 is used as a work area or the like of the processor 10, and also stores a BIOS (Basic Input / Output System) executed when the management server 1 is started, various setting information, and the like.

The storage 12 stores various programs such as application programs. A database storing data used for each process may be built in the storage 22.

The transmission / reception unit 13 connects the management server 1 to the network. The transmission / reception unit 13 may be provided with a short-range communication interface of Bluetooth (registered trademark) and BLE (Bluetooth Low Energy). The transmission / reception unit receives a request from the corporate terminal and transmits the generated mission information to the information acquisition device.

The input / output unit 14 is an information input device such as a keyboard and a mouse, and an output device such as a display.

The bus 15 is commonly connected to each of the above elements and transmits, for example, an address signal, a data signal, and various control signals.

<Information acquisition device 2>

The information acquisition device 2 constitutes a part of the information providing system by executing information processing via communication with the management server 1.

FIG. 3 is a diagram showing an example of a functional block diagram of the information acquisition device 2. The configuration shown is an example, and functions other than these may be added.

The information acquisition device 2 is information on the outside world that can be acquired by some kind of sensor device, such as video, image, sound, and temperature. Examples of the purpose of acquiring images and images include portable communication devices such as smartphones and tablet terminals with cameras, AR goggles and the like, and simple camera devices and the like. Further, although the above is mentioned, it may be a general-purpose computer such as a personal computer.

As shown in the figure, the information acquisition device 2 includes at least a processor 20, a memory 21, a storage 22, a transmission / output unit 23, an input / output unit 24, and the like, which are electrically connected to each other through a bus 25. Since these functions are the same as those of the management server 1 described above, detailed description thereof will be omitted here.

The information acquisition device 2 according to the present embodiment is a camera installed at a predetermined place like a so-called surveillance camera.

The above-mentioned input / output unit 24 includes at least a camera unit 242 as an input unit.

The camera unit may be a normal fixed camera, or may have a variable viewing direction by rotating the camera unit. Further, it may be capable of moving its own position.

As shown in FIG. 4, the information acquisition device 2 according to the present embodiment has a determination unit, a processing unit, and an acquisition unit in addition to the reception unit and the transmission unit as functional blocks. These functions will be described later.
<Data>
As shown in FIG. 4, the mission information according to the present embodiment has at least determination condition information, and may also include processing processing information and reward information. The entire mission information, the determination condition information, the processing processing information, or at least one of the reward information may be generated by the management server 1 or may be partially or wholly generated on the information acquisition device side. ..

The judgment condition information is information acquired by the acquisition unit of the information acquisition device 2 (position information, image / video by the camera, time, temperature, pressure, operation status, and other sensors provided by the information acquisition device, directly or indirectly. Information (functional package) necessary for determining (recognizing) whether or not the acquired information) satisfies a predetermined determination condition is included. The information to be acquired may be a plurality or a combination thereof.

The processing processing information is information (functional package) for processing the information obtained from the acquisition unit when the above-mentioned determination conditions are satisfied.

The determination condition information and processing processing information described above include, for example, a logic for extracting an object from an image (video) acquired by a camera unit or the like. The logic can employ various techniques necessary for extracting an object, such as an image recognition model and a voice recognition model (when the target is voice).

For example, a fixed camera installed on the side of the road may be used to "extract specific information from the information acquired by the camera. In this case, the presence or absence of a vehicle running on the road and the type of vehicle are used. , License plate, condition, etc. may be vehicle information.

For example, in the case of mission information to the effect of "information on the location of empty cans that have fallen in the city", the judgment condition information is to have the camera unit (acquisition unit) constantly acquire images and determine whether or not there are empty cans in the images. It is information for making a determination (recognition), and the processing processing information is instruction information for acquiring the position information of the information acquisition device when it is determined that the empty can has fallen.

Also, in the case of mission information to the effect of "information on the license plate of a vehicle abandoned in the city", the judgment condition information is determined by having the camera unit (acquisition unit) constantly acquire an image and whether or not there is an abandoned vehicle in the image. The processing information is information for cutting out the license plate from the abandoned vehicle and information for recognizing the characters of the license plate and extracting it as text information.

In the present embodiment, it may be determined whether or not the user is in a specific state by using only the determination condition information, and the result of the determination may be transmitted to the server as the determination result information. That is, the information itself or a part thereof acquired by the information acquisition device may be transmitted to the management server without using the processing information.

<Processing flow>
Subsequently, with reference to FIGS. 1, 4 and 5, the flow of processing of the information providing system according to the present embodiment will be described. As shown in FIG. 1, the management server 1 according to the present embodiment is connected to a plurality of information acquisition devices 2 and 2'. Note that FIG. 1 is a schematic diagram, and the management server 1 does not have to be one, and each function may be distributed to a plurality of terminals via the cloud.

In the present embodiment, an information recipient (such as a company that desires the information) (not shown) makes a request to the management server 1. Requests include, for example, "I want to find out the traffic volume on a certain road", "I want to know the traffic volume and attributes of passersby in a certain area", "I want to know information on empty cans left in the city", and "I want to collect information on illegal parking". "," I want to know the purchased product of the user who entered the convenience store ", etc., for example, it relates to the visual information of the user obtained mainly by using the camera unit.

At this time, the company may set a reward for the information that it wants to know or collect. For example, "If you find one empty can, it costs 0.1 yen", "If you find one abandoned vehicle, it costs 0.7 yen", "If you provide merchandise information purchased at a convenience store, it costs 8 yen". The method of setting the reward may be set for each number / number of units, set according to a group number, or set according to the time searched for.

The management server 1 transmits the request as the above-mentioned mission information to the information acquisition device, and causes the information acquisition device to perform a predetermined process. The mission information includes, for example, items such as "empty can / 0.1 yen", "abandoned vehicle / 0.7 yen", and "field of view information in convenience store / 8 yen".

When the HMDs of the users P1 to Pn are considered as the information acquisition devices, the camera unit mounted on each HMD can acquire the information of the field of view V1 to the field of view Vn.

In this case, the user who has received the mission information can select whether or not to participate in the mission specified by the mission information. When a user participates in a mission, participation information is transmitted to the management server 1, and the identification information of the mission and the identification information of the user (information acquisition device) are associated and managed by participating.

<Processing on the information acquisition device 2 side>
More specifically, as shown in FIG. 4, the receiving unit of the information acquisition device 2 receives the mission information transmitted from the management server 1. At this time, of the mission information, only the determination condition information or the processing processing information may be received as the mission information.

The mission information is input to the control unit. Hereinafter, each process performed when the mission information regarding the above-mentioned "traffic volume and vehicle attribute of a certain road" and "traffic volume and passer attribute of a certain area" is received will be described.

When these mission information is received, the judgment unit activates the function of the camera unit based on the judgment condition information to constantly acquire the image, and uses image recognition technology to determine whether or not there is a car or a passerby in the image. judge. At this time, information on the vehicle type of the vehicle and information on the attributes of passersby may be further acquired as determination result information. The cameras may be common cameras or may require different ones. The processing unit may acquire position information from GPS based on the determination result. The position information is output to the transmission unit as processing information and transmitted to the management server 1.

In addition to the above-mentioned functions, the processing unit also trims only the number of the car in the acquired image, applies a mosaic to the part other than the number, and extracts only the number of empty cans from the image and numerical values. It may be converted into data (other format) or processed to extract the brand of the empty can. Further, it may be transmitted together with other information obtained by the information acquisition device such as time information.

If necessary, the raw data acquired by the acquisition unit may be output to the transmission unit without any processing.

Subsequently, when "information on the license plate of a vehicle left in the city" is received, the judgment unit activates the function of the camera unit based on the judgment condition information to constantly acquire the image, and also in the image. Whether or not there is a stopped vehicle is determined by image recognition technology. The processing part extracts only the part of the license plate from the image. The extracted information is output to the transmission unit and transmitted to the management server 1 as processing information.

In addition to the above-mentioned functions, the processing unit also trims only the license plate part of the image, applies a mosaic to the part other than the license plate, and extracts only the license plate text from the image and texts. It may be converted into data or information on the vehicle type of the vehicle may be extracted. Further, in addition to these information, position information may be acquired from GPS.

The image recognition model in the present embodiment is input to the image to be processed. Next, the area to be recognized is extracted from the input image. For example, when the object to be recognized is an object (person) related to the object information, the area determined to be the object is extracted from the supplied image. Multiple areas may be extracted from one image.

Then, the feature amount is extracted from the extracted area (image of the area). The content of the extracted features is specified by the recognition process.

As the recognition process (image recognition model), for example, a method suitable for the recognition target such as HMM (Hidden Markov Model) or SVM (Support Vector Machine) is used. Then, the feature amount suitable for the method to be used is extracted, and the parameter is held.

<Processing on the management server side 1>
As shown in FIG. 5, the data transmitted from the information acquisition device 2 is received by the receiving unit of the management server 1. The received data is input to the control unit and the reward is calculated in the reward calculation unit.

The reward is calculated according to the evaluation conditions. Evaluation conditions are carried out from various viewpoints. For example, if the information transmitted from the information acquisition device is not the information to be provided by the mission information, it will be 0 (that is, if a processing error or the like has occurred on the information acquisition device side, or an unauthorized operation has been performed. If it was, etc.). Further, even if the information satisfies the target information, if there is another user who has already transmitted the same information, the reward is low because the information is not new as data.

In this way, the evaluation conditions are defined from the viewpoint of eligibility as target information and usefulness of data. The viewpoint of evaluation may be other than this.

The reward calculation unit corrects and determines the reward to be given to the user based on the above evaluation conditions. The compensation can be adjusted by multiplying by a coefficient according to the standard amount or by increasing or decreasing the constant, but the method is not limited to this.

The fixed reward is notified to the information acquisition device 2.

<Example>
FIG. 6 shows the outside world information that can be acquired by the camera beside the road. For example, the vehicle type (RV / sedan, etc.) and vehicle name of the vehicle in the video may be superimposed on the actual video (as shown in the figure) as the determination result information and transmitted to the management server side.

In FIG. 7, a vehicle stopped on the road is recognized as an abandoned vehicle, and a cursor is displayed on the license plate of the vehicle. In this case, the information of the license plate is transmitted to the management server 1.

Similarly, in FIG. 7, an empty can discarded on the road is recognized, and a cursor is displayed on the empty can. In this case, the information that the empty can was on the road is transmitted to the management server 1.

The above-mentioned raw data, determination result information, and processing information may be transmitted sequentially, or may be performed after a certain amount of data has been collected.

As described above, the management server 1 grants the set reward to the user (information acquisition device) according to the analysis result transmitted from the information acquisition device.

FIG. 7 is an image diagram illustrating the types of information transmitted from the information acquisition device 2 to the management server 1. As described above, not all the information acquired by the camera unit is transmitted to the management server 1. In other words, only the information related to the detection target included in the mission information is extracted and transmitted to the management server 1 from the outside world information.

For example, the following three patterns can be considered for the information transmitted from the information acquisition device to the management server 1. In the case of an empty can, only the "presence / absence" information of the detection target is transmitted to the management server 1 (1). Further, in the case of an abandoned vehicle, when the image data of only the license plate part is transmitted to the management server 1 (2), the character information is further analyzed from the license plate (that is, the information of (2) is processed). ) It may be sent to the management server 1 (3). Further, information obtained from another sensor group may be added and transmitted (4).

As described above, in the present embodiment, of the information acquired by the camera unit of the information acquisition device, only the portion necessary for the mission information is transmitted to the extracted management server 1, so that the data is transmitted. In addition to weight reduction, the following effects can be obtained.

That is, according to the present embodiment, since it is not necessary to transmit so-called raw data to the management server, it is possible to prevent security, leakage of personal information, and unnecessary disclosure.

Further, by using the present invention, the user can provide only necessary information to the management server while constantly activating the camera unit. That is, it is possible to save the trouble of activating the camera unit only when the detection target is photographed by the camera unit (when an abandoned vehicle is found, an empty can is found, etc.).

In addition, by receiving multiple mission information, it is possible to obtain multiple rewards, and it is possible to encourage active participation in this system. As a result, it is possible to improve the efficiency of providing information to the company that requested the mission information.

The reward information may be changed according to the amount and content of the information received from the information acquisition device. Further, the user side may control the amount and content of the information transmitted to the management server 1.

In the present embodiment described above, the information acquisition device 2 may transmit the determination result information and the processing information to the management server 1 or may transmit the processing information at predetermined time intervals according to predetermined transmission conditions. It may be transmitted according to the transmission instruction.

The embodiments described above are merely examples for facilitating the understanding of the present invention, and are not intended to limit the interpretation of the present invention. It goes without saying that the present invention can be modified and improved without departing from the spirit thereof, and the present invention includes its equivalents.

As shown in FIG. 7, the present invention can be applied not only to an HMD but also to, for example, an unmanned aerial vehicle equipped with a camera unit (drone or the like), an automobile having an automatic driving function, or the like. In this case, an analysis unit for analyzing the detection target of the mission information may be provided on the unmanned aircraft or the automobile side.

Further, the above-mentioned mission information is not limited to the example, and may be, for example, the provision of information on a specific person, object, phenomenon, or the like, or information on the user's home.

The information providing system according to the present invention also has the following configurations.

[Structure 1]
An information providing system including a management server and a plurality of information acquisition devices connected to the management server via a network.
The information acquisition device is:
Receives mission information including judgment condition information and processing processing information;
The first external world information is acquired based on the determination condition information, and it is determined whether or not the first external world information satisfies the determination condition;
When the first outside world information satisfies the determination condition, the second outside world information is acquired, and the second outside world information is processed based on the processing processing information to generate processing information;
Sending the processing information to the management server,
Information provision system.

[Structure 2]
The information providing system described in Configuration 1,
The information acquisition device transmits the processing information to the management server according to the accumulation status.
Information provision system.

[Structure 3]
The information providing system according to the configuration 1 or the configuration 2.
The management server includes a mission information management server that manages the mission information and a processing information management server that receives the processing information.
The information acquisition device is:
The mission information is received from the mission information management server;
Sending the processing information to the processing information management server,
Information provision system.

[Structure 4]
The information providing system according to any one of the first to third configurations.
The first means for acquiring information on the outside world is different from the second means for acquiring information on the outside world.
Information provision system.

[Structure 5]
The information providing system according to any one of the above configurations 1 to 4.
The second outside world information is obtained by the same acquisition means as the first outside world information.
Information provision system.

[Structure 6]
The information providing system according to any one of configurations 1 to 5.
The information acquisition device is provided with a sound collecting unit.
At least one of the first external world information and the second external world information is voice information acquired by the sound collecting unit.
Information provision system.

[Structure 7]
The information providing system according to any one of configurations 1 to 6.
The management server grants the user a reward associated with the mission information according to the processing information.
Information provision system.

[Structure 8]
The information providing system according to any one of configurations 1 to 7.
The management server evaluates the processing information acquired from the information acquisition device according to the evaluation conditions, and evaluates the processing information.
The reward is calculated based on the evaluation.
Information provision system.

[Structure 9]
The information providing system according to the configuration 8.
The content of the processing processing information can be changed within a predetermined range by designation from the information acquisition device.
The management server corrects the reward according to the designation.
Information provision system.

[Structure 10]
An information providing system including a management server and a plurality of information acquisition devices connected to the management server via a network.
The information acquisition device is:
Receive mission information including judgment condition information;
The first external world information is acquired based on the determination condition information, and it is determined whether or not the first external world information satisfies the determination condition;
Generates information regarding the presence or absence of an object satisfying the determination condition in the first external world information;
Send the generated information to the management server.
Information provision system.

[Structure 11]
An information providing system including a management server and a plurality of information acquisition devices connected to the management server via a network.
The information acquisition device is:
Receive mission information including judgment condition information;
The first external world information is acquired based on the determination condition information, and it is determined whether or not the first external world information satisfies the determination condition;
When the first outside world information satisfies the judgment condition, the second outside world information is acquired;
Sending the second outside world information to the management server,
Information provision system.

[Structure 12]
An information providing system including a management server and a plurality of information acquisition devices connected to the management server via a network.
The information acquisition device is:
Receive mission information including judgment condition information;
The first external world information is acquired based on the determination condition information, and it is determined whether or not the first external world information satisfies the determination condition;
Judgment result information based on the above judgment is generated;
The determination result information is transmitted to the management server.
Information provision system.

1 Management server 2, 2'Information acquisition device

Claims (14)

Including user terminal and management server
The user terminal stores mission information including determination condition information necessary for determining whether or not an image satisfies a determination condition and processing processing information instructing how to process the image.
An image is acquired from the camera, it is determined whether or not the image satisfies the determination condition, and when the image satisfies the determination condition, the second information obtained by processing the image according to the processing processing information is created. The second information is transmitted to the management server, and the second information is transmitted to the management server.
The management server stores a compensation information for calculating a reward in relation to the transmission information, in response to the second information, calculates the reward to be given to the user based on the reward information,
Information processing system. The management server evaluates the second information according to the evaluation conditions , and calculates the reward given to the user based on the evaluation.
The information processing system according to claim 1. If the management server has received the same information as the second information in the past, the management server reduces the reward.
The information processing system according to claim 1. The information processing system according to any one of claims 1 to 3 , wherein the user terminal transmits the second information to the management server according to a predetermined transmission condition. The information processing system according to any one of claims 1 to 3 , wherein the user terminal transmits the second information to the management server at predetermined time intervals. The information processing system according to any one of claims 1 to 3 , wherein the user terminal transmits the second information to the management server according to a transmission instruction. The information processing apparatus according to any one of claims 1 to 6.
Wherein processing the image in accordance with the processing information includes extracting a portion of the image, the information processing apparatus. The information processing apparatus according to any one of claims 1 to 6.
Wherein processing the image in accordance with the processing information includes cutting a predetermined condition is satisfied part from among the image information processing apparatus. The information processing apparatus according to any one of claims 1 to 6.
Wherein processing the image in accordance with the processing information includes applying a mosaic at least a portion other than the predetermined condition is satisfied portion in the image, the information processing apparatus. The information processing apparatus according to any one of claims 1 to 6.
Wherein processing the image according to the processing information comprises reading a character string by analyzing the image, the information processing apparatus. The information processing apparatus according to any one of claims 1 to 6.
Processing the image in accordance with the processing information, when the image satisfies the determination condition, causing incidental acquires sensor information from the sensor, the information processing apparatus. The information processing apparatus according to claim 11.
The sensor is an information processing device that acquires at least one of position information, an image, time, temperature, atmospheric pressure, operating status, or voice. The information processing apparatus according to any one of claims 1 to 6.
The processing to process the image according to the information, when the first of the images obtained from the camera is the determination condition is satisfied, to obtain a second image from said camera, said second image An information processing device that includes processing all or part of. The information providing system according to any one of claims 1 to 6.
The processing to process the image according to the information includes reading information about a particular those included in the image, the information providing apparatus.

Images