WO2020204015A1 - Information processing system, information processing device, and program - Google Patents

Abstract

The purpose of the present invention is to efficiently perform support for store operation including a work procedure and arrangement of an assistant in store management and support for education of the assistant on the basis of a situation inside a store and a situation outside the store.　In this information processing system, an imaging control unit 201 of a camera 2 images an object included in a range of 360 degrees of an omni-direction therearound. A generation unit 202 generates imaged image data of the object. An acquisition unit 101 of a server 1 acquires the imaged image data imaged and generated by each of the plurality of cameras 2, including at least a part of the inside and the outside of a store R. An operation support unit 102 displays operation support information for a store staff T of the store R on the basis of the imaged image data. An education support unit 103 displays education support information for the store staff T of the store R on the basis of the imaged image data. Thus, the above-mentioned problem is solved.

Description

Information processing systems, information processing devices, and programs

The present invention relates to an information processing system, an information processing device, and a program.

In recent years, the following issues (1) to (3) have existed in Japan.
That is, (1) "regional revitalization" aimed at correcting the situation where social capital such as politics, economy, and population is concentrated in urban areas and increasing the vitality of the entire country, and the local economy. There is a problem that "regional revitalization" aimed at improving power and increasing population has become a dead letter. As shown in Fig. 1A, it is said that this is due to the limitations of the government-led budget for balamaki and the commercialization of so-called "events", which are theoretical strategies on the desk.
In addition, (2) there is an urgent need to take measures against foreign tourists visiting Japan (inbound). As shown in Fig. 1A, this is due to the response to the tourism nation strategy based on the Tourism Nation Promotion Basic Law, the responsibility as a host country for international competitions such as the Olympic Games, and the increase and diversification of foreign tourists (inbound). It is also said that it has become.
In addition, (3) there is a problem that social demands are becoming more sophisticated. As shown in Fig. 1A, this is due to the response to systems such as HACCP (Hazard Analysis and Critical Control Point), the trend toward cashless society, and the so-called black-out (company) request due to work style reforms. It is also said that it has become.

As one of the methods for solving the problems listed in (1) to (3) above, there is a method of giving "town""earningpower". As shown in Fig. 1B, this is a "Sampo Yoshi" in which the three parties, consisting of managers, professional human resources, and finance, practice "Kasegu Machizukuri" in a well-balanced and fully exerted manner. It is a concept-based method. Specifically, the above three parties try to energize the person who provides the product or service by supporting the person who provides the product or service "by any means".
In other words, for example, by actively supporting restaurants that cook and provide ingredients harvested in the area, the "town" will be equipped with "earning power", and as a result, The problems listed in (1) to (3) above can be solved. Such a concept is also called "digital" hospitality "HACCP".

For example, the following technologies exist as conventional technologies that can be applied when supporting restaurants and the like. That is, in Patent Document 1, in order to enable the store manager to grasp the actual state of customer service, the analysis of the customer service status of the store is based on the data of the captured image captured by the camera that captures the inside of the store. The device that performs the above is described.

Japanese Patent No. 5314200

However, in the conventional technology including the technology described in Patent Document 1, since the analysis target is limited to the data of the image captured by the camera installed in the "store", it can have a great influence on the operation of the store. "Out-of-store" status is not included in the analysis.
The situation outside the store includes, for example, the situation of various events held around the store, the weather situation of the location of the store for people who come and go around the store, etc., all of which can affect the number of visitors to the store. It becomes a factor.

The present invention has been made in view of such a situation, and supports store operations including the work procedure and arrangement of assistants in store operation based on the situation inside the store and the situation outside the store. The purpose is to provide a method that can efficiently support the education of assistants.

In order to achieve the above object, the information processing system of one aspect of the present invention is
In an information processing system that supports a provider who provides goods or services to a customer at a predetermined place and an assistant of the provider.
Equipped with multiple imaging devices and information processing devices,
The plurality of image pickup devices
It has a generation means for capturing a subject included in a range of 360 degrees in all directions around the subject and generating data of the captured image of the subject.
The information processing device
An acquisition means for acquiring data of an captured image including at least a part of the field and the periphery of the field, which is captured and generated by each of the plurality of image pickup devices.
Based on the data acquired by the acquisition means, an operation support means for executing a first process for supporting an operation performed by the assistant in at least a part of the place and the surroundings of the place, and an operation support means.
Based on the data acquired by the acquisition means, the education support means for executing the second process for supporting the education of the assistant, and the education support means.
To be equipped.

Further, based on the data of the captured image that would be obtained if each of the plurality of imaging devices is arranged at the field and the plurality of predetermined positions around the field, the field and the surroundings of the field. It is possible to provide a proposal means for proposing the arrangement location and the number of the plurality of image pickup devices in the above.

Further, the information processing device according to one aspect of the present invention is
In an information processing device included in an information processing system that supports a provider who provides goods or services to a customer at a predetermined place and an assistant of the provider.
An acquisition means for acquiring data of an captured image including at least a part of the field and the periphery of the field, which is captured and generated by each of the plurality of image pickup devices.
Based on the data acquired by the acquisition means, an operation support means for executing a first process for supporting an operation performed by the assistant in at least a part of the place and the surroundings of the place, and an operation support means.
Based on the data acquired by the acquisition means, the education support means for executing the second process for supporting the education of the assistant, and the education support means.
To be equipped.

Further, based on the data of the captured image that would be obtained if each of the plurality of imaging devices is arranged at the field and the plurality of predetermined positions around the field, the field and the surroundings of the field. It is possible to provide a proposal means for proposing the arrangement location and the number of the plurality of image pickup devices in the above.

The information processing method and program according to one aspect of the present invention are the processing methods and programs corresponding to the above-mentioned information processing system according to one aspect of the present invention.

According to the present invention, based on the situation inside the store and the situation outside the store, support for store operation including the work procedure and arrangement of assistants in store operation and support for education of assistants are efficiently performed. be able to.

It is a figure which shows the social background in recent years. It is a figure which shows the social background in recent years. It is a diagram different from FIG. 1A. It is a figure which shows the outline of the store operation support service which can be realized by the information processing system of this invention. It is a figure which shows the type of a business targeted by this service. It is a figure which shows the effect by providing this service. It is an image diagram which shows the specific example of this service shown in FIG. 1A and FIG. 1B. It is a figure which shows the specific example of a plurality of cameras installed inside and outside a store. It is a flow chart which shows the flow of the proposed service in this service. It is an image diagram which shows the imaging result information. It is a figure which shows the outline of the structure of the information processing system of this invention. It is a block diagram which shows the hardware configuration of the server of FIG. FIG. 5 is a functional block diagram showing an example of a functional configuration required for various processes executed by the server of FIG. 9. It is a figure which shows an example of a common infrastructure operation model.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Service overview]
FIG. 2 is a diagram showing an outline of a store operation support service (hereinafter, referred to as “the service”) that can be realized by the information processing system of the present invention.

This service is a service that improves productivity by supporting providers who provide goods or services to customers in a predetermined place and their assistants.
For example, as an application example of this service, FIG. 2 shows a store manager P who provides food and drink and a customer service to customer C at a store R (predetermined place), and a store which is an assistant thereof. An example of supporting staff T is shown.

The store R is a restaurant operated by the store manager P. The store R provides the customer C who visits the store with so-called “local production for local consumption” dishes prepared by cooking ingredients harvested in the area.
The business partner M is a person who supports the business of the store R as a related business operator who is a business partner of the store R.
As described above, the customer C is a user of the store R.

This service can be realized by the cooperation of the server 1 (cloud), the camera 2, the store terminal 3, the business partner terminal 4, and the customer terminal 5 that constitute the information processing system according to the present invention.

The server 1 executes various processes for realizing this service while appropriately communicating with the camera 2, the store terminal 3, the business partner terminal 4, and the customer terminal 5. The server 1 also functions as a cloud that can be managed by AI (artificial intelligence).
The server 1 has, for example, information for supporting the operation of the store manager P in the store R (hereinafter referred to as “operation support information”) and store staff based on the data of the captured image transmitted from the camera 2. Generates information for supporting T's education (hereinafter referred to as "education support information").
The operation support information and the education support information generated by the server 1 are presented to at least one of the store manager P and the store staff T.
The presentation method is not particularly limited, and is presented by displaying the operation support information and the education support information on the manager terminal (not shown) operated by the store manager P or the store terminal 3 described later. May be good. In addition, other methods (for example, presentation by voice or presentation on paper media) may be used.

Further, the server 1 has captured image data (hereinafter, "simulation data") that would have been obtained if the plurality of cameras 2 had been arranged at each of the plurality of predetermined positions in the store R and its surroundings. To generate).
Then, the server 1 generates information (hereinafter, referred to as “proposal information”) indicating the optimum combination as the arrangement location and the number of the plurality of cameras 2 in the store R and its surroundings based on the generated simulation data.
A specific method for generating proposal information based on simulation data will be described later with reference to FIG.

The camera 2 is a camera installed at a plurality of locations inside and outside the store R. The camera 2 captures a subject included in the periphery of the installation location and generates data of the captured image of the subject. The specific configuration of the camera 2 will be described later with reference to FIG.
Since the camera 2 is an IoT (Internet of Things) device connected to a predetermined network N (FIG. 8) such as the Internet, the generated image data is uploaded to the cloud (server 1) at any time.

The store terminal 3 is an information processing device connected to a predetermined network N (FIG. 8) such as the Internet, and is connected to a cloud (server 1) like the camera 2. The store terminal 3 acquires operation support information, education support information, and the like from the cloud (server 1) and displays these information. As a result, the productivity of the store R can be improved.
The store terminal 3 is referred to as a dedicated application program (hereinafter referred to as "staff-only application") for receiving the provision of a service (hereinafter referred to as "staff support service") that supports the business of store staff T among the services. Call) is installed. The staff support service includes a service for displaying the above-mentioned operation support information and education support information on the store terminal 3.
In the following, unless otherwise specified, when the expression "store staff T operates the store terminal 3", it means that the store staff T activates the staff-dedicated application installed on the store terminal 3 to perform various types. It shall mean performing an operation.

The business partner terminal 4 is an information processing device operated by the business partner M.
The business partner terminal 4 is a service (hereinafter referred to as "business partner support service") for supporting the business partner M who intends to provide business support (for example, various managements) to the store manager P among the services. A dedicated application program (hereinafter referred to as "customer-only application") for receiving the service is installed.
The business partner support service includes information to support business strategies, including information on business creation (for example, product development), information to support legal compliance, and information on customer creation. A service to be displayed on the business partner terminal 4 is included.
By providing the business partner support service, the productivity of the store R can be further improved by the business support from the business partner M, and the business opportunities of the business partner M itself can be expanded. As a result, it is possible to contribute to the support of "optimization" described later with reference to FIG.
In the following, unless otherwise specified, when the expression "customer M operates the business partner terminal 4", the business partner M activates the business partner dedicated application installed on the business partner terminal 4. It shall mean that various operations are performed.

The customer terminal 5 is an information processing device operated by the customer C.
The customer terminal 5 is provided with a service for promoting the use of the store R and supporting the customer C who actually uses the store R (hereinafter referred to as "customer support service") among the services. A dedicated application program (hereinafter referred to as "customer-only application") is installed. The customer support service includes a service for displaying information on the creation of customer satisfaction on the customer terminal 5.
By providing the customer support service, customer communication including media and CRM (Customer Relationship Management) will be achieved between the store R and the customer C.
As a result, the customer C can use the service that meets various consumer needs, and can contribute to the support of "growth" described later with reference to FIGS. 3A and 3B.
In the following, unless otherwise specified, when the expression "customer C operates the customer terminal 5", it means that the customer C activates the customer-specific application installed on the customer terminal 5 and performs various operations. It shall mean to do.

In this service, at least a part of various information handled by each of the store terminal 3, the business partner terminal 4, and the customer terminal 5 is sent to the regional information agency and the regional financial institution via the cloud managed by the server 1. Provided to.
Here, the "regional information agency" refers to an agency that specializes in acquiring and providing basic data related to a specific region. Regional intelligence agencies include, for example, regional versions of intelligence banks. In addition, "regional financial institution" refers to a financial institution whose main business base is in a specific area, specifically, regional banks, second regional banks, credit unions, credit unions, agricultural cooperatives, etc. Refers to fishery cooperatives, labor banks, etc.

Regional information agencies and regional financial institutions that receive various types of information through this service can provide business support to businesses and local communities in other regions based on the provided information. As a result, it is possible to prevent regional revitalization and regional revitalization from becoming a mere ghost.

FIGS. 3A and 3B are diagrams showing the types of businesses targeted by this service and the effects of providing this service.

FIG. 3A shows a diagram that visualizes the types of businesses that should be targeted as the target of providing this service.
Specifically, in Fig. 3A, the brand value of the business evaluated based on the acquired standards, etc. is defined as the "brand level" (vertical axis), and the business is evaluated temporally as the "business stage" (horizontal axis). Business classification is made.

Here, the "hospitality standard certification" shown in FIG. 3A is certified for each business operator that provides the service based on the standard certification system established to "visualize" the quality of the service. It refers to a standard.
Of the certifications that make up "hospitality standard certification,""redcertification" is a standard that allows businesses enthusiastic about service improvement efforts to receive certification, and is "red certification" and "gold certification." , "Navy blue certification", and "Purple certification" exist.
Of these, the "red certification" shown in FIG. 3A is a standard that is certified by the business operator himself declaring that the conditions for granting conditions are met (conforming).
Of the certifications that make up "hospitality standard certification,""goldcertification" is not shown, but it is given to businesses that have been certified by a third party to provide services that exceed customer expectations. It is a certified standard. A business operator certified with "gold certification" is considered to have a higher service level than a business operator certified with "red certification".
Of the certifications that make up the "hospitality standard certification," the "navy blue certification" shown in Fig. 3A is for businesses that have been certified by a third party to provide services with their own ingenuity. It is a certified standard. A business operator certified with "navy blue certification" is considered to be a business operator with a higher service level than a business operator certified with "gold certification".
Of the certifications that make up "hospitality standard certification,""purplecertification" is not shown, but a third party indicates that it provides services that greatly exceed customer expectations (that is, "hospitality"). It is a standard given to businesses certified by. Businesses that have been granted "purple certification" are considered to have a higher service level than businesses that have been granted "navy blue certification".

In addition, in Fig. 3A, "Ministry of Health, Labor and Welfare HACCP B compliance" is centered on general hygiene management, and based on the concept of HACCP, important control points are set and managed according to the type of industry and condition to the extent possible. It refers to the business operator for which records are being recorded.
In addition, in Fig. 3A, "HACCP certification" whose business stage is after the "transformation period" and whose brand level is "upper" or higher means that HACCP is functioning in the in-house hygiene management system. A business operator evaluated by a person (HACCP certification body). The HACCP certification body includes, for example, the industry group to which the business operator belongs, each local government, and the like.
In addition, in Fig. 3A, "ISO22000", whose business stage is after the "transformation period" and whose brand level belongs to "high-end", is the international standard of "food safety management system-requirements for organizations involved in food chains". A business operator that has been certified by the standard.
In addition, in Fig. 3A, "FSSC22000", whose business stage is after "growth / maturity" and whose brand level belongs to "high-end", is an international standard for food safety management systems in which ISO22000 is reinforced with additional requirements. A certified business operator.

Of these, the businesses that should be targeted for the provision of this service are those that belong to the filled-in part of Fig. 3A. That is, it becomes a business operator that satisfies the hospitality standard certification "navy blue certification" and "Ministry of Health, Labor and Welfare HACCP B compliance".

FIG. 3B shows a growth model of the business receiving this service.
As shown in Fig. 3B, "strengthening management", "securing human resources", and "improving operational efficiency" in the business are required to be "optimized", and this "optimization" has been realized. By doing so, "surplus funds" are created.
On the other hand, "growth" is required for "branding", "customer attraction UP", and "unit price UP" of products and services in the business, so the above "surplus funds" should be realized in order to realize this "growth". Is invested.
By this "investment", a business (for example, a store business) can be "grown", and if the "growth" spreads to peripheral businesses, it can eventually contribute to the revitalization of the local economy. ..

FIG. 4 is an image diagram showing a specific example of the service shown in FIGS. 1A and 1B.

In FIG. 4, the store R in which the cameras 2-1 to 2-7 are installed, a plurality of store staff T engaged in providing services in the store R, and the store R are used or can be used. A plurality of customers C are drawn.

The cameras 2-1 and 2-2 are cameras installed on the outer wall of the store R, and each capture a predetermined range outside the store R. Specifically, the camera 2-1 captures at least the subject included in the imaging regions A1-1 and A1-2 indicated by the broken line. The camera 2-2 captures at least the subjects included in the imaging regions A2-1 and A2-2 indicated by the broken lines.
Cameras 2-3 are cameras installed outside the entrance of the store R, and at least capture a subject included in the imaging region A3 indicated by the broken line.
Cameras 2-4 and 2-5 are cameras installed near the ceiling of the floor of the store R, and each capture a predetermined range of the floor of the store R. Specifically, the camera 2-4 captures at least the subject included in the imaging region A4 indicated by the broken line. The camera 2-5 captures at least the subject included in the imaging region A5 indicated by the broken line.
The camera 2-6 is a camera installed in the kitchen (kitchen) of the store R, and at least captures a subject included in the imaging region A6 indicated by the broken line.
The camera 2-7 is a camera installed in the warehouse of the store R, and at least captures a subject included in the imaging region A7 shown by the broken line.

In this way, since seven cameras 2-1 to 2-7 are installed everywhere inside and outside the store R, the data of the captured images of the cameras 2-1 to 2-7 can be used. Based on any analysis is possible. For example, by performing analysis by AI (artificial intelligence), it is possible to generate operation support information and education support information without blind spots.

Here, the specific installation mode of the cameras 2-1 to 2-7 is not particularly limited, but it is preferable that the camera is installed in consideration of being a camera from a human perspective.
In other words, it is preferable that the cameras 2-1 to 2-7 are installed so as to function as a substitute for the line of sight of the store manager P.
Specifically, the height of the installation location of the cameras 2-1 to 2-7 from the ground (floor) is preferably about 1.8 m to 2.5 m. As a result, the facial expressions of the store staff T and the customer C can be confirmed more accurately.
Further, it is preferable that the imaging regions of the cameras 2-1 to 2-7 are set so that at least a part of them overlap with each other. As a result, the data of the captured images of the cameras 2-1 to 2-7 can be seamlessly connected.

In addition, the following services and the like can be provided to the store staff T of the store R shown in FIG.
That is, by monitoring the traffic around the store R using the cameras 2-1 to 2-3 installed on the outer wall or the entrance / exit of the store R, the operation support information most suitable for the store staff T is generated. Can be presented.
Specifically, the number and appearance of a plurality of customers C existing as subjects in the imaging areas A1-1 and A2-1, the weather existing as subjects in the imaging areas A1-2 and A2-2, and the vicinity of the store R. Based on the events held in, for example, the operation related to the preparation of audience seats, the placement of store staff T (whether or not breaks are possible, etc.), the preparation in the kitchen (kitchen) (for example, burning), and the setting of lighting and air conditioning on the floor. It is possible to generate and present ration support information.

Further, the hospitality can be improved based on the visit situation of the customer C to the store R.
That is, by monitoring the visit status of the customer C using the cameras 2-3 installed at the entrance / exit of the store R, the operation support information and the education support information most suitable for the store staff T are generated and presented. be able to.
Specifically, for example, based on the number and state of customer C existing as a subject in the imaging area A3, the flow line of visits from the outdoors, or information based on face recognition using the camera 2-3, for example, customer C visits the store. Guidance of store staff T is performed according to the timing of. As a result, it is possible to avoid the loss of the visit opportunity for the customer C.
In addition, it is possible to take individual measures for each customer C based on the information about the customer C obtained from the captured image data. In other words, hospitality can be improved. In other words, it becomes possible to provide services based on the so-called "hospitality" that people provide to people.
In addition, it is possible to realize analytics based on information about customer C. In other words, it is possible to find out knowledge such as specific patterns and correlations hidden in information about customer C while making full use of various analysis methods and algorithms. As a result, for example, it is possible to adjust the lighting, sound, air conditioning, etc. according to the visit of the customer C.
In addition, the response (customer service) to the customer C by the store staff T can be monitored and accumulated as a business record. As a result, it becomes possible to evaluate the response (customer service) quality in real time by comparing with the customer service manual in the store R, and to distribute the E-learning content based on the real-time evaluation.

Further, by monitoring the table service provided by the store staff T, it is possible to generate and present the operation support information and the education support information that are optimal for the store staff T.
Specifically, using cameras 2-4 and 2-5 installed on the floor of store R, the availability of tables placed on the floor, the condition of customer C who visited the store, and the serving of food and drinks. The situation is monitored.
As a result, guidance to the table according to the number of customers C, instructions for the table set, care by the staff by detecting orders and troubles from customer C who came to the store, appropriate bashing of vacant tableware (work to lower the tableware) ) Is possible. In addition, it is possible to increase sales by recommending additional orders according to the amount and content of food and drink of customer C.
In addition, the service provision status to the customer C by the store staff T can be monitored and accumulated as a business record. This makes it possible to evaluate the response (customer service) quality in comparison with the customer service manual at store R, propose revisions to the customer service manual based on better quality services, and distribute e-learning content based on the real-time evaluation. Become.

In addition, by monitoring the accounting process for customer C by the store staff T and the status of seeing off customer C, it is possible to generate and present the operation support information and the education support information that are optimal for the store staff T. ..
Specifically, by monitoring the customer C who leaves the store and the store staff T corresponding to the customer C who leaves the store, the cameras 2-4 and 2-5 installed on the floor of the store R are used. For example, the following services can be realized.
That is, recording of customer C's staying time and behavior during stay, seamless payment corresponding to price multiplexing by face recognition, automatic notification of customer C's dine and dash, etc., customer C's see-off and addition by store staff T Services etc. can be realized.
In addition, by monitoring the causal relationship between the behavior of the customer C and the status of the customer service provided by the store staff T, it is possible to grasp the service pattern that contributes to the sales of the store R and propose effective promotion measures.

Further, by monitoring the status of various operations of the store staff T, it is possible to generate and present the operation support information and the education support information that are optimal for the store staff T.
Specifically, using the cameras 2-4 and 2-5 installed on the floor of the store R and the cameras 2-6 installed in the kitchen, the store staff T performs work such as cooking, cleaning, and replenishment, and materials. By monitoring the usage status of the above, the state of the staff, etc., for example, the following services can be realized.
That is, a record showing who performed what kind of work and what kind of error occurred, confirmation of whether the work was correct according to the work manual, and work that may cause an error (for example, the wrong equipment or material). Issuance of warnings (alerts, etc.) for performing (use), calculation of required inventory by accurately grasping the amount of work and usage fee, work instructions for cleaning and various replenishment, timing when store staff T starts work It is possible to realize a shift record based on the record of the store staff, confirmation and support of the health condition of the store staff T that can be judged from the appearance, and the like.
It is also possible to evaluate the work quality of the store staff T in real time and propose a manual for performing more effective work.

In addition, by monitoring the inventory and loss of products, it is possible to generate and present operation support information and education support information that are optimal for the store staff T.
Specifically, the delivery work (for example, labeling, delivery place) by the delivery company (for example, the customer M) can be optimized by using the camera 2-7 installed in the warehouse of the store R. In addition, it is possible to optimize the disposal method of waste and thoroughly manage hygiene.
In addition, by monitoring the delivery status, inventory status, disposal loss status, etc., the following effects can be expected, for example.
That is, thorough inventory management by the first-in first-out method, enabling operation with deposited inventory (that is, charging only for what is used), preventing loss by optimizing inventory management for half-cooking, preventing theft of ingredients by store staff T, It is possible to realize automation of additional orders according to the number of inventories, optimization of inventory storage at the time of delivery, and clarification of responsibilities by managing the history of information about the person who took out the inventory.

FIG. 5 is a diagram showing a specific example of a plurality of cameras 2 installed inside and outside the store R.

As described above, the specific configuration of the camera 2 is not particularly limited, but for example, by adopting the camera 2 shown in FIG. 5, this service can be provided in a more suitable environment. That is, it is possible to generate operation support information and education support information with more detailed contents.
Specifically, the camera 2 of FIG. 5 is configured to include a main body portion 211 and a lens portion 212, and can capture a subject included in a range of 360 degrees in all directions around the subject, and an captured image of the subject. It is an ultra-high resolution camera that can generate the data of. The term "ultra-high resolution" as used herein means that the value indicating the resolution is particularly high, and refers to a camera having a resolution of, for example, 4K (8 million pixels) or more.

FIG. 6 is a flow chart showing the flow of the proposed service among the services.

As described above, the server 1 generates information (proposal information) indicating the optimum combination as the arrangement location and the number of the plurality of cameras 2 in the store R and its surroundings based on the generated simulation data. The generated proposal information is presented to the store manager P. Hereinafter, such a service will be referred to as a "proposal service".
As a result, the store manager P can efficiently receive the service while maximizing the cost effectiveness.

Specifically, as shown in FIG. 6, in the construction in which a plurality of cameras 2 are arranged in the store R, the server 1 generates simulation data before the construction (step SS1), and the server 1 generates simulation data based on the generated simulation data. , Generate proposal information. The proposal information generated by the generated server 1 is presented to the store manager P.

Here, the simulation data can be generated by using, for example, design / simulation software using 3D rendering technology.
In the generation of simulation data, the accumulated past data, the input contents to various parameters, and the face data can be referred to. In addition, AI (artificial intelligence) used to generate simulation data and various data (sensors, etc.) can be plugged in one after another. Further, the camera 2 can be embedded as the interior of the store R. Further, as an installation mode of the camera 2, it can be installed in a light bulb (pendant light), a table candle, an artificial flower, or the like. In addition, the communication environment (Wifi, etc.) and power supply environment of the installed camera 2 can be simulated. In addition, it can be simulated so that it is installed at the height of the faces of the store staff T and the customer C.
The specific method of proposing the proposal information to the store manager P is not particularly limited. It may be presented by displaying the proposal information on the manager terminal (not shown) operated by the store manager P. In addition, other methods (for example, presentation by voice or presentation on paper media) may be used.
In addition, the presentation of the proposal information can be an opt-in type service performed based on a request from the store manager P.

During the construction (step SS2), the store manager P performs the construction based on the proposal information presented from the server 1. That is, the store manager P performs the construction according to the simulation data and the information related to the design included in the presented proposal information.
Here, the proposed information presented is preferably information that can be proposed business-wise. For example, the information on the interior and layout of the store R is information that an architect or a furniture store can propose business-wise.

After the construction (step SS3), the store manager P starts the operation in the store R in which each of the plurality of cameras 2 is installed at a predetermined position.
By using the proposal service, the store manager P can smoothly proceed with various preparations from the design of the store R to the actual start of operation, and can be obtained by using this service even after the start of operation. The benefits can be maximized.

FIG. 7 is an image diagram showing imaging result information.

The captured image data captured by the plurality of cameras 2 includes information for uniquely identifying the captured image data (hereinafter, referred to as "image ID") and information regarding the time when the captured image was captured (hereinafter, "Time stamp"). Information (referred to as "information") and information regarding the imaged location (hereinafter referred to as "location information") are associated with each other, and are stored and managed in the imaging result DB181 as imaging result information.
The imaging result information is combined with AI (artificial intelligence) to form the framework of the digital infrastructure required to provide this service.

As shown in FIG. 7, the imaging result information includes a history (hereinafter, referred to as “communication log”) regarding communication between the store staff T and the customer C, between the store staff T, and between the customer C. .. From the communication log, information on the sensibilities of the store staff T and the customer C, who are the subjects of communication (hereinafter referred to as "sensitivity data"), can be extracted. The extracted Kansei data is used as one of the basis data for generating operation support information and education support information.

Further, the imaging result information includes not only the store staff T and the customer C, but also the sensing of the subject included in the omnidirectional 360-degree range around each of the plurality of cameras 2 installed inside and outside the store R. Data (hereinafter referred to as "sensing log") is included. From the sensing log, physical information (hereinafter referred to as "physical data") regarding each of the subjects that are the objects of sensing can be extracted. The extracted physical data is used as one of the basis data for generating operation support information and education support information.

In addition, since the imaging result information includes all the results related to the provision of products or services provided to customer C at store R, it is used for traceability or as marketing material. be able to.

[System configuration]
FIG. 8 is a diagram showing an outline of the configuration of the information processing system of the present invention.

The information processing system shown in FIGS. 1A and 1B (hereinafter referred to as "this system") includes a server 1, cameras 2-1 to 2-n (n is an integer value of 1 or more), a store terminal 3, and the like. The business partner terminal 4 and the customer terminal 5 are connected to each other via a predetermined network N such as the Internet.
This system is a system required to provide the above-mentioned service.
Hereinafter, when it is not necessary to distinguish the cameras 2-1 to 2-n individually, these are collectively referred to as "camera 2".

The server 1 is an information processing device managed by the service provider G. The server 1 executes various processes in order to manage the operations of the cameras 2-1 to 2-n, the store terminal 3, the business partner terminal 4, and the customer terminal 5.
Each of the cameras 2-1 to 2-n is a camera installed at a plurality of locations inside and outside the store R. Each of the cameras 2-1 to 2-n captures a subject included in the vicinity of the installation location and generates data of the captured image of the subject.
Key image IDs, Timestamp information, and Location information are shared and associated with the data of the captured images captured by each of the cameras 2-1 to 2-n, and the imaging result DB181 is linked as the imaging result information. It is stored and managed in.

(Store terminal)
As described above, the store terminal 3 is an information processing device operated by the store staff T. The store terminal 3 is composed of a personal computer, a smartphone, a tablet, and the like.
The store terminal 3 receives an input operation required for the store staff T to use the staff support service, and transmits information indicating the input content to the management server 1. In addition, the store terminal 3 displays various information transmitted from the server 1 by itself or under the control of the server 1.

(Customer terminal)
As described above, the business partner terminal 4 is an information processing device operated by the business partner M. The business partner terminal 4 is composed of a personal computer, a smartphone, a tablet, and the like.
The business partner terminal 4 receives an input operation required for the business partner M to use the business partner support service, and transmits information indicating the input content to the management server 1. Further, the business partner terminal 4 displays various information transmitted from the server 1 by itself or under the control of the server 1.

(Customer terminal)
As described above, the customer terminal 5 is an information processing device operated by the customer C. The customer terminal 5 is composed of a personal computer, a smartphone, a tablet, and the like.
The customer terminal 5 receives an input operation required for the customer C to use the customer support service, and transmits information indicating the input content to the management server 1. Further, the customer terminal 5 displays various information transmitted from the server 1 by itself or under the control of the server 1.

[Hardware configuration]
Next, the hardware configuration of the server 1 that executes various processes in this system will be described.
FIG. 9 is a block diagram showing a hardware configuration of the server 1 of FIG.

The server 1 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a bus 14, an input / output interface 15, an output unit 16, and an input unit 17. A storage unit 18, a communication unit 19, and a drive 20 are provided.

The CPU 11 executes various processes according to the program recorded in the ROM 12 or the program loaded from the storage unit 18 into the RAM 13.
Data and the like necessary for the CPU 11 to execute various processes are also appropriately stored in the RAM 13.

The CPU 11, ROM 12 and RAM 13 are connected to each other via the bus 14. An input / output interface 15 is also connected to the bus 14. An output unit 16, an input unit 17, a storage unit 18, a communication unit 19, and a drive 20 are connected to the input / output interface 15.

The output unit 16 is composed of various liquid crystal displays and the like, and outputs various information.
The input unit 17 is composed of various hardware leads and the like, and inputs various information.
The storage unit 18 is composed of a DRAM (Dynamic Random Access Memory) or the like, and stores various data.
The communication unit 19 communicates with other devices (for example, cameras 2-1 to 2-n in FIG. 8, store terminal 3, customer terminal 4, customer terminal 5) via a network N including the Internet. Control.

The drive 20 is provided as needed. A removable medium 30 made of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted on the drive 20. The program read from the removable media 30 by the drive 20 is installed in the storage unit 18 as needed. Further, the removable media 30 can also store various data stored in the storage unit 18 in the same manner as the storage unit 18.

Although not shown, the store terminal 3, the customer terminal 4, and the customer terminal 5 also have the above-mentioned hardware configuration.

[Functional configuration]
Next, the functional configuration of the server 1 having the hardware configuration of FIG. 9 will be described.
FIG. 10 is a functional block diagram showing an example of a functional configuration required for various processes executed by the server 1 of FIG.

As shown in FIG. 10, in the CPU 11 of the server 1, the acquisition unit 101, the operation support unit 102, the education support unit 103, and the proposal unit 104 function.

The acquisition unit 101 acquires the data of the captured image including at least a part of the field and the periphery of the field, which is captured and generated by each of the plurality of imaging devices.
Specifically, the acquisition unit 101 acquires the data of the captured image including at least a part of the store R and the periphery of the store R, which are captured and generated by the cameras 2-1 to 2-n, respectively.
As described above, the acquired image data is stored and managed in the image capture result DB 181 as the image capture result information in association with the key image ID, Timestamp information, and Location information.

The operation support unit 102 executes the first process for supporting the operation performed by the staff in at least a part of the place and the surroundings of the place based on the data of the captured image acquired by the acquisition unit 101.
Specifically, the operation support unit 102 generates operation support information based on the image pickup result information stored in the image pickup result DB181. Further, the operation support unit 102 presents the generated operation support information to at least one of the administrator terminal (not shown) and the store terminal 3.

The education support unit 103 executes the second process for supporting the education of the staff based on the data of the captured image acquired by the acquisition unit 101.
Specifically, the education support unit 103 generates education support information based on the image pickup result information stored in the image pickup result DB181. Further, the education support unit 103 presents the generated education support information to at least one of the manager terminal (not shown) and the store terminal 3.

The proposal unit 104 is based on the data of the captured image that would be obtained if each of the plurality of imaging devices is arranged at the field and the plurality of predetermined positions around the field, and the surroundings of the field and the field. We propose the arrangement location and the number of multiple image pickup devices in.
Specifically, the proposal unit 104 generates simulation data and proposal information based on the simulation data, and presents the simulation data to at least one of the manager terminal (not shown) and the store terminal 3.

Further, as shown in FIG. 10, in a CPU (not shown) of the camera 2, the image pickup control unit 201 and the generation unit 202 function.

The image pickup control unit 201 executes control for photographing a subject included in the vicinity of the installation location of the own machine.
The generation unit 202 generates the data of the captured image of the captured subject as a result of the control by the image pickup control unit 201.
The generated captured image data is transmitted to the server 1 at a predetermined timing. The timing at which the generated captured image data is transmitted to the server 1 is not particularly limited. It may be transmitted in real time together with the generation of the captured image data, or may be transmitted every few seconds or every few minutes.

By cooperating with the server 1 having the above functional configuration and the camera 2, each process related to the provision of this service can be executed.
As a result, the service provider G can provide this service to each of the store manager P, the store staff T, the business partner M, and the customer C.

(Common infrastructure management model)
FIG. 11 is a diagram showing an example of a common infrastructure operation model.

By positioning the various information accumulated in this service as a resource or a common base of the local community, it will be possible to operate and deploy it in a consortium type. Hereinafter, such a model will be referred to as a "common infrastructure operation model".

As shown in Fig. 11, the common infrastructure management model is "market", "target organization", "management body", "cooperation between regional financial institutions and VC (venture capital)", and "collaboration business". This is a consortium-type management model in which "persons (investors)", "management councils", and "solution providers" cooperate with each other.

Specifically, the "market" in the common infrastructure management model is a place for business activities by "target organizations, etc." As a result, the "target organization, etc." can obtain profit (sales).

The "solution provider" in the common infrastructure operation model is a person who provides services related to "planning, development, and maintenance" to "target organizations, etc." As a result, the "solution provider" can obtain profits (development costs, etc.).

The "operation body" in the common infrastructure operation model provides maintenance services for standard data resources owned by the "target organization, etc." In addition, the "operating body" will fund its own data resources owned by the "target organization, etc." In addition, the "management body" manages the "management council". As a result, the "operating body" can obtain profits (returns, operating expenses, etc.).

The "operation council" in the common infrastructure operation model provides various "certifications" and "support" to "solution providers". As a result, the "management council" can earn profits (membership fee and certification fee).

The "collaboration company (investor)" in the common infrastructure management model supports the "management body". As a result, the "collaboration business operator (investor)" can obtain profit (revenue share) from the "operating body (investor)". In addition, revenue (commissions, data usage fee billing, etc.) can be obtained from "target organizations, etc." through "cooperation between regional financial institutions and VC (venture capital)".

Although one embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and modifications, improvements, etc. within the range in which the object of the present invention can be achieved are included in the present invention. Is.

For example, according to this service, in addition to the above-mentioned merits, the following merits can be enjoyed.
That is, it is possible to grasp the spatial rendering and the sensitivity information of the store staff T and the customer C. Further, by using the camera 2 as an infrared camera or a night-vision camera, it is possible to support even if the imaging region is dark. Further, when the camera 2 is provided with a temperature sensor, it is possible to grasp the temperature by, for example, far infrared rays. In addition, the discharge state can be grasped by ultraviolet rays. In addition, by linking with an AR (augmented reality) lens, proposal information can be presented using a laser or MAP. In addition, digital support is possible even if the level of work proficiency is low.

Further, for example, in the above-described embodiment, the store R, which is a restaurant, has been described as an example of the application target of this service, but this is merely an example, and the application target of this service is a restaurant such as a restaurant. Not limited. For example, it can be applied to all places and fields such as warehouses, factories, schools, welfare / medical / nursery facilities, sports / entertainment / craftsmen, events, offices, civil engineering / construction sites, etc.

When this service is applied to a warehouse, for example, the following services can be provided by installing multiple cameras 2 at predetermined positions and monitoring the operating status of the warehouse, the movement of importers and exporters, the inventory status, etc. It can be realized.
That is, instructions on the location of inventory items, additional orders according to inventory status, appropriate management of inventory loss such as expiration date by the first-in first-out method, proper allocation of personnel and equipment, impact in the event of trouble such as an accident in the warehouse It is possible to grasp the range and realize the response instruction.

When this service is applied to a factory, for example, the following services can be provided by installing multiple cameras 2 at predetermined positions and monitoring the factory operating status, staff movements, inventory status, production status, etc. Can be realized.
That is, confirmation of staff work status and placement status (for example, error check), early warning (alert, etc.) in processes where errors can occur, response instructions that minimize loss when an error occurs, automation linked with sensors It is possible to realize automation, transfer of materials according to the operating status, additional orders according to the inventory status, and the like.

When this service is applied to schools, for example, by installing multiple cameras 2 at predetermined positions and monitoring the learning status of children / students, the content of teachers' guidance, the locations of children / students / teachers, etc. The following services and the like can be realized.
That is, development of an educational environment that is safe for the community and parents, efficient procurement of necessary teaching materials through regional cooperation (for example, sharing, ecology, etc.), and learning support for children and students (for example, if the movement of the hand stops). (Service that teaches), confirmation of whether the teacher's guidance is appropriate, development of a more appropriate teaching method, grasping who is where, prevention of missing persons by guiding personnel, etc. can be realized.

When this service is applied to welfare / medical / childcare facilities, multiple cameras 2 are installed at predetermined positions to monitor the user status, staff status, user / staff location, etc. For example, the following services can be realized.
That is, prevention of wandering and safety management of the elderly, reduction of the burden on staff, appropriate response such as express of staff in the event of trouble, immediate response to requests from users (for example, toilets, meals, etc.), when an accident occurs It is possible to realize records of accidents, analysis of their causes, trucks of accident signs, and instructions for accident prevention.

When this service is applied to fields such as sports, performing arts, craftsmen, etc., multiple cameras 2 are installed at predetermined positions, and the situation of the learner (instructor), the situation of the instructor, and the result (result) By monitoring the process of the above, for example, the following services can be realized.
That is, proposal of improvement method based on evaluation of implementation result, analysis of gap between instruction content and effect, proposal of necessary instruction, analysis of effective learning method (instruction method) according to learner (instructor)・ It is possible to realize proposals, analysis of tacit knowledge that is a craftsmanship, and proposals for teaching methods. It can also prevent lost technology and create training content for technology succession.

When this service is applied to an event, the status of vacant seats, the status of visitors, and the status of staff can be obtained by installing multiple cameras 2 at predetermined positions or using a drone or other flying object equipped with the cameras 2. By monitoring the status of the contents, for example, the following services can be realized.
That is, optimization / verification of event management manuals, evaluation of the effects of contents such as halftime shows, analysis / prediction of congestion status and guidance of vacant seats, navigation to seats tailored to visitors, service providers such as drink sales, etc. Instructions, records when troubles occur and prevention by predicting signs of troubles, congestion analysis and securing of flow lines at the end of events, analysis of evacuation routes when disasters occur, management of customer dynamics according to the excitement of contents ( For example, when to go on sale) can be realized.

When this service is applied to an office, multiple cameras 2 should be installed at predetermined positions to monitor the status of staff, the status of installed equipment, the status of visitors, etc., the space within a predetermined range including the office, etc. Therefore, for example, the following services can be realized.
That is, analysis of work efficiency and communication, proposal of measures to improve productivity (for example, promotion of breaks), sleep of unnecessary equipment, contribution to energy saving, automatic call of the person in charge at the time of visitor, security management It is possible to realize countermeasures (for example, management of entry / exit), optimization of space, proposal of methods for effectively utilizing dead space, and the like.

When this service is applied to civil engineering and construction sites, accidents can be prevented by, for example, setting up poles in a space, installing a plurality of cameras 2 at predetermined positions, and monitoring the work status.

That is, this service can be realized by forming a digital network for each of research and development, production, distribution, service, and community.
Specifically, for example, linking e-learning, AI speakers, multilingual menus, personal remittances, check-in / check-out systems, ordering systems, traceability systems, high-resolution monitoring systems, communication media, social capital credit evaluation systems, etc. So, this service can be realized.

Further, the hardware configuration shown in FIG. 9 is merely an example for achieving the object of the present invention, and is not particularly limited.

Further, the functional block diagram shown in FIG. 10 is merely an example and is not particularly limited. That is, it is sufficient that the information processing system is provided with a function capable of executing the above-mentioned series of processes as a whole, and what kind of functional block is used to realize this function is not particularly limited to the example of FIG. ..

Further, the location of the functional block is not limited to FIG. 10, and may be arbitrary.
Further, one functional block may be configured by a single piece of hardware, a single piece of software, or a combination thereof.

When the processing of each functional block is executed by software, the programs constituting the software are installed on a computer or the like from a network or a recording medium.
The computer may be a computer embedded in dedicated hardware. Further, the computer may be a computer capable of executing various functions by installing various programs, for example, a general-purpose smartphone or a personal computer in addition to a server.

The recording medium containing such a program is not only composed of removable media, which is distributed separately from the device main body to provide the program to each user, but also is preliminarily incorporated in the device main body to each user. It is composed of the provided recording media and the like.

In the present specification, the steps for describing a program recorded on a recording medium are not necessarily processed in chronological order according to the order, but are not necessarily processed in chronological order, but in parallel or individually. It also includes the processing to be executed.

Further, in the present specification, the term of the system means an overall device composed of a plurality of devices, a plurality of means, and the like.

In summary, the information processing system to which the present invention is applied need only have the following configuration, and can take various embodiments.
That is, the information processing system to which the present invention is applied is
In an information processing system that supports a provider (for example, store manager P) who provides goods or services to a customer (for example, customer C) at a predetermined place, and an assistant (for example, store staff T) of the provider.
A plurality of imaging devices (for example, camera 2) and an information processing device (for example, server 1) are provided.
The plurality of image pickup devices
It has a generation means (for example, generation unit 202 in FIG. 10) that images a subject included in a range of 360 degrees in all directions around the subject and generates data of the captured image of the subject.
The information processing device
An acquisition means (for example, acquisition unit 101 in FIG. 10) for acquiring image data including at least a part of the field and the periphery of the field, which is imaged and generated by each of the plurality of image pickup devices.
Based on the data acquired by the acquisition means, the operation support means (for example, the figure) that executes the first process for supporting the operation performed by the assistant in at least a part of the place and the surroundings of the place. 10 Operation Support Department 102) and
Based on the data acquired by the acquisition means, an education support means (for example, the education support unit 103 in FIG. 10) that executes a second process for supporting the education of the assistant, and
To be equipped.

As a result, it is possible to efficiently support the operation of the assistant in the place and the education of the assistant based on the situation of the place (for example, store R) and the situation around the place.

Further, based on the image data (for example, simulation data) that would be obtained if each of the plurality of imaging devices is arranged at the field and the plurality of predetermined positions around the field, the field is used. And the proposal means (for example, the proposal unit 104 of FIG. 10) that proposes the arrangement location and the number of the plurality of image pickup devices around the place (for example, proposal information).
Can be provided.

As a result, the manager of the place can efficiently receive this service while maximizing cost effectiveness.

Further, the information processing device to which the present invention is applied is
In an information processing device included in an information processing system that supports a provider who provides goods or services to a customer at a predetermined place and an assistant of the provider.
An acquisition means for acquiring data of an captured image including at least a part of the field and the periphery of the field, which is captured and generated by each of the plurality of image pickup devices.
Based on the data acquired by the acquisition means, an operation support means for executing a first process for supporting an operation performed by the assistant in at least a part of the place and the surroundings of the place, and an operation support means.
Based on the data acquired by the acquisition means, the education support means for executing the second process for supporting the education of the assistant, and the education support means.
To be equipped.

As a result, it is possible to efficiently support the operation of the assistant in the place and the education of the assistant based on the situation of the place and the situation around the place.

Further, based on the data of the captured image that would be obtained if each of the plurality of imaging devices is arranged at the field and the plurality of predetermined positions around the field, the field and the surroundings of the field. Proposal means for proposing the arrangement location and the number of the plurality of image pickup devices in the above can be provided.

As a result, the manager of the place can efficiently receive this service while maximizing cost effectiveness.

1: Server, 2,2-1 to 2-n: Camera, 3: Store terminal, 4: Business partner terminal, 5: Customer terminal, 11: CPU, 12: ROM, 13: RAM, 14: Bus, 15: Input / output interface, 16: Output unit, 17: Input unit, 18: Storage unit, 19: Communication unit, 20: Drive, 30: Removable media, 101: Acquisition unit, 102: Operation support unit, 103: Education support unit, 104: Proposal unit, 201: Imaging control unit, 202: Generation unit, 212: Lens unit, 211: Main unit, 181: Imaging result DB, G: Service provider, T: Store staff, R: Store, P: Store Administrator, M: Business partner, C: Customer, A1-1, A1-2, A2-1, A2-2, A3 to A7: Imaging area, SS: Each step in this service

Claims (5)

1. In an information processing system that supports a provider who provides goods or services to a customer at a predetermined place and an assistant of the provider.
   Equipped with multiple imaging devices and information processing devices,
   The plurality of image pickup devices
   It has a generation means for capturing a subject included in a range of 360 degrees in all directions around the subject and generating data of the captured image of the subject.
   The information processing device
   An acquisition means for acquiring data of an captured image including at least a part of the field and the periphery of the field, which is captured and generated by each of the plurality of image pickup devices.
   Based on the data acquired by the acquisition means, an operation support means for executing a first process for supporting an operation performed by the assistant in at least a part of the place and the surroundings of the place, and an operation support means.
   Based on the data acquired by the acquisition means, the education support means for executing the second process for supporting the education of the assistant, and the education support means.
   Information processing system equipped with.
2. Based on the data of the captured image that would be obtained if each of the plurality of imaging devices was arranged at the field and the plurality of predetermined positions around the field, the field and the surroundings of the field. The information processing system according to claim 1, further comprising a proposing means for proposing the arrangement location and the number of a plurality of imaging devices.
3. In an information processing device included in an information processing system that supports a provider who provides goods or services to a customer at a predetermined place and an assistant of the provider.
   An acquisition means for acquiring data of an captured image including at least a part of the field and the periphery of the field, which is captured and generated by each of the plurality of image pickup devices.
   Based on the data acquired by the acquisition means, an operation support means for executing a first process for supporting an operation performed by the assistant in at least a part of the place and the surroundings of the place, and an operation support means.
   Based on the data acquired by the acquisition means, the education support means for executing the second process for supporting the education of the assistant, and the education support means.
   Information processing device equipped with.
4. Based on the data of the captured image that would be obtained if each of the plurality of imaging devices was arranged at the field and the plurality of predetermined positions around the field, the field and the surroundings of the field. The information processing apparatus according to claim 3, further comprising a proposing means for proposing the arrangement location and the number of a plurality of imaging devices.
5. For information processing equipment included in an information processing system that supports a provider who provides goods or services to a customer at a predetermined place and an assistant of the provider.
   The acquisition step of acquiring the data of the captured image including at least a part of the field and the periphery of the field, which is captured and generated by each of the plurality of imaging devices, and
   Based on the data acquired by the acquisition means, an operation support step for executing a first process for supporting an operation performed by the assistant in at least a part of the field and the surroundings of the field, and an operation support step.
   Based on the data acquired by the acquisition means, an education support step for executing a second process for supporting the education of the assistant, and an education support step.
   A program that executes control processing including.
   

Images