JP2022145822A - Video processing apparatus, video processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To count the number of times that a store clerk provides services to customers without requiring complicated design operations.

SOLUTION: A video processing apparatus includes an analysis unit, a counting unit, and a storage unit. The analysis unit analyzes video data of a business scene in a store, and determines, for each image frame in the video data, whether a customer service behavior of an employee has occurred in each table. The counting unit counts the number of times determined by the analysis unit to include a customer service behavior, as the number of customer services. The storage unit stores the number of customer services. The analysis unit includes a feature quantity extraction unit and a behavior determination unit. The feature quantity extraction unit extracts feature quantities, by image frame, from the video data. The behavior determination unit determines whether a customer service behavior of an employee has occurred in each table, on the basis of the extracted feature quantities.

SELECTED DRAWING: Figure 5

COPYRIGHT: (C)2023,JPO&INPIT

Description

The embodiments relate to a video processing device, a video processing method, and a program.

The revisit rate is an important management index in the service industry represented by the restaurant industry, and there are various consulting methods to increase the revisit rate of customers to the store. Above all, evaluating the number of times an employee visits a customer's table and responds (number of customer service) is a basic matter for increasing the revisit rate. Therefore, there is a need to objectively measure the number of customer visits in stores, and in recent years, services (businesses) have been developed to meet this need.

For example, there is a method in which a consultant disguised as a customer visits a store and visually counts the number of times the customer is served within a certain period of time. A smarter method is to have an analyst look at the data (video data) shot inside the store, tag the scenes, and count the number of scenes tagged as "customer service". Furthermore, there is known a human behavior determination device that determines the behavior of a person included in a video.

Patent No. 5285575

If the human behavior determination device is applied, it may be possible to automatically detect customer service behavior in a store. However, with the existing technology, it is necessary to manually design a feature quantity corresponding to customer service behavior, a dictionary corresponding to behavior conditions, and the like. For this reason, a large amount of work by specialists is required, and the cost is also increased. It is not easy to introduce the system, and it is also difficult to expand to other industries on a large scale (horizontal expansion).

Accordingly, it is an object of the present invention to provide a video processing device, a video processing method, and a program capable of counting the number of visits to customers without requiring design work.

According to an embodiment, a video processing device includes an analysis section, a counting section, and a storage section. The analysis unit analyzes the video data of the business scene of the store, and determines whether or not the customer service behavior of the employee occurs at each table for each image frame of the video data. The counting unit counts the number of times the analyzing unit determines that the customer service behavior is included as the customer service frequency. The storage unit stores the number of customer visits. The analysis unit includes a feature amount extraction unit and an action determination unit. The feature amount extraction unit extracts the feature amount from the video data for each image frame. The behavior determination unit determines whether or not the employee's customer service behavior occurs at each table based on the extracted feature amount.

FIG. 1 is a block diagram showing an example of a video processing device according to an embodiment. FIG. 2 is a functional block diagram showing an example of functions provided in the video processing device 1 of FIG. FIG. 3 is a diagram showing an example of a data table stored in the storage unit 6. As shown in FIG. FIG. 4 is a functional block diagram showing an example of functions of the analysis unit 11. As shown in FIG. FIG. 5 is a diagram showing an example of a neural network of the feature quantity extraction unit 113. As shown in FIG. FIG. 6 is a flow chart showing processing of the video processing device 1 . FIG. 7 is a flow chart showing the processing of the analysis unit 11. As shown in FIG. FIG. 8 is a diagram showing an example of an image frame cut out from video data. FIG. 9 is a diagram showing another example of an image frame cut out from video data. FIG. 10 is a conceptual diagram of a consulting system that focuses on the number of customer visits.

Embodiments will be described below with reference to the drawings. The drawings referred to are schematic. In the following description, elements having the same function and configuration are denoted by common reference numerals.

FIG. 1 is a block diagram showing an example of a video processing device according to an embodiment. The video processing apparatus 1 is a computer including a processor 2 , random access memory (RAM) 3 , read only memory (ROM) 4 and storage section 6 . The video processing device 1 further includes a display 5 , an I/O section 7 and an interface (I/F) section 8 .

The processor 2 controls the operation of the video processing device 1 as a whole. For example, the processor 2 executes a video processing program in response to a user's operation or a command from a host device (not shown). The processor 2 also manages the memory space of the RAM 3 and storage unit 6 .

The processor 2 implements various functions described in the embodiments by loading and executing programs stored in the storage unit 6 . The processor 2 is a CPU (Central Processing Unit), MPU (Micro Processing Unit), or an application specific integrated circuit (ASIC)), a programmable logic device (for example, a simple programmable logic device (Simple Programmable Logic Device (SPLD), Complex Programmable Logic Device (CPLD), and Field Programmable Gate Array (FPGA)). It is also possible to combine the CPU with a GPU (Graphics Processing Unit).

The RAM 3 stores programs and data loaded from the storage unit 6 . Also, the RAM 3 is used as a work area for the processor 2 . A semiconductor memory such as a DRAM is used as the RAM 3, for example.

The ROM 4 is a nonvolatile memory in which control programs, control data, and the like are stored in advance. The ROM 4 holds, for example, a BIOS (Basic Input/Output System).

The storage unit 6 stores video files used by the user for analysis and data generated by video processing. The storage unit 6 also holds various programs 6a used for video processing and data related to the programs. The program 6a held in the storage unit 6 is read out and developed in the RAM 3 when the video processing device 1 executes the program. As the storage unit 6, for example, an SSD (Solid State Drive) or a hard disk drive is used. It should be noted that the storage unit 6 only needs to be able to store data, and other recording media may be used as the storage unit 6 .

The display 5 displays, for example, a GUI (Graphical User Interface) corresponding to various programs under the control of the processor 2 . For example, the display 5 is used to reproduce the results of video processing.

The I/O unit 7 is a human-machine interface that mainly receives user operations, and is connected to a mouse 9, a keyboard, and the like.
The I/F unit 8 is an interface with, for example, a LAN (Local Area Network), and can connect the video processing apparatus 1 to an external network (such as the Internet), a cloud computing system, or the like via a hub (HUB) or the like. and

Note that the configuration shown in FIG. 1 is an example, and the video processing apparatus 1 may have a configuration different from this. For example, the video processing device 1 does not have to include the storage unit 6 and the display 5 . In this case, the storage unit 6 and the display 5 are externally connected to the video processing device 1 .

FIG. 2 is a functional block diagram showing an example of functions provided in the video processing device 1 of FIG. The video processing device 1 includes an analysis unit 11, a counting unit 12, and a tallying unit 13 as processing functions according to the embodiment. The analyzing unit 11, the counting unit 12, and the tallying unit 13 can be understood as processes generated when the program 6a in the storage unit 6 is executed. That is, the program 6 a includes instructions for causing the video processing apparatus 1 as a computer to function as the analyzing section 11 , the counting section 12 , the tallying section 13 and the storage section 6 .

The analysis unit 11 acquires and analyzes video data obtained by photographing business scenes of a store, and determines scenes including employee customer service behavior. That is, the analysis unit 11 determines whether or not the customer service behavior of the employee occurs at each table for each frame of the video data. The determination result is output as an inference probability (likelihood) indicated by a numerical value between 0 and 1, for example.

For example, if a fixed-point camera is installed in the store facing the customer's seats, it is possible to acquire video data capturing the customer service behavior of the employees. The video data may be acquired by real-time streaming, or may be downloaded after being stored in a network server. Alternatively, it may be recorded on a recording medium such as a DVD (Digital Versatile Disk) and given to the analysis unit 11 after the fact.

The counting unit 12 performs arithmetic processing based on the action determination result calculated by the analyzing unit 11, and converts the likelihood into the number of customer visits. For example, the calculation process is a process of taking a moving average of the output of the analysis unit 11, and counting as one reception if the state where the likelihood is equal to or greater than the threshold continues for a predetermined length of time or longer. In other words, the counting unit 12 counts the number of times that the analysis unit 11 determines that the customer service behavior is included. Then, the counting unit 12 stores the number of receptions for each customer in the storage unit 6 in association with auxiliary information (metadata) attached to the video data.
The tallying unit 13 tallies the output of the counting unit 12 and calculates the average number of visits to customers.

The storage unit 6 stores a program 6a for realizing the image processing function according to the embodiment in a format that the processor 2 can read and execute. In addition, the storage unit 6 stores the number of visits counted by the counting unit 12 and supplementary information related to the number of visits, for example, in the form of a data table.

FIG. 3 is a diagram showing an example of a data table stored in the storage unit 6. As shown in FIG. The data table is table format data in which video data is associated with supplementary information (metadata) such as store type (brand name), store name indicating which store the video was shot in, date, and time zone. Further, the average number of receptions counted by the counting unit 13 is associated with each video data (video data 1, 2, . . . ) and recorded.

FIG. 4 is a functional block diagram showing an example of functions of the analysis unit 11. As shown in FIG. The analysis unit 11 includes a frame acquisition unit 111, a preprocessing unit 112, a feature amount extraction unit 113, an action determination unit 114, a frame storage unit 115, and a model storage unit 116 as processing functions according to the embodiment.

The frame acquisition unit 111, the preprocessing unit 112, the feature amount extraction unit 113, and the behavior determination unit 114 use the resources of the cloud computing system when the program 6a in the storage unit 6 is executed. Interface).

When using the Web API from the program 6a, if an HTTP (HyperText aTransfer Protocol) request is sent to the cloud, the HTTP response is, for example, XML (Extensible Markup Language), HTML (HyperText Markup Language), JSON (JavaScript (registered trademark) Object Notation) and various image file formats. The program 6a includes instructions for executing a series of procedures related to this request-response. That is, the program 6a of the storage unit 6 includes commands for causing the video processing device 1 as a computer to function as the frame acquisition unit 111, commands for causing it to function as the preprocessing unit 112, and commands for causing it to function as the feature quantity extraction unit 113. and a command for functioning as the action determination unit 114 .

The frame acquisition unit 111 cuts out image frames of the input video data and stores them in the frame storage unit 115 as image frames 115a. The frame storage unit 115 stores the image frames 115a acquired by the frame acquisition unit 111 .
The preprocessing unit 112 performs preprocessing using the past image frames 115 a accumulated in the frame storage unit 115 and the image frames processed by the frame acquisition unit 111 . That is, the preprocessing unit 112 preprocesses the video data, calculates the optical flow and gradient between frames, and shapes the data as data corresponding to the input to the feature amount extraction unit 113 .

In the embodiment, it is considered that the feature quantity extraction unit 113 uses a neural network to extract the feature quantity of the image. Therefore, the preprocessing unit 112 generates multi-dimensional vector data including the calculated optical flow and gradient, and the image frame of the video data, and passes it to the feature extraction unit 113 . For example, if (RGB) three-dimensional data is given as an input image frame, two dimensions of the optical flow (XY axis direction) between frames and two dimensions of the gradient (XY axis direction), 7 dimensions in total x the number of pixels is input to the input layer of the neural network.

The feature quantity extraction unit 113 has a neural network having an input layer, an intermediate layer and an output layer. This neural network reflects the machine learning model 116 a stored in the model storage unit 116 . The machine learning model 116a is read into the feature quantity extraction unit 113 when calculating the feature quantity of the image frame, and is updated according to the machine learning algorithm when GT (Grand Truth) is given. The feature quantity extraction unit 113 obtains the feature quantity of the image frame by convoluting the vector data acquired from the preprocessing unit 112 based on the machine learning model 116a. That is, the feature amount corresponding to the current time of the input image frame is output from the output layer.

The behavior determination unit 114 determines whether or not an employee's customer service behavior occurs in each table at the current time based on the feature amount output from the feature amount extraction unit 113 . For example, it is possible to determine whether or not the target image frame includes the customer service behavior of the employee by threshold determination using the likelihood indicated by the feature amount.

FIG. 5 is a diagram showing an example of a neural network of the feature quantity extraction unit 113. As shown in FIG. In FIG. 5, an input image frame (7-dimensional preprocessed image frame including RGB, optical flow between frames (XY directions), and gradients (XY directions) is input to the input layer 41. Further, The input layer 41 may be supplied with not only the current frame, but also an image frame (1-frame previous image, 2-frame previous image, .

The convolutional feature quantity extraction layer 42 including intermediate layers is composed of one or more neural networks, and extracts an image spatially (XY pixel direction) or spatially and temporally (three-dimensional direction: XY pixel direction + time direction). convolve the Spatial convolution and temporal convolution may be implemented in separate neural networks. Furthermore, a structure may be employed in which arithmetic processing is performed on the outputs of a plurality of neural networks.

The time-series information storage layer 43 including the output layer is configured by a recursive neural network, and its internal variables hold the internal state vectors of the previous time. A vector (behavior determination output) giving a behavior determination result is output by calculation using the feature amount vector output by the convolutional feature amount extraction layer 42 and the retained internal state vector. At the same time, an internal state vector corresponding to the current time is calculated, and this internal state vector is held until the operation at the next time.
Next, a plurality of embodiments will be described based on the above configuration.

(First embodiment)
FIG. 6 is a flow chart showing processing of the video processing device 1 . In FIG. 6, the video processing device 1 inputs video data to the analysis unit 11 (step S21). The analysis unit 11 analyzes the input video data, digitizes the action determination result for each frame, and outputs the result (step S22). That is, the analysis unit 11 determines a scene (or frame) including the employee's customer service behavior, and outputs the result (step S22).

Next, based on the output from the analysis unit 11, the counting unit 12 counts the number of times (the number of receptions) determined to include the reception behavior (step S23). The number of visits to customers may be output as an average value for each customer. The number of receptions output by the counting unit 12 is stored in the storage unit 6 in association with the auxiliary information of the video data (step S24).

The procedure from step S21 to step S24 is repeated for all the video data (step S25), and when the processing up to the last video data is completed, the counting unit 13 stores the auxiliary information of the video data stored in the storage unit 6. and the number of visits to customers are counted (step S26).

FIG. 7 is a flow chart showing the processing of the analysis unit 11. As shown in FIG. The processing procedure shown in FIG. 7 mainly corresponds to step S22 in FIG.
In FIG. 7, the feature quantity extraction unit 113 reads the machine learning model 116a from the model storage unit 116 into the storage area of the RAM 3 (step S31). Further, the frame acquisition unit 111 extracts one frame from the image frames 115a (step S32), and stores the history of the time-series image frames together with the time information (time stamp) in the frame storage unit 115 (step S33).

Next, the preprocessing unit 112 acquires a preset number of image frames from the frame storage unit 115 and calculates the optical flow and gradient between frames. Then, the calculated optical flow, gradient, and image frame are put together and converted into a format that can be input to the neural network of the feature amount extraction unit 113 to generate vector data (step S34).

Next, the feature amount extraction unit 113 inputs the vector data given from the preprocessing unit 112 to the neural network, and extracts feature amounts related to customer service behavior (step S35). The behavior determination unit 114 performs arithmetic processing on the feature amount output from the feature amount extraction unit 113, and outputs a determination result as to whether or not the employee's customer service behavior occurs for each table (step S36).

Next, the analysis unit 11 determines whether or not the input image frame (or at the time corresponding to the image frame) is provided with customer service behavior learning tag information (step S37). The customer service behavior learning tag information is so-called GT (Grand Truth), and in the first embodiment, it is tag information given by a person while referring to video data.

If customer service behavior learning tag information is attached to the image frame (YES in step S37), the analysis unit 11 updates the machine learning model (step S38). For example, a loss function can be obtained from the difference between the prediction result (likelihood) obtained from the neural network and the customer service behavior learning tag information, and the machine learning model can be updated by the back propagation method. The updated machine learning model is stored in the model storage unit 116 (step S38). Then, the procedure of steps S31 to S38 is repeated until the end point of the video data is reached (step S39).

FIG. 8 is a diagram showing an example of an image frame cut out from video data. When the frame shown in FIG. 8(a) appears, when a human (analyst) judges that "this frame indicates (customer service)", for example, by clicking the mouse 9 (FIG. 1), the customer action is performed. Learning tag information (GT) is provided. Then, the machine learning model is updated, and when the image frame shown in FIG. 8(b) appears, for example, a high score such as 0.8 is given as the likelihood of this frame. For example, if the threshold value is 0.5, the frame in FIG. 9A is counted as "customer service".

On the other hand, the likelihood of the scene shown in FIG. 9A may be 0.1, and the likelihood of the scene shown in FIG. 9B may be 0.2. In this case, none of the frames are counted as customers. Then, the number of times GT is given is increased, and the machine learning model is updated with the passage of time, increasing the determination probability of the "customer service" behavior.

The important point is that in the above process, there was no need to manually design the feature quantity corresponding to the customer service behavior or the dictionary corresponding to the behavior conditions. That is, according to the image processing apparatus 1 according to the first embodiment, the customer service behavior in the store is determined by arithmetic processing on the machine side based on the given GT, and the accuracy is improved.

As described above, in the video processing apparatus 1 according to the first embodiment, an image frame, an optical flow between frames, a 7-dimensional image of a gradient, or its time-series data are input vectors, and a convolutional neural network and a recursive neural network are used. Deep learning using a neural network can realize a recognition method that counts the number of customer visits from video data.

In other words, it is possible to realize learning that takes into account time-series elements such as the color of clothes that identify employees and the movements of customers by inputting seven dimensions. In addition, it is possible to eliminate the disadvantage of the existing technology, which is that extracting the feature amount of customer service and creating a dictionary requires the labor and cost of an expert. In other words, by estimating three-dimensional movements that take into account the employee's state and chronological movements, and adding tag information that indicates whether or not customer service behavior occurs to the learning data of deep learning, customer service behavior can be performed. It is possible to tally the number of visits to customers without manually designing the feature values and dictionaries. In addition, it can be used for consulting to improve the store revisit rate.

For these reasons, according to the first embodiment, it is possible to provide a video processing device, a video processing method, and a program capable of counting the number of visits to customers without requiring complicated design work.

(Second embodiment)
In the second embodiment, a technique for utilizing the automatically counted number of visits to customers for consulting for improving the store revisit rate will be described.

FIG. 10 is a conceptual diagram of a consulting system focusing on the number of visits to customers. Calculate aggregated data, etc. Depending on the scale of the company that operates chain stores, the number of stores can reach several hundred, and the amount of video data obtained from the image sensors installed in each store is enormous. Such data is provided to the video processing device 1 as so-called big data. The results obtained by the big data analysis of the video processing device 1 are passed to a server owned by a consulting company, for example, and used for analysis of various data related to store management.

The second embodiment focuses on an index called KPI (Key Performance Indicator). KPI is an index that has long been known in the field of statistics, and in this embodiment, the KPI obtained by statistically processing the counted number of customer visits is used to provide useful knowledge for store management. think of getting

The server 100 statistically processes the number of visits to each store, total data, etc. obtained from a large amount of video data, and quantifies indices such as the number of visits and the revisit rate as KPIs. Then, it analyzes the relationship between the number of customer visits and an index such as sales for each store. Through such big data analysis, for example, it is possible to obtain knowledge such as ``a decrease in the number of customer visits leads to a decrease in the revisit rate, which appears as a sign of sluggish sales several months later''.

With conventional technology, attention is focused only on data related to accounting that can be easily obtained, such as sales, number of customers, and number of products obtained from POS registers, such as "sales by store", and customer service is It was difficult to incorporate indicators that are difficult to evaluate objectively, such as the "number of customer visits", such as whether the customer was attentive, and changes in customer behavior, such as the rate of repeat visits. For this reason, management decisions are made based on so-called sensory reports from the field, short-term management analysis based on the field approach, and subjective judgments based on past empirical rules, and management efforts do not necessarily lead to results. There was a difficult side.

On the other hand, in the second embodiment, the "number of customer visits" is quantified as a KPI, and the relationship with other indexes (sales, etc.) is found through big data analysis. Therefore, according to the second embodiment, it is possible to provide consulting for store management based on objective indicators, instead of the conventional store management based on reports that rely on intuition, thereby improving store operations. be able to help.

In addition, this invention is not limited to the said embodiment.
For example, in FIG. 2, the program 6a used for video processing is stored in the storage unit 6, but the present invention is not limited to this. For example, the program executed by the video processing device 1 may be held in a server (not shown) on the network. In this case, when the video processing device 1 executes video processing, various programs are distributed to the video processing device 1 from a server on the network. The video processing apparatus 1 receives various programs, develops these programs in the RAM 3 (FIG. 1), and executes video processing.

Further, in the embodiment, the likelihood of customer service behavior is calculated for each image frame using a neural network, and the presence or absence of customer service behavior is determined by threshold determination. Alternatively, a tag (flag data) meaning "hospitality behavior" may be added directly to the frame indicating the likelihood equal to or higher than the threshold, and the number of tags may be counted to obtain the same effect as above. can. In order to realize this, the analysis unit 11 is provided with a function of analyzing a video file containing the business scene of the store and outputting a tagged video file in which the scene containing the customer service behavior of the employee is tagged, The counting section 12 may be provided with a function of counting the number of tags in the tagged video file, and the storage section 6 may store the counted number of tags.

According to such a configuration, it becomes possible to automatically generate video data (tagged video file) to which the "hospitality behavior" tag is attached, and it is possible to obtain merits such as being able to distribute to customers at the time of consulting. .

Also, in the description of FIG. 5, a mode in which the current frame and the previous frames are input to the convolutional feature extraction layer 42 is shown, but the current frame, the previous frames, and the Of course, it is also possible to enter subsequent frames. It should be noted that the current frame is a frame to be judged as to whether or not "customer service behavior" is included, and does not mean the frame at the current point in time.

When a recorded video is input as video data, for example, time series of periods before and after the current frame, such as "..., 2 frames before, 1 frame before, current frame, 1 frame after, 2 frames after,...". Data can be input to a neural network. In this case, the time-series information storage layer 43 holds the internal state vectors of both the previous time and the next time, and transmits the internal state vector of the current time to the previous and subsequent times. In other words, it is possible to realize a calculation process of "thinking about how it has been moving so far and how it will move in the future to serve customers".

That is, when video data recorded as media data is used, calculations from the input layer 41 to the convolutional feature amount extraction layer 42 can be performed simultaneously for all times in principle. Then, in the time-series information storage layer 43, a processing procedure may be executed in which changes in the internal state vectors are calculated separately in order from the past and the future, and when they join at the current time, they are calculated.

Furthermore, in the embodiments, convolutional neural networks and recurrent neural networks have been described, but neural networks such as recurrent neural networks, deep belief networks, deep Boltzmann machines, and layered autoencoders can also be applied. .

A program for realizing each device and system described above is recorded in a computer-readable recording medium, and the program recorded in this recording medium is read into a computer system and executed, thereby performing execution processing. may Note that the "computer system" is not limited to hardware such as an OS and peripheral devices, and may include communication networks and cloud computing systems.

While embodiments of the invention have been described, the embodiments are provided by way of example and are not intended to limit the scope of the invention. This novel embodiment can be embodied in various other forms, and various omissions, replacements, and modifications can be made without departing from the scope of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are included in the scope of the invention described in the claims and its equivalents.

Some or all of the above embodiments can also be described as (Appendix 1) and (Appendix 2), for example. However, it is not limited to the contents of (Appendix 1) and (Appendix 2). (Appendix 1)
Equipped with a processor capable of analyzing video data and a memory,
The processor
Analyze the video data of the business scene of the store, determine the scene including the customer service behavior of the employee, count the number of times it is determined that the customer service behavior is included,
A video processing device that stores the number of times in the memory.

(Appendix 2)
an analysis unit that analyzes a video file containing business scenes of a store and outputs a tagged video file in which scenes containing customer service behavior of employees are tagged;
a counting unit for counting the number of tags in the tagged video file;
and a storage unit that stores the counted number of tags.

DESCRIPTION OF SYMBOLS 1... Video processing apparatus, 2... Processor, 3... RAM, 4... ROM, 5... Display, 6... Storage part, 6a... Program, 7... I/O part, 8... I/F part, 9... Mouse, 11 Analyzing unit 12 Counting unit 13 Aggregating unit 41 Input layer 42 Convolutional feature amount extraction layer 43 Time-series information storage layer 111 Frame acquisition unit 112 Preprocessing unit 113 Features Quantity extraction unit 114 Action determination unit 115 Frame storage unit 115a Image frame 116 Model storage unit 116a Machine learning model 100 Server.

Claims (13)

an analysis unit that analyzes video data of a business scene of a store and determines whether or not an employee's customer service behavior occurs in each table for each image frame of the video data;
a counting unit that counts the number of times the analysis unit determines that the service behavior is included as the number of receptions;
A storage unit that stores the number of receptions,
The analysis unit is
a feature quantity extraction unit that extracts a feature quantity from the video data in units of image frames;
A video processing device, comprising: an action determination unit that determines whether or not the customer service action of the employee occurs in each of the tables, based on the extracted feature amount. 2. The image according to claim 1, wherein the behavior determination unit determines whether or not the target image frame includes customer service behavior of the employee by threshold determination using the likelihood indicated by the feature amount. processing equipment. The feature quantity extraction unit is
3. The neural network according to claim 2, comprising an input layer for inputting vector data based on said video data, an output layer for outputting said feature quantity, and obtaining said feature quantity by a convolution operation based on a machine learning model. video processing equipment. The analysis unit is
A preprocessing unit that preprocesses the video data to generate multi-dimensional vector data including the image frames, an optical flow between the image frames, and a gradient between the image frames, and inputs the data to the input layer. 4. The video processing device of claim 3, further comprising: 5. The video processing device according to claim 4, wherein the preprocessing unit preprocesses the video data to generate time-series data of the vector data and inputs the data to the input layer. 6. The video processing device according to claim 3, wherein said neural network includes a convolutional neural network and a recurrent neural network. an analysis process in which a computer analyzes video data of business scenes of a store and determines whether or not an employee's customer service behavior occurs at each table for each image frame of the video data;
a step in which the computer counts the number of times the customer service action is determined to be included in the analysis process as the number of customer service times;
a step in which the computer stores the number of receptions,
The analysis process is
a feature amount extraction process in which the computer extracts feature amounts from the video data in image frame units;
and an action determination step in which the computer determines whether or not the customer service action of the employee occurs in each of the tables based on the extracted feature amount. 8. The computer determines whether or not the target image frame includes the customer service behavior of the employee in the behavior determination process by threshold determination using the likelihood indicated by the feature amount. The video processing method described in . The computer comprises an input layer for inputting vector data based on the video data and an output layer for outputting the feature amount in the feature amount extraction process, and extracts the feature amount by a convolution operation based on a machine learning model. 9. The video processing method according to claim 8, wherein said feature amount is extracted by a neural network obtained. The analysis process is
The computer preprocesses the video data to generate multi-dimensional vector data including the image frames, optical flow between the image frames, and gradients between the image frames, and inputs the data to the input layer. 10. The image processing method of claim 9, further comprising a pre-processing step of: 11. The video processing method according to claim 10, wherein in said preprocessing step, said computer preprocesses said video data to generate time-series data of said vector data, and inputs said data to said input layer. 12. The video processing method according to any one of claims 9 to 11, wherein said neural network includes a convolutional neural network and a recurrent neural network. the computer,
a command for functioning as an analysis unit that analyzes video data of a business scene of a store and determines whether or not an employee's customer service behavior occurs at each table for each image frame of the video data;
a command for functioning as a counting unit that counts the number of times the analysis unit determines that the service action is included as the number of receptions;
a command for functioning as a storage unit that stores the number of receptions;
a command for functioning as a feature quantity extraction unit that extracts a feature quantity from the video data in units of image frames;
and a command for functioning as an action determination unit that determines whether or not the customer service action of the employee occurs in each of the tables, based on the extracted feature amount.

Images