TW201918934A - Intelligent image information and big data analysis system and method using deep learning technology by integrating video monitoring and image deep learning technology to provide acial expression recognition information and motion recognition information - Google Patents

Abstract

The invention discloses an intelligent image information and big data analysis system and method using deep learning technology, which includes an image capturing device, an information processing device, and a cloud image recognition unit. The image capturing device is used to capture the image of an object, and the object image at least comprises a human image. The information processing device is signally connected with the image capturing device through the signal transmission module so as to collect the received human images. The cloud image identification unit is signally connected with the information processing device through the network transmission module so as to receive the human image. A deep learning algorithm module built in the cloud image identification unit is used to perform image recognition. The deep learning algorithm module can perform image recognition of facial expressions and motions on human images, so as to obtain at least one facial expression recognition information and motion recognition information, so that the identification cloud service of facial expression recognition and motion recognition can be provided by integrating the video monitoring and image deep learning technology.

Description

 Smart image information and big data analysis system using deep learning technology  

The invention relates to a smart image information and big data analysis system and method using deep learning technology, in particular to an intelligent cloud service capable of providing facial expression and limb motion recognition by integrating video monitoring and image deep learning technology. Image big data analysis technology.

In recent years, with the breakthrough of artificial intelligence technology, the recognition success rate of biometrics including face detection recognition, voiceprint recognition, iris comparison and fingerprint identification has reached the level of commercial application. In the technical field of biometrics, especially the face recognition technology, since it belongs to non-contact identification, it has been increasing the importance of the relevant technical field. Furthermore, according to the patent search, the representative patents with face recognition function are as follows:

1. Invention Patent No. 201317903 "Face Recognition Monitoring Management Method" patent, which discloses a face recognition function and a text mark to provide monitoring management, and a monitoring tool for solving a conventional technique is prone to fatigue or visual fatigue. .

2. The new patent No. M432892 "Face Detection and Identification Device" discloses a face that uses a selection module, a filter module, a block module and a calibration module to simplify the process and reduce the amount of calculation. Identification device.

3. The invention patent No. 201220211 "High-definition image-based face recognition and monitoring system" is composed of an image capturing module, a face recognition module, a data processing module and a search module to provide an oversized The scene can identify multiple people at the same time and has a person search mechanism and a monitoring system for event restoration.

It is known from the above that although the above patent can realize the recognition function of the face; however, in practical application, the face recognition is more susceptible to the influence of light, brightness, facial expression changes and the like, thereby increasing the difficulty of image recognition, This problem is to greatly improve the success rate of image recognition. At this stage, the relevant technical field is to use image deep learning to improve this problem. As for the representative patent using image deep learning, the new type M488698 "Smart" The image-based customer analysis system and the new M443899 "active identification of the customer-level intelligent point of sale (POS) system" and other patents are shown. Among them, the No. M488698 patent can calculate the customer layer property bagging rate analysis data, provide the store with a clear understanding of the current business results, as a follow-up business model and marketing strategy; and the M443899 patent can be based on the identified age layer, gender, Search for the corresponding suitable advertisements in the shelf products, instantly play and record the time stamps, identify the customer layer and the advertisement attention degree, and then return the relevant data through the transmission management for subsequent analysis.

Although these patents can be applied to general stores to achieve the above-mentioned functions, these patents cannot be applied to places such as restaurants, parking lots, companies, homes, and access control areas due to technical building factors, so that they cannot be used in the above places. The effective use of further image recognition has detracted from the competitiveness and value of the business. Therefore, these patents have not been perfected and there is still a need for further improvement.

In view of this, there is no patent or paper that integrates intelligent video surveillance and image deep learning technology to provide smart city image recognition cloud service, and based on the urgent needs of related industries, the creators of the present invention continue to Under the arduous efforts of research and development, a set of inventions different from the above-mentioned prior art and patents has finally been developed.

A first object of the present invention is to provide a smart image information and big data analysis system and method using deep learning technology, which mainly provides a restaurant, a department store, a home security, a parking lot, by integrating video monitoring and image deep learning technology. Smart city image recognition cloud service for facial expression recognition and body motion recognition in film and television advertisements and corporate schools. The technical means adopted for achieving the first object of the present invention include an image capturing device, an information processing device, and a cloud image recognition unit. The image capturing device is used to capture an image of an object, and the image of the object includes at least a human image. The information processing device is connected to each image capturing device signal through the signal transmission module to collect the recipient image. The cloud image recognition unit is connected to the information processing device through the network transmission module to receive the human image. The cloud image recognition unit has a deep learning calculation module for performing image recognition, and the deep learning calculation module can face the human image. Image recognition of facial expressions and body movements to obtain at least one facial expression recognition information and body motion recognition information.

A second object of the present invention is to provide a big data analysis system that can avoid intelligent video information that affects physical and mental health by improperly broadcasting live images, such as bleeding, violence, and pornography. The technical means for achieving the second object of the present invention include an image capturing device, an information processing device, and a cloud image recognition unit. The image capturing device is used to capture an image of an object, and the image of the object includes at least a human image. The information processing device is connected to each image capturing device signal through the signal transmission module to collect the recipient image. The cloud image recognition unit is connected to the information processing device through the network transmission module to receive the human image. The cloud image recognition unit has a deep learning calculation module for performing image recognition, and the deep learning calculation module can face the human image. Image recognition of facial expressions and body movements to obtain at least one facial expression recognition information and body motion recognition information. The image of the object includes at least one live image, and the cloud image recognition unit receives the live image through the network transmission module and the information processing device, and the deep learning algorithm module can perform violence and blood smash on the live image. And the image recognition of the erotic image, when the deep learning calculation module detects that the live image has one of the image content such as violence, bloodyness or pornography, the playback of the live image is interrupted; or the image processing of the mosaic is performed.

A third object of the present invention is to provide a big data analysis system with intelligent image information for advertising trademark identification and playing time and frequency statistics for commercial application analysis. The technical means adopted for achieving the first object of the present invention include an image capturing device, an information processing device, and a cloud image recognition unit. The image capturing device is used to capture an image of an object, and the image of the object includes at least a human image. The information processing device is connected to each image capturing device signal through the signal transmission module to collect the recipient image. The cloud image recognition unit is connected to the information processing device through the network transmission module to receive the human image. The cloud image recognition unit has a deep learning calculation module for performing image recognition, and the deep learning calculation module can face the human image. Image recognition of facial expressions and body movements to obtain at least one facial expression recognition information and body motion recognition information. The image of the object includes at least one video image, and the cloud image recognition unit further comprises a trademark feature database with a plurality of trademark sample images, and a trademark feature data and a The trademark identification data corresponding to the trademark feature data, the cloud image recognition unit receives the movie image through the network transmission module and the information processing device, and extracts the feature of the movie image into at least one trademark feature image, and then The trademark feature database identifies the trademark feature data that matches the feature of the trademark feature image, and reads the trademark identification data that the feature meets, and then outputs the corresponding trademark identification information, so that the deep learning calculation module can The film and television image is identified by the advertisement trademark, and the number of times and the playing time of the advertisement trademark appearing in the film and television image are counted, and the number of times of the advertisement trademark and the statistical information of the playing time are output.

1‧‧‧ places

10‧‧‧Image capture device

20‧‧‧Information processing device

21‧‧‧Signal transmission module

30‧‧‧Cloud Image Identification Unit

31‧‧‧Network Transmission Module

32‧‧‧Deep learning calculus module

320‧‧‧Deep learning model

33‧‧‧Statistical Analysis Unit

34‧‧‧Image Feature Database

40‧‧‧Electronic board

41‧‧‧Access control equipment

50‧‧ ‧ people image

51‧‧‧Face

60‧‧‧ elbow

61‧‧‧ knee

1 is a schematic diagram of the implementation of a specific architecture of the present invention.

2 is a schematic diagram of the implementation of the training phase of the deep learning model of the present invention.

3 is a schematic diagram of the implementation of the operational prediction phase of the deep learning model of the present invention.

FIG. 4 is a schematic diagram of an implementation of extracting facial expression features from a human image in the present invention.

FIG. 5 is a schematic view showing the implementation of the limb movement feature extracted from the human image of the present invention.

Fig. 6 is a schematic view showing the implementation of the present invention applied to a restaurant.

Fig. 7 is a schematic view showing the implementation of the present invention applied to a department store.

Fig. 8 is a schematic view showing the implementation of the present invention applied to home security.

Figure 9 is a schematic view showing the implementation of the present invention applied to a parking lot.

In order to allow the reviewing committee to further understand the technical features of the present invention and the technical means for achieving the object of the present invention, it will be described in detail by way of specific embodiments and drawings: please refer to FIGS. 1 to 3 to achieve the present invention. The specific embodiments of the first object include technical features such as the image capturing device 10, the information processing device 20, and the cloud image recognition unit 30. The plurality of image capturing devices 10 are respectively disposed in at least one location 1 for capturing an object of the location 1 to generate at least one object image, and the object image includes at least a human image. The information processing device 20 is coupled to each of the image capturing devices 10 via a signal transmission module 21 (such as a Bluetooth, USB, RS232, etc. transmission module) for collecting recipient images and other object images. The cloud image recognition unit 30 (such as a cloud server) is connected to the information processing device 20 through a network transmission module 31 (such as a combination of a network and a data machine or a router) for receiving images of people and other objects. The cloud image recognition unit 30 has a deep learning algorithm module 32 with a deep learning function to perform image recognition. The deep learning algorithm module 32 can perform facial image recognition and body motion image recognition on the human image to obtain at least A facial expression recognition information and body motion recognition information for subsequent commercial use.

Please refer to the location 1 shown in Figure 6~9 for restaurant, department store, store, home, building, parking lot, school, company or access control equipment.

As shown in FIG. 1 to FIG. 4, the cloud image recognition unit 30 further includes an image feature database 34 having a plurality of sample images embedded therein. The depth learning algorithm module 32 extracts features of the human sample image to include a face. And the facial expression feature data is set, and a corresponding facial expression recognition data is set in each facial expression feature data. When the deep learning calculation module 32 receives the at least one human image 50 captured in the instant, the image feature database 34 is displayed. The facial expression feature data corresponding to the characteristics of the sample image of the person is identified, and the facial expression recognition data corresponding to the feature is read, and the corresponding facial expression recognition information is output.

Specifically, the facial expression recognition information may be at least two expressions that are slightly happy, happy, extremely happy, slightly sad, sad, extremely sad, slightly unpleasant, unpleasant, or extremely unpleasant. It can be a facial expression of joy, nos, sorrow, and joy, but it can also be a nervous, frightening facial expression.

As described above, the above embodiment can be applied to a place 1 such as a restaurant or a department store, and the image capturing device 10 is installed near the electronic board 40 to detect whether or not the electronic device is viewed by the customer who views the electronic board 40. Kanban 40, according to which analysis of which kind of advertising is more attractive to what kind of consumer group, and to carry out deep learning calculations including face and facial expressions, and then get the customer's face 51 expression? For example, a smile means satisfaction with the provision of goods or services, as shown in Figure 4; on the contrary, facial expressions, disappointment, and dissatisfaction indicate dissatisfaction or extreme dissatisfaction with the provision of goods or services, The advertisement image or movie suitable for the customer group can be played on the electronic board 40 to achieve the purpose of accurate marketing of the product. Among them, it is more suitable to recognize the movement information of the limbs, such as the nod movement of the limbs, indicating that it is satisfied with the provision of goods or services; on the contrary, the movement of the limbs of the head indicates that it is unsatisfied or extremely dissatisfied with the provision of goods or services.

Specifically, the cloud image recognition unit 30 further includes an image feature database 34 (such as a data server; but not limited to) a built-in human sample image, and the deep learning algorithm module 32 images the human sample. The feature capture includes a limb motion feature data, and a corresponding limb motion recognition data is set in each limb motion feature data. When the deep learning calculation module 32 receives a continuous number of consecutive human images, Then, the image feature database 34 identifies the limb motion feature data that matches the characteristics of the human sample image, and reads the limb motion recognition data that the feature meets, and then outputs the corresponding limb motion recognition information.

Further, the above-mentioned limb motion recognition information may include abnormal limb motion recognition information, such as an abnormal limb motion recognition information of climbing (for a predetermined time (for example, within five seconds), the elbow portion is higher than the arm thereof. When the height of one half of the original elbow and the knee is higher than the height of one third of the original knee when the thigh is extended downward, as shown in Fig. 5c, it is determined that the abnormal limb movement of the climb, Abnormal physical movement recognition information (such as abnormal hand movements that are determined to be stolen by inserting your hand into someone's pocket or purse), and abnormal physical movement recognition information of the robbing (for example, within a predetermined time (for example, one minute) Inside), after approaching at least one other person's distance of fifty centimeters, quickly lift the leg or raise the hand and move away from the other person by at least five meters within five seconds to determine the abnormal limb movement for robbing) Abnormal physical movement recognition information of the fight (such as at least two people close to fifty centimeters, and within a predetermined time (for example, within two seconds) to raise their legs or raise their hands, etc. , or holding a weapon (such as a knife, a stick) and doing a limb movement such as lifting a leg or raising a hand, can be judged as an abnormal limb movement of the fight, and an abnormal physical movement recognition information of violence (such as one person's body is close to another person fifty centimeters) Within a predetermined time (for example, within two seconds), doing a limb movement such as raising a leg or raising a hand, or holding a weapon (such as a knife or a stick) and doing a limb movement such as lifting a leg or raising a hand, An abnormal physical movement recognition that is determined to be violent, and an abnormal physical movement recognition information (such as a person showing a genital organ or a person who is compulsively hugged by others and a rebellious action by another person, can be judged as an abnormal physical movement of the sputum), pornography Abnormal physical movement recognition information (such as at least one person doing a genital or at least two naked to do a hug, you can determine the abnormal physical movements of erotic), roaring abnormal physical movement identification information (such as making angry expressions and The mouth is in a pre-part time (for example, within one second) to quickly open and raise the hand and other physical movements, which can be determined as an abnormal limb movement of the roar or an abnormal limb that cannot be recumbent Information for identification (e.g., lying on the floor of the body for too long more than a predetermined time (e.g. two minutes), to afford the abnormality is determined lying body movements), and other identifying information abnormal body movements.

As described above, the above embodiment can be applied to a place 1 such as a restaurant, a department store, or a home building, and the image capturing device 10 is installed in a restaurant, a department store, or a wall of a home building. Monitor whether the customer has made physical movements such as climbing, stealing, snatching, fighting, violence, swearing, roaring or lying down; or monitoring whether the home building has a small climb and climbs over the wall. The limb movement, after the deep learning calculation and identification, finds that the above abnormal limb movement, it sends a warning signal, and makes an emergency response related alarm processing.

As shown in Fig. 5a~c, in sequence, there are small movements that climb over the wall, the gangsters hold the knives to hurt people, and the old man can't afford to move.

Referring to FIG. 1 to FIG. 3, the specific embodiments for achieving the second object of the present invention include technical features such as the image capturing device 10, the information processing device 20, and the cloud image recognition unit 30. The plurality of image capturing devices 10 are respectively disposed in at least one location 1 for capturing an object of the location 1 to generate at least one object image, and the object image includes at least one human image. The information processing device 20 is connected to the image capturing device 10 via a signal transmission module 21 (such as a Bluetooth, USB, RS232, network, analog communication device, etc.) to collect the recipient image. The cloud image recognition unit 30 (such as a cloud server) is connected to the information processing device 20 through a network transmission module 31 (such as a combination of a network and a data machine or a router, and adopts an HLS or RTMP communication method). In the cloud image recognition unit 30, the cloud image recognition unit 30 has a deep learning algorithm module 32 with a deep learning function to perform image recognition. The depth learning algorithm module 32 can perform facial expression, age, gender, and emotion on the human image. Image recognition of wearing, and body movements to obtain at least one facial expression recognition information and body motion recognition information for subsequent commercial use. The image of the object includes at least one live image (such as a live video image played from an electronic billboard, a mobile phone, or a computer), and the cloud image recognition unit 30 receives the live image through the network transmission module 31 and the information processing device 20. The deep learning calculus module 32 can perform violent, bloody and erotic image recognition on the live image. When the deep learning calculus module 32 detects that the live image has one of the violent, bloody or pornographic images, the interruption is interrupted. Play live video (such as turning off the TV or computer through hardware control; or electronic signage); or do mosaic image processing.

Referring to FIG. 1 to FIG. 3, the specific embodiments for achieving the third object of the present invention include technical features such as the image capturing device 10, the information processing device 20, and the cloud image recognition unit 30. The plurality of image capturing devices 10 are respectively disposed in at least one location 1 for capturing an object of the location 1 to generate at least one object image, and the object image includes at least one human image. The information processing device 20 is coupled to each of the image capturing devices 10 via a signal transmission module 21 (such as a Bluetooth, USB, RS232, etc. transmission module) for collecting the recipient images. The cloud image recognition unit 30 (such as a cloud server) is connected to the information processing device 20 through a network transmission module 31 (such as a combination of a network and a data machine or a router) for receiving human images and cloud images. The recognition unit 30 has a deep learning algorithm module 32 with a deep learning function to perform image recognition. The deep learning algorithm module 32 can perform facial image recognition and body motion image recognition on a human image to obtain at least one available. Subsequent commercial use of facial expression recognition information and body motion recognition information. The image of the object includes at least one video image (such as a video image played by an electronic billboard or a computer), and the cloud image recognition unit 30 further includes an image feature database 34 having a plurality of trademark sample images, and A trademark feature data and a trademark identification data corresponding to the trademark feature data are set in each of the trademark sample images, and the cloud image recognition unit 30 receives the video image through the network transmission module 31 and the information processing device 20, and the video image is The feature capture includes at least one trademark feature image, and the image feature database 34 identifies the trademark feature data that matches the feature of the trademark feature image, and reads the trademark identification data that the feature meets, and then outputs the corresponding trademark identification information. The deep learning calculus module 32 can identify the advertisement trademark of the film and television image, and count the number of times and the playing time of the advertisement trademark appearing in the film and television image, and then output the number of times of the advertisement trademark and the statistical information of the playing time.

In addition, referring to the embodiment shown in FIG. 1, the present invention further includes a statistical analysis unit 33, which may be a data processing server or a software built in the cloud image recognition unit 30. The module is configured to receive, count, and analyze other image recognition information performed by the deep learning calculus module 32, such as person flow identification, traffic identification, customer identification, face recognition, gender recognition, age recognition, Meal identification, license plate recognition, vehicle identification, limb movement behavior recognition, parking status recognition, product identification, live image recognition, film and television advertisement recognition, character recognition, trademark recognition; or identification information such as commuting. Specifically, the statistical analysis unit 33 may perform a statistical/analysis information for the image identification information, and the statistical/analysis information may be a human flow statistics, gender statistics, age statistics, hotspot analysis, customer layer analysis, visitor statistics, and customer dynamics. One of the information such as line analysis, electronic billboard analysis, traffic statistics, and parking space management analysis.

More specifically, the deep learning calculus module 32 (ie, the deep learning algorithm) used in the present invention may be a convolutional neural network (CNN) algorithm, an expert system algorithm, or a random forest algorithm. Wisdom algorithm. When the deep learning calculus module 32 executes, the following steps are included:

(a) The training phase step, as shown in FIG. 2, is to first establish at least one deep learning model 320, and input a huge amount of object sample images and image recognition parameters (such as features for identification or comparison) in the deep learning model 320. Data), and the depth learning model 320 tests the correct rate of image recognition, and then determines whether the image recognition correct rate is sufficient. When the judgment result is yes, the identification result is output and stored; when the judgment result is no, the depth learning model is made. 320 implements self-correction learning by adjusting image recognition parameters or other means.

(b) running the prediction phase step, as shown in FIG. 3, inputting the image of the object captured in the deep learning model 320, and performing predictive image recognition by the deep learning model 320 to obtain at least one facial expression recognition information. And body movement identification information.

Specifically, the deep learning model 320 includes a plurality of classifiers of different attributes (such as a human profile, a face profile, a hair profile, an expression profile, a meal profile, a vehicle profile, a license plate profile, a car profile, and a trademark profile). And the deep learning calculus module 32 further includes a feature image capturing step, wherein the local feature of the sample visual image is captured as the object sample image, and the object sample image image is input into the deep learning model 320, The comparison is performed by a classifier of the same attribute to output the comparison result to the depth learning model 320 as a basis for judging the output identification information.

Please refer to FIG. 1 and FIG. 6 for the application example of the invention applied to the restaurant place 1, mainly for the customer group, preferences, loyalty, etc., through the actual image identification analysis, assisting the industry to formulate appropriate commercial countermeasures and improve The quality and warmth of the internal and external service. The specific services include the following:

1. Visitor statistics service, which installs an image capturing device 10 near the entrance and exit of the restaurant to capture the visual image of the flow of people entering and leaving the entrance and exit, and image recognition of the human silhouette of the human flow image. Then, the flow identification information is obtained, and the number of customers entering and exiting is counted by the statistical analysis unit 33 to obtain the flow statistics information. Here, the basis for the subsequent processing of the catering business analysis business trend and employee scheduling optimization can be provided.

2. The electronic kanban push service is installed in the vicinity of the electronic kanban 40 to display the deep learning calculations including the face, gender and age profile of the customer who views the electronic kanban 40, thereby obtaining The identification information of the customer group is analyzed by the statistical analysis unit 33 to analyze the customer group analysis information of the instant viewing customer. Here, the menu image or movie suitable for the customer group can be played on the electronic board 40 to enhance the service warmth of the restaurant. degree.

3. Customer-level analysis service is based on gender statistics, age statistics and flow statistics to analyze the customers of each stratosphere segment and achieve precise marketing.

4. Meal identification, the three image capturing device 10 is installed on the ceiling of the restaurant, to capture the object image on the table, and to carry out the depth learning calculation of the meal outline for the object image of the meal. Identification, and then get the meal identification information, and then the statistical analysis unit 33 counts the cold popularity of each meal to suggest that the restaurant operator to promote the popular meal, and to improve or replace the cold meal.

5. Customer identification, which obtains face recognition information through the above-mentioned deep learning calculation and identification, and then identifies the consumer behavior of the locked face recognition information to determine whether the customer is the first consumption, re-consumption, regular customer, member or The blacklist, hereby, can improve the quality of service by proactively identifying the customer base.

Please refer to FIG. 1 and FIG. 5 for the application example of the present invention applied to the retail department store place 1. The hotspot analysis and the dynamic line statistics are mainly used to make the merchant understand the thinking of different ethnic consumers and set the relative Popular items are more easily exposed. The specific services include the following:

1. The customer discriminates that an image capturing device 10 is installed in the retail department store 1 to capture the image of the person's face and obtain the face recognition information through the above-mentioned deep learning calculation and identification. The customer who has the locked face recognition information is identified by the consumer behavior to determine whether the customer is the first time consumer, re-consumer, regular customer, member or blacklist. Here, the customer quality can be improved by actively identifying the customer group.

2. The electronic kanban push service is installed in the vicinity of the electronic kanban 40 to detect whether the electronic kanban 40 is viewed by the customer who views the electronic kanban 40, and analyzes which advertisement is more attractive for consumption. The ethnic group, and the deep learning calculation identification including the face, the gender, the expression (such as the smile indicates the service satisfaction) and the age profile, and then the identification information of the customer group is obtained, and the statistical analysis unit 33 analyzes the customer group of the instant viewing customer. By analyzing the information, an advertisement image or a movie suitable for the customer group can be played on the electronic billboard 40 to achieve accurate marketing of the product.

3. Customer-level analysis service is based on gender statistics, age statistics and flow statistics. In addition to counting the age of customers to adjust the style of selling goods, we can analyze the customers of different stratosphere customers and count the customer visit rate. And the customer rate, and achieve the purpose of accurate marketing and customer loyalty.

4. Hot spot analysis, the three image capturing device 10 is installed on the ceiling of the retail department store 1 to capture the object image on the product on the cabinet, and perform the deep learning calculation of the product outline on the object image of the product. After identification, the product identification information is obtained, and the statistical analysis unit 33 counts the unpopular/popular stay areas of each commodity to suggest that the retail department store improve the merchandise display mode and optimize the aisle performance.

5. The visitor statistics service is to install the four image capturing device 10 in the vicinity of the entrance and exit of the retail department store, to capture the image of the person who enters and exits the entrance and exit, and perform the image of the human silhouette on the human image. Identification, and then get the flow identification information, and then the statistical analysis unit 33 counts the number of customers entering and exiting to obtain the flow statistics information. Here, the store can be grasped from the peak time to provide the basis for subsequent processing such as service staff scheduling optimization.

6. The customer moves the line, obtains the moving line information according to the flow statistics, and adjusts the moving line according to the moving line information to optimize the product display mode.

Please refer to FIG. 1 and FIG. 7 for the application example of the present invention applied to the home security site 1, mainly by using face and license plate recognition to help preserve the community condition, and to prevent suspicious small entry into the community through the identification of physical action behaviors. Or close care to care for the elderly who are inconvenient to move, to achieve the effect of building a smart community. The specific service content includes the following: face recognition/detection, and an image capturing device 10 is installed near the home access control device 41 to capture the image of the person entering and leaving the person, and capture the feature through the feature. The technology extracts the face image, and then obtains the face recognition information through the above-mentioned deep learning calculation and identification to determine whether the person is a household member at home, and then opens the access control device 41, and saves the face image and records of the person. Time, this can effectively deter the criminal psychology of unscrupulous outsiders.

2. License plate recognition, the second image capturing device 10 is installed in the vicinity of the access control device 41 of the home parking lot to capture the image of the object entering and leaving the vehicle, and through the feature extraction technology, the license plate image is taken out and then passed through the above depth. The license recognition information is obtained by learning the calculation to determine whether the vehicle is owned by the household at home, and the access control device 41 is turned on.

3. Limb action detection, the three image capture device 10 is installed in the home, in order to continuously capture the image of several members of the members of the body movement behavior, and will continue to take several pieces of body movements The image obtains the body motion behavior recognition information through the above-mentioned deep learning calculation and identification to determine whether the member has an abnormal condition, and then alerts the emergency contact person to perform the corresponding treatment.

Please refer to FIG. 1 and FIG. 8 for an application example of the parking lot site 1 applied to the public area of the present invention, mainly through the identification of the vehicle, so that the parking lot can intelligently guide different sizes of vehicles to the appropriate parking space, effectively Plan your parking space.

In addition, in addition to the application embodiment of the above-mentioned site 1, the present invention can also be applied to a company, a school, and the like, mainly by using face and license plate recognition to determine whether the person or the vehicle is a member of the company, the school, or The member is all, yes, the access control device 41 is turned on, and the electronic fence mechanism can be used to detect whether there is any person intrusion, stay, etc., if there is abnormal limb movement behavior, the instant alarm guard inspection, not only that, but also Replacing traditional commuter clocks and sensor tags with face and license plate recognition to avoid inconvenience and trouble caused by lost commuter cards and sensor tags.

In addition, in addition to the application embodiment of the above-mentioned site 1, the present invention can also be applied to the application implementation of the inappropriate image filtering of the movie and television advertisement, mainly through intelligent identification, and the product is automatically recommended by the public, and the program of the favorite character is pushed. Broadcasting, not suitable for children, can also be automatically graded or processed. The specific services include the following:

1. Image recognition, the image capturing device 10 can be installed to capture the position of the image frame of the electronic board, the television or the computer screen, so that the image containing violence, bloodyness and erotic color can be automatically detected, according to the background setting (automatic Apply fog or mosaic to the screen to avoid children watching.

2. Product identification, using the electronic display board 40 to capture the product, using the image capture device 10 on the electronic board 40 to detect the viewer's expression and behavioral response, thereby identifying whether it is one of the goods of interest, in order to achieve accurate Marketing; or detecting the gender and age of the viewer, and then playing an advertisement image suitable for the person.

3. Character recognition, identify the movie star characters from the daily viewing program, and distribute the recommended related film and television news content by the viewing rate.

4. Trademark identification, identify the trademark of the product in the advertisement, use the click rate, use the advertisement to distribute according to the data analysis, and then the content to be viewed will receive the relevant product advertisement.

Furthermore, the convolutional neural network obtains the visual image from the image capturing device, and then performs image preprocessing, feature extraction, feature selection and feature data input, and then into reasoning and predictive identification. On the other hand, the deep learning essence of convolutional neural networks is to build more useful features by constructing machine learning models with multiple hidden layers and massive training data, so as to improve the accuracy of classification or prediction. Convolutional neural networks use massive training data to learn feature recognition, in order to characterize the rich intrinsic information of the data. Since the convolutional neural network is a weight-sharing network structure, in addition to reducing the complexity of the network model, the number of weights can be reduced. This advantage is more obvious when the input of the network is a multi-dimensional image, so that the image can be directly used as the input of the network, avoiding the complicated feature extraction and data reconstruction process in the traditional image recognition algorithm. Object classification is almost always based on statistical features, which means that certain features must be extracted before resolving. The convolutional neural network avoids explicit feature sampling and implicitly learns from the training data. This makes the convolutional neural network distinct from other neural network-based classifiers, merging feature extraction functions into multilayer perceptrons through structural reorganization and reduced weights. It can process grayscale images directly and can be used directly to process image-based classifications. The convolutional network has the following advantages in image processing compared with the general neural network: the input image and the network topology can be well matched; the feature extraction is performed simultaneously with the pattern classification, and simultaneously generated in the training; Sharing can reduce the training parameters of the network, making the neural network structure simpler and more adaptable.

The above is only a possible embodiment of the present invention, and is not intended to limit the scope of the patents of the present invention, and the equivalent implementations of other changes according to the contents, features and spirits of the following claims should be It is included in the patent of the present invention. The invention is specifically defined in the structural features of the request item, is not found in the same kind of articles, and has practicality and progress, has met the requirements of the invention patent, and has filed an application according to law, and invites the bureau to approve the patent according to law to maintain the present invention. The legal rights of the applicant.

Claims (10)

 A smart image information and big data analysis system using deep learning technology, comprising: at least one image capturing device for capturing an image of an object of at least one object, the object image comprising at least one image; at least one information The processing device is connected to each of the image capturing device signals through a signal transmission module for collecting and receiving the human image; and at least one cloud image recognition unit transmits the signal through the network transmission module and the information processing device The connection is configured to receive the image of the person, and the cloud image recognition unit has at least one deep learning calculation module with a deep learning training function to perform image recognition, and the deep learning calculation module can perform a facial expression on the image of the person And image recognition of the limb movement to obtain at least one facial expression recognition information and limb motion recognition information.    The smart image information and the big data analysis system using the deep learning technology described in claim 1, wherein the cloud image recognition unit further comprises an image feature database with a plurality of human sample images, the deep learning calculation module. Extracting the feature of the sample image to include a facial expression feature data, and setting a corresponding expression recognition data for each facial expression feature data, when the deep learning calculation module receives at least instant capture When the image of the person is imaged, the facial expression feature data corresponding to the feature of the sample image of the person is recognized in the image feature database, and the expression recognition data corresponding to the feature is read, and a corresponding face is output. Facial expression information.    The smart image information and big data analysis system using the deep learning technology described in claim 2, wherein the facial expression recognition information is selected from a slightly happy, happy, extremely happy, slightly sad, sad, extremely sad, slightly not At least two expressions of pleasing, unpleasant, and extremely unpleasant.    The smart image information and the big data analysis system using the deep learning technology described in claim 1, wherein the cloud image recognition unit further comprises an image feature database with a plurality of human sample images, the deep learning calculation module. The feature of the human sample image is extracted to include a limb motion characteristic data, and a corresponding limb motion recognition data is set in each of the limb motion feature data, and the deep learning calculus module receives the continuous number of consecutive captures. When the image of the person is imaged, the body motion feature data corresponding to the feature of the sample image is identified in the image feature database, and the body motion recognition data corresponding to the feature is read, and a corresponding limb is output. Motion recognition information.    The smart image information and the big data analysis system using the deep learning technology described in claim 4, wherein the limb motion recognition information is an abnormal limb motion recognition information, and the abnormal limb motion recognition information is selected from the group consisting of climbing, stealing, and At least one of the abnormal limb movement identification information of snatching, fighting, violent, defamatory, erotic, roaring, and recumbent; the abnormal limb movement recognition information of the climbing is within five seconds, the elbow is higher than the arm When the height is one-half of the original elbow and the knee is higher than the one-third of the height of the original knee when the thigh is extended downward, it is determined that the abnormal limb movement of the climb; the theft Abnormal limb movement recognition information, one person puts his hand into a person's pocket or purse, that is, an abnormal physical movement determined as stealing; the abnormal physical movement recognition information of the snatch is within one minute, approaching at least one other person After a distance of fifty centimeters, you can quickly lift your leg or raise your hand and move away from the other person by at least five meters within five seconds. Abnormal limb movement; the abnormal limb movement recognition information of the fight is determined by the fact that at least two people are close to fifty centimeters in body and perform mutual limb movements such as raising their legs or raising their hands within two seconds. An abnormal physical movement of the fighting; the abnormal physical movement recognition information of the violence is determined by one person's body being within 50 centimeters of another person, and within two seconds of doing a leg movement or raising a hand, etc. An abnormal physical movement for violence; the abnormal physical movement recognition information of the sputum is that one person reveals his genital organs or the person is forced to hug a person and the other person makes a rebellious action, and then the abnormal physical movement can be determined; The abnormal physical movement recognition information of the pornography is that at least one person performs an action of revealing at least one of the genitals or at least two naked persons to make a hug, and can determine the abnormal physical movement of the erotic; the abnormal physical movement recognition information of the roar It is judged to be a roar when a person’s face makes an expression of anger and his mouth is quickly opened and raised in one second. The abnormal limb movement; the abnormal limb movement recognition information that can not be recuperated is one person who has been lying on the ground for more than two minutes, and can be judged as an abnormal limb movement that cannot be recumbent.    The smart image information and the big data analysis system using the deep learning technology described in claim 1, wherein the object image includes at least one live image, and the cloud image recognition unit transmits the information through the network transmission module The device receives the live video image, and the deep learning calculation module can perform violent, bloody and pornographic image recognition on the live video image, and when the deep learning calculation module detects that the live video image is violent, bloody or pornographic, etc. When an image content is used, the playback of the live video is interrupted; or the image processing of the mosaic is performed.    The smart image information and the big data analysis system using the deep learning technology described in claim 1, wherein the object image includes at least one video image, and the cloud image recognition unit further includes a plurality of built-in trademark sample images. An image feature database, and each of the trademark sample images is provided with a trademark feature data and a trademark identification data corresponding to the trademark feature data, and the cloud image recognition unit receives the same through the network transmission module and the information processing device a video image, and capturing the feature of the film and television image as containing at least one trademark feature image, and identifying the trademark feature data corresponding to the feature of the trademark feature image in the image feature database, and reading the feature matching The trademark identification data is further outputted with a corresponding identification information of the trademark, so that the deep learning calculation module can identify the advertisement trademark of the film and television image, and count the number of times and the playing time of the advertisement trademark appearing in the film and television image. In turn, the number of times the advertisement trademark and the play time statistics are output.    A method for analyzing a big data using deep learning intelligent image information, comprising the steps of: providing at least one image capturing device, at least one information processing device, and at least one cloud image recognition unit; capturing the at least one image capturing device An image of the object of at least one object, wherein the image of the object includes at least one image; the information processing device is coupled to each of the image capturing device signals through a signal transmission module for collecting and receiving the image of the person; and the cloud The image recognition unit is coupled to the information processing device via a network transmission module for receiving the image of the person. The cloud image recognition unit has at least one deep learning algorithm module with deep learning training function for performing image recognition. The deep learning calculus module can perform image recognition of facial expressions and limb movements on the human image to obtain at least one facial expression recognition information and limb motion recognition information.    The big data analysis method for applying the deep learning optical image information according to claim 8, wherein the deep learning calculation module comprises the following steps: (a) a training phase step, and at least one deep learning is established. Model, and input a huge amount of sample object image in the deep learning model, and test the correct rate of image recognition by the deep learning model, and then determine whether the image recognition correct rate is sufficient. When the judgment result is yes, the identification result is output and Storing; when the judgment result is no, the deep learning model is self-corrected learning; and (b) running the prediction phase step, the object is captured by the deep learning model, and the object is predicted by the deep learning model. Sexual image recognition to obtain at least one of the facial expression recognition information and the limb motion recognition information.    The big data analysis method for applying the deep learning smart image information according to claim 9, wherein the deep learning model comprises a plurality of different attribute classifiers, and the deep learning calculation module further comprises a feature image capturing step The local feature of the image of the sample object is captured as the image of the sample object, and then the sample visual image is input into the deep learning model, and then compared by the classifier of the same attribute to compare The result is output to the deep learning model to output the facial expression recognition information and the judgment basis of the limb motion recognition information as the deep learning model.