JP6796301B1 - Information processing system, information processing device, terminal device, server device, program, or method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more appropriately utilize data obtained from an image in an example system of the present invention. SOLUTION: Based on an image acquisition unit that acquires an image from the one or a plurality of image pickup devices, in which one or a plurality of image pickup devices each have a plurality of parking spaces as an image pickup field, and the one or a plurality of the images. A system including a charge setting unit for setting a charge for a first parking space in the plurality of parking spaces. Based on the one or a plurality of the images, the charge setting unit charges a charge for the second parking space in the plurality of parking spaces different from the charge for the first parking space, and a charge different from the charge for the first parking space. Set to system. When the fullness information of the first parking area including the first parking space is higher than the fullness information of the second parking area including the second parking space, the charge setting unit of the first parking space A system that sets the charge higher than the charge for the second parking space. [Selection diagram] Fig. 2

Description

The techniques disclosed in this application relate to information processing systems, information processing devices, server devices, programs, or methods.

In recent years, information processing using images has been performed, but the information in the images is not always properly used.

Japanese Unexamined Patent Publication No. 6-223299 Japanese Unexamined Patent Publication No. 2014-123356 JP-A-2016-75499

Therefore, various embodiments of the present invention provide an information processing system, an information processing device, a terminal device, a server device, a program, or a method in order to solve the above problems.

The first system according to one embodiment is
An image acquisition unit that acquires one or more images from the one or more imaging devices, each of which has a plurality of parking spaces as an imaging field of view.
A charge setting unit that sets a charge for the first parking space in the plurality of parking spaces based on the one or more images.
System with. Here, the charge for the first parking space may be the charge for use per unit of the garden such as 1 hour, 3 hours, 6 hours, 12 hours, 24 hours, and 1 week, or at 9:00 on the same day and the next day. It may be a charge for use until the deadline, such as up to. Further, the image used for setting the charge of the first parking space may be a single image or a plurality of images. When it is a plurality of images, it may be a plurality of images captured by one imaging device, or may be each image captured by a plurality of imaging devices. In the present invention, an image captured by an imaging device having one parking space as an imaging field of view may be used.

The second system according to one embodiment is
Based on the one or a plurality of the images, the charge setting unit charges a charge for the second parking space in the plurality of parking spaces different from the charge for the first parking space, and a charge different from the charge for the first parking space. Set to
The first system. Here, the second parking space may be a parking space different from the first parking space, and the first parking space and the second parking space may be imaged by the same imaging device or may be imaged by different imaging devices. The first parking space and the second parking space may be included in the same image, or may be included in different images.

The third system according to one embodiment is
When the fullness information of the first parking area including the first parking space is higher than the fullness information of the second parking area including the second parking space, the charge setting unit of the first parking space The charge is set higher than the charge of the second parking space.
First or second system.

The fourth system according to one embodiment is
The image utilization department
Using the images captured by one or more imaging devices including the first parking area including the first parking space in the imaging field of view, the full / empty information of the first parking area is generated.
Using the images captured by one or more imaging devices including the second parking area including the second parking space in the imaging field of view, the full / empty information of the second parking area is generated.
Any one of the first to third systems.

The fifth system according to one embodiment is
When the charge for the first parking space is the first charge at the first predetermined time, the charge setting unit charges the charge for the first parking space at the second predetermined time different from the first predetermined time. A second charge different from the first charge is set based on one or more of the images.
Any one of the first to fourth systems. Here, the first predetermined time and the second predetermined time may be various temporal information such as time, day, day of the week, month and day, respectively. The charges for the same first parking space may be set at different charges at different times. In the same first parking space, it is not always necessary to set different charges at different times, and the same charges may be set.

The sixth system according to one embodiment is
The charge setting unit sets the charge for the first parking space by using the parking usage history of the plurality of parking spaces including the first parking space.
Any one of the first to fifth systems.

The seventh system according to one embodiment is
An image acquisition unit that acquires images from one or more imaging devices with a plurality of parking spaces as the imaging field of view.
A rate setting unit that sets the monetary value for parking in the first parking space in the plurality of parking spaces based on the one or more images.
System with.

The eighth system according to one embodiment is
The monetary value includes the monetary value available in the commercial facility relating to the parking space.
Seventh system. Here, the monetary value that can be used in the commercial facility may be, for example, a coupon, a gift certificate, points, or the like that can be used in the commercial facility.

The ninth program according to one embodiment is
System,
An image acquisition means for acquiring an image from one or more imaging devices, each of which has a plurality of parking spaces as an imaging field of view.
A charge setting means for setting a charge for a first parking space in the plurality of parking spaces based on the one or more images.
A program to operate as.

The tenth program according to one embodiment is
Use the memory provided by the system,
Ninth program.

The eleventh program according to one embodiment is
Use the arithmetic unit provided in the system,
Ninth or tenth program.

The twelfth method according to one embodiment is
System,
An image acquisition step of acquiring an image from one or more imaging devices, each of which has a plurality of parking spaces as an imaging field of view.
A charge setting step for setting a charge for a first parking space in the plurality of parking spaces based on one or more of the images.
How to run as.

The thirteenth method according to one embodiment is
The memory provided in the system is used.
Twelfth method.

The fourteenth method according to one embodiment is
The arithmetic unit provided in the computer is used.
The twelfth or thirteenth method.

The fifteenth program according to one embodiment is
A program for operating a computer as any one of the first to eighth systems.

According to one embodiment of the present invention, image information can be utilized more appropriately.

FIG. 1 is a block diagram showing a function of a system according to an embodiment.
FIG. 2 is a block diagram showing a function of a system according to an embodiment.
FIG. 3 is a block diagram showing a function of a system according to an embodiment.
FIG. 4 is an example of a screen displayed by the system according to the embodiment.
FIG. 5A is an example of a data structure used by the system according to an embodiment.
FIG. 5B is an example of a data structure used by the system according to an embodiment.
FIG. 6 is an example of a display screen of a system according to an embodiment.
FIG. 7 is an example of a display screen of a system according to an embodiment.
FIG. 8 is an example of a data structure used by the system according to the embodiment.
FIG. 9A is an explanatory view of a license plate.
FIG. 9B is an example of a real-world captured image used by the system of one embodiment.
FIG. 9C is an example of a pseudo-captured image used by the system of one embodiment.
FIG. 10A is an example showing a situation in which an image pickup apparatus according to a system according to an embodiment identifies using an image captured.
FIG. 10B is an example showing a situation in which an image pickup apparatus according to a system according to an embodiment identifies using an image captured.
FIG. 10C is an example showing a situation in which an image pickup apparatus according to a system according to an embodiment identifies using an image captured.
FIG. 10D is an example showing a situation in which an image pickup apparatus according to a system according to an embodiment identifies using an image captured.
FIG. 11 is an example showing an overall configuration of a system according to an embodiment.
FIG. 12 is an example of a parking lot targeted by the system according to one embodiment.
FIG. 13 is an example of a parking lot targeted by the system according to one embodiment.
FIG. 14 is an example of a parking lot targeted by the system according to the embodiment.
FIG. 15 is an example of an overall configuration relating to a system according to an embodiment.
FIG. 16 shows an example of a system flow according to an embodiment.
FIG. 17 shows an example of a system flow according to an embodiment.
FIG. 18 shows an example of a system flow according to an embodiment.
FIG. 19 shows an example of a system flow according to an embodiment.
FIG. 20 shows an example of a system flow according to an embodiment.
FIG. 21 is an example of a parking lot targeted by the system according to one embodiment.
FIG. 22 is an example of a data structure used by the system according to an embodiment.
FIG. 23 is a block diagram showing a configuration of an information processing apparatus according to an embodiment.
The examples corresponding to Examples 1 to 3 are reference examples of the invention described in the claims of the present application.

1. 1. Outline An example system according to the documents of the present application is a system that supports parking. One example system may include a management system that manages the parking of the vehicle. Also, an example system may include a management system and one or more imaging devices that are directly or indirectly connected to such a management system. In addition, an example system is one or a plurality of user terminal devices used by system users, a management system that can be connected to such a user terminal, and one that is directly or indirectly connected to the management system. Alternatively, a plurality of imaging devices may be provided. In addition, the management system may be directly or indirectly connected to one or more administrator terminal devices that control such a management system. In addition, the management system may be installed in association with the parking lot and directly or indirectly connected to a display device visible from the entrance or passage of the parking lot.

In the documents of the present application, the term "system" is used as a superordinate concept of a management system and a terminal device, and the system may include a management system and may not include one or more terminal devices, or may include one or more terminal devices. It may include the management system and may not include the management system, or may include both the management system and one or more terminal devices. The terminal device may be used as a superordinate concept of the user terminal device and the administrator terminal device. The system may consist of one or more information processing devices. Further, the term "fee" may be a term meaning an amount of monetary value.

The user terminal, the administrator terminal, the management system, the imaging device, and the display device may be directly or indirectly connected by a communication device such as the Internet.

The terminal device may have one or more position measuring devices. The position measuring device may have a position measuring function using GPS or a base station.

The terminal device may include a communication device. The communication device may be capable of communicating with an image pickup device installed in the vehicle in which the terminal device is installed. The communication device may be a wireless type or a wired type. In the case of wireless type, it may be WIFI, BLUETOOTH or the like. The imaging device installed in the parking lot may be a fixed type or a detachable type.

The terminal device may be a portable terminal device that can be carried by a person. For example, the portable terminal device may be a smartphone, a PDA, or the like.

2. 2. Functions of an Example System The functions of an example system will be described with reference to FIGS. 1 to 3. FIG. 1 is a block diagram showing a specific example of a function related to a user terminal device of an example system, and FIG. 2 is a block diagram showing a specific example of a function related to a management system of an example system. Is a block diagram showing a specific example of the function related to the administrator terminal device of an example system.

2.1. Functions in the User Terminal Device Some or all of the following functions of the user terminal device may be realized by software that commands some or all of the following functions to execute the information processing device. Such software may be downloaded and installed in the information processing device. In the documents of the present application, such software may be referred to as a "parking application". The parking application may have all the functions included in the user terminal device described below, or may have some functions thereof.

In the documents of the present application, the "user" may be a person who uses the parking application. Therefore, the user may or may not be the payer of the payment instrument described below. Further, the user may be a person who has installed the parking application, or may be a person who simply temporarily uses the parking application without installing it.

In addition, the parking application may be configured so that a plurality of persons can use one or a plurality of vehicles in use. When multiple users use it, one ID and one authentication means may be set to not distinguish between multiple users, or multiple users may be distinguished by using the ID and authentication means. It may be configured to be used. In the latter, for example, one ID and a plurality of authentication means may be set, or the same or different authentication means as the plurality of IDs may be set. In this way, the configuration may be such that a plurality of users can be distinguished by the ID and / or the authentication means. Further, when the configuration is such that a plurality of users can be distinguished, the functions that can be used below may be different. For example, a user who can set user information in the user information management unit and a user who can use the function related to user support described later may be different. In addition, available users may be set here for each function provided by the user information management unit and the user support unit.

Hereinafter, the function of the parking application will be described, but some or all of the following functions of the user terminal device may be available by accessing one or more specific URLs on the Internet. In this case, the same function may be realized by accessing one or more specific URLs on the Internet from within the user terminal device. That is, the information that can be input to the parking application may be accessed and input on one or more specific URLs on the Internet. In addition, the information that can be output and displayed in the parking application may be displayed by accessing it on one or more specific URLs on the Internet. It should be noted that the security of access on the Internet may be improved by authenticating the user.

2.1.1. User Information Management Department The user information management department has a function of managing user information. In the documents of the present application, the user information includes the user's name, the user's nickname, the user's address, the user's ID, the user's registration authentication information, the user's vehicle information, and the user's payment means. It may be a term including information, usage history information related to the user, and the like. The user information management unit may have a function of accepting input related to user information. In addition, the user information management unit may have a function of displaying information related to user information.

The vehicle information related to the user may be information about one or more vehicles registered in association with the user in the parking application. For example, the vehicle information may be information that identifies the vehicle, an image related to the appearance of the vehicle, information about the vehicle, and the like.

The information for identifying the vehicle may be information related to the vehicle license plate of the vehicle, or may be a part or all of the numbers in the license plate or an image obtained by capturing the license plate. In the documents of the present application, since the vehicle specified by the information for identifying the vehicle is the vehicle to be used, it may be referred to as "used vehicle".

The image relating to the appearance of the vehicle may be an image of the vehicle being used. The image of the vehicle to be used may be an image taken from the front direction, the rear direction, the lateral direction, the oblique method, or the like of the vehicle to be used.

The information about the vehicle may be, for example, the manufacturer of the vehicle used, the vehicle type of the vehicle used, the name of the vehicle used, information indicating the characteristics of the appearance of the vehicle used, and the like.

The vehicle type of the vehicle may be information that can be classified from various viewpoints. For example, it may be classified by the Road Transport Vehicle Law, such as ordinary automobiles, small automobiles, light automobiles, large special automobiles, etc., it may be classified by buses, trucks, large motorcycles, motorcycles, etc., and wheels. It may be classified by the number, the size of the vehicle, or the total exhaust amount of the engine.

The payment means information relating to the user may be any information on the means that can be paid for the parking act. For example, payment method information includes information on payment using a credit card, information on payment using a debit card, information on payment using electronic money, information on an account held by a payer at a financial institution, and the like. Not limited to these.

The user information management unit may store one or more payment means information. Further, the user information management unit may store only one payment means information in association with one or more user vehicles, or may associate and store a plurality of payment means information. In the former case, there is an advantage that payment means for parking of one or more vehicles are integrated into one. On the other hand, when different payment means information is associated and stored for a plurality of use vehicles, there is an advantage that the use of each use vehicle can be set to be paid by each payment means. For example, if one family has two vehicles, there is an advantage that different payment methods can be set for each vehicle.

In addition, the user information management unit may store a plurality of payment means information in association with each other for one vehicle. In this case, it may have a function of setting how to allocate the parking fee for the parking act for one user to a plurality of payment means information. With such an allocation function, for example, when one vehicle is used by different people in the family, there is an advantage that the payment method of the parking fee can be changed.

The allocation may be variously set. The allocation may be done manually (manually) or may have the ability to allocate automatically. If the latter has an automatic allocation function, the allocation conditions may be set in advance.

The function that can set the allocation may be, for example, a function of allocating the parking fee, a function of allocating using the parking place, or a function of allocating using the parking date and time. Alternatively, it may be a function of allocating using the position information of the user terminal.

The function of allocating the parking fee may be a function of setting a specific fee for the parking fee as at least one payment means, or a function of setting the ratio of the allocation for the parking fee. Here, the parking fee may be a fee for each parking, a fee for a predetermined period, or a total fee at a payment time such as every month. As an example of setting a specific fee for the total fee, for example, when the monthly usage fee is 10,000 yen and there are two payment method information, credit card A and account A. In the above, 3000 yen may be used as a credit card A, 7000 yen may be used as an account A, and so on. By being able to set a specific charge, there is an advantage that the burden can be set according to the user's request even for the usage behavior of one user.

Further, the function of setting the allocation ratio may be 40% for the credit card A, 60% for the account A, or the like. Here, two payment method information are illustrated, but there may be three or more. This has the advantage that even if it is troublesome to set a specific charge, it can be easily set according to the ratio.

As for the allocation function for parking places, payment means information may be set for each prefecture, each municipality, each parking lot, and the like. For example, since the users of the parking lot at work and the parking lot at business trips are known in advance, even a family member who uses one vehicle for multiple people can efficiently allocate payment method information. There is an advantage that it can be set.

In addition, the parking date and time allocation function can be divided into, for example, business days and non-business days, daytime (for example, 6:00 to 18:00) and nighttime (for example, 18:00 to 6:00), and the month. It may be divided into the first half and the second half of.

If the location information of the user terminal can be used, the location information of the user terminal and the payment method information are associated and stored, and the parking place and the location information of the user terminal are matched with each other. It may be an allocation function that adopts payment method information. In this case, since the parking action can be specified in association with the location information of the user terminal, there is an advantage that the payment method associated with the user terminal can be specified.

Further, regarding the setting and change of the payment method, the payment method that can be set and changed may be different for each user. That is, for each payment method, information identifying one or more users is associated and stored, and only each associated user can set whether to use each payment method or not. You can. For example, if one user is associated with payment instruments A, B, C, but payment instrument D is not, such user can use A, B, C as payment instruments, Payment instrument D may be configured so that it cannot be used. Here, the payment means A, B, and C may be account information, a credit card, a debit card, or the like in which the user is the holder or the person responsible for payment. On the other hand, in the payment means D, a user different from such a user may be the holder or the person responsible for payment.

FIG. 4 is an example of a screen for setting a payment method. Here, the user may input the payment means information that he / she can use into the user terminal in advance. For example, six types of information of payment means information 1 to 6 may be input, and the user information management unit may be able to store the input information and display the input information. The user may select a payment method for each of the one or more vehicles used. In this case, one or a plurality of pre-stored payment methods may be selected and input for one vehicle to be used, and the user information management unit stores the input information and inputs the input. It may be possible to display the provided information. Here, a plurality of payment methods may be selected. In addition, the user may input one or more conditions when using each payment method, and the user information management unit can store the input information and display the input information. It may be there. For example, it may be set with the conditions to which each payment instrument applies. In this figure, payment means 3 and 4 are associated with each other and set in association with the conditions of weekdays and holidays.

The user authentication information may be information that can authenticate the user. The user information management unit may have an authentication function capable of authenticating a user by using such authentication information. The authentication function may be one or more authentication functions such as face authentication, fingerprint authentication, voiceprint authentication, PIN authentication, and puzzle-type authentication. It has the advantage of preventing falsification of payment method information and information leakage. The authentication means may be a personal identification number, face recognition, fingerprint authentication, voiceprint authentication, or the like, and the authentication information may be information that can be compared and verified to match each authentication means.

The usage history information relating to the user may include history information regarding parking. The history information may be parking history information for each user or parking history information for each vehicle used. The parking history information may include, for example, information on parking time, information on parking lot, image captured at the time of parking, parking fee-related information, etc., but may be a part or all of them.

Information about parking time may include parking date, parking start time, parking end time, and / or parking duration, and the like.

Information about the parking lot includes the name of the parking lot, the abbreviation of the parking lot, the nickname of the parking lot, the name of the facility to which the parking lot belongs, the name of the facility to which the parking lot is attached, the address of the facility, and the abbreviation of the facility. , URL related to the parking lot (for example, URL of the parking lot, URL of the facility to which the parking lot belongs, URL of the facility to which the parking lot is attached), and / or a map showing the location related to the parking lot (parking lot) Map, map of the facility to which the parking lot belongs, map of the facility to which the parking lot is attached), etc. may be included.

The parking fee-related information may include a parking fee for each parking act, a payer in the case of payment, a payment method in the case of payment, and the like.

FIG. 5A is an example of data for storing user information. As the user information, the user ID, the user's name, the authentication information, the vehicle information, and the payment method information are stored in association with each other. In this example, one family is composed of three people, has two vehicles, and is available for each vehicle. Therefore, for each person, for each vehicle, the payment means information is associated and stored. That is, Mr. A stores the means 1 in association with the vehicle A and the means 2 in association with the vehicle B as payment means information. On the other hand, for Mr. B, the means 1 which is the same means is associated and stored for both the vehicle A and the vehicle B. On the other hand, Mr. C remembers the means 2 in association with only the vehicle A, and does not store the means for the vehicle B. One example system may allow the vehicle B to be used or may not be able to use it in a state where such payment means are not stored. In the former case, regarding the use of vehicle B, by making the payment means memorized by the time of payment, or by associating and memorizing at least one payment means for one vehicle, the corresponding person. If there is no payment, it may be processed so that payment is automatically made from at least one such associated and stored payment instrument. In the latter case, it may be indicated that the payment method is not specified at the time of use and the parking fee must be paid at the time of parking. In such a case, in the system of one example, the payment method is not specified when Mr. C, who is a user, starts the application to use the vehicle used (for example, vehicle B in this example). And / or may provide information about the need to specify payment instruments. The means of provision may be voice, display, or the like, as described above. The vehicle B, which is the vehicle to be used, may be specified by inputting the information for identifying the vehicle to be used from the parking application by Mr. C, who is the user, or the position information such as GPS of the vehicle to be used. And Mr. C may be identified by comparing with the position information of the mobile terminal related to Mr. C based on the identification by the activation of this application.

Further, FIG. 5B is an example in which each user, the payment means, and the conditions to which the payment means is applied are stored in association with each other. For example, for the user ID 001, the payment means 1 is applied to the use of the vehicle A when parking in a business-related parking lot. For example, when the payment method 1 is associated with the settlement in the company, as shown in this figure, when the vehicle A is parked in the business-related parking lot, the vehicle B is used on weekdays, and the vehicle B is used at night. In the case of use, it is also possible to configure that the payment means 1 is used whenever the user ID 002 parks the vehicle A at the place A and the user ID 003 uses the vehicle A. By doing so, by associating with the conditions, there is an advantage that the payment method can be allocated without having to set the payment method for each parking act in relation to the conditions such as the parking place and the parking time.

2.1.2. Image capturing unit The image capturing unit has a function of capturing an image. The image capturing unit has a function of storing the captured image.

The image capturing unit may have an OCR function. The image capturing unit may acquire information from the captured image by using the OCR function. In this case, the target information may be automatically acquired according to the target to be imaged, that is, according to the target of the captured image, or the type of the image to be captured in advance is received by the input of the user. In the above, the target information may be acquired.

The object to be imaged may be a payment means, and may be the above-mentioned credit card, debit card, electronic money, bank account card, bank account book, or the like. The image capturing unit may acquire information that identifies the payment means according to each image pickup target. When the image capturing unit automatically acquires the target information, it may have a function of specifying these targets according to the characteristics of these payment means. For example, if you are imaging a credit card, you can identify that you are imaging a credit card, and if you are imaging a bank account card, you can identify that you are imaging a bank account card. You can do it. In addition, these imaging targets may be specified by user input. For example, the information to be imaged may be input by the user, such as imaging a credit card, imaging a debit card, imaging a bank account card, and the like. In the parking act, the vehicle used may be shared by a plurality of users, and the payment means may be different for each of the users, so there is a great advantage that the payment means can be easily input.

Further, the image capturing unit may include a license plate as an image capturing target. In this case, the image capturing unit may use the OCR function to acquire information on the specific character string of the license plate, or use the machine-learned identification function to obtain information on the specific character string of the license plate. Information may be obtained. In the latter case, the machine-learned identification function may be provided by the image capturing unit itself in the terminal device, or the license plate information may be acquired by using the machine-learned identification device outside the terminal device. Good. When using the machine-learned identification device outside the terminal device of the latter, the image capturing unit transmits the target image to the identification device and receives the license plate information identified by the identification device. Information on such license plates may be obtained. The machine-learned identification function may be stored in the information processing device in the terminal device. Since the object to be identified is limited in advance, there is an advantage that a high identification function can be realized even in a simple information processing device such as in a terminal device.

Note that, unlike the above, the image capturing unit may simply image the license plate, acquire the image, and transmit the image to the management system. Further, the image capturing unit does not have to store only the captured image and does not have the information of the specific character string of the corresponding license plate. This is because the image capturing unit stores the image including the license plate, so that the user can understand which vehicle is used even if he / she does not have the information of the specific character string of the license plate. is there.

When the image capturing unit captures an image, information that can be used as a reference for the outer frame of the license plate to be imaged may be displayed on the display screen in the terminal device on which the imaging target is displayed. FIG. 6 shows an example of a display screen displaying an image pickup target for which a license plate is to be imaged with a smartphone. The license plate 62 is displayed on the right side of the screen in the screen 61. Therefore, the display 63 of the outer frame, which is a display of reference information, is displayed, and it is recommended that the size corresponds to the outer frame. In this way, the image capturing unit in the terminal device has a function of displaying such reference information, thereby providing a function of supporting the acquisition of an image to the extent that the information on the license plate can be accurately read. May have.

The image captured by the image capturing unit may be stored in association with the payment means information obtained from such an image or the license plate information. In addition, such image and / or license plate information may be transmitted to the management system.

2.1.3. User Support Department The vehicle support department has a function of supporting users. The user support unit may have a function of supporting the parking behavior of the user, or may have a function of promoting the parking behavior of the user.

The user support department may support the user by displaying the information. The displayed information may be information that is automatically displayed, or information that receives input from the user and corresponds to such input.

The user support department has a function of managing the parking area. In the documents of the present application, a space in which one vehicle can be parked may be referred to as a parking space. The parking space may be one in which the vehicle is parked or one in which the vehicle is not parked. In addition, one or more parking spaces may be referred to as a parking area. Therefore, the parking area may be a term of a superordinate concept of the parking space. Further, since the parking area is one or more parking spaces, it may be wider than the so-called parking lot or narrower than the parking lot. Further, one parking area may be a place managed by only one manager, or may be managed by a plurality of managers. In addition, one manager may manage a plurality of parking areas.

Further, in the documents of the present application, the information indicating the status of the parking area may be referred to as "full vacancy information". The term full vacancy information means that the parking area consisting of only one parking space is parked or empty, and that part or all of the parking area consisting of multiple parking spaces is parked or parkable. It may be a superordinate concept of information about a certain case. In addition, the fullness information may be a superordinate concept of information on the degree of parking or parking. For example, for a parking space where 10 cars can be parked, the full vacancy information for the state where 3 cars are parked is almost empty, but the full vacancy information for the state where 8 cars are parked is almost full. It may be information. The fullness information may include an absolute number indicating the number of parkable spaces, or may include a relative value including the percentage of parkable spaces. In addition, the fullness information may be ranked to indicate the degree of fullness. For example, when the state in which all parking spaces are parked and the state in which all parking spaces are parkable are ranked as A to E in order, the full vacancy information includes one of the information in such ranking. It may be a thing. Further, the fact that the fullness information α of the parking area A is higher than the fullness information β of the parking area B means that the fullness information α is more crowded than the fullness information β if the information indicates the degree of the fullness information. It may indicate a high degree state. For example, if the full / empty information is an absolute value or a ratio, the fact that the full / empty information α is higher than the full / empty information β may be a value higher than the full / empty information α and higher than the full / empty information β. In the documents of the present application, in a parking area including one or more parking spaces, the term indicating a space in which one or more vehicles can be parked may be referred to as a parkable area.

The user support unit may have a function of providing information on the availability of the parking area. The function of providing full / empty information may include a function of displaying on the screen of the user terminal, or may include a function of guiding by voice using a voice generator provided in the user terminal. Other provided functions may be included.

When the user support unit displays the parkable area on the screen, there is an advantage that one or more parkable spaces can be visually understood. Further, the parkable area may be displayed as an area in the parking lot. In this case, there is an advantage that the parkable area in the parking lot can be understood. Further, in addition to the parkable area, the position of the target vehicle to be used in the parking lot may be displayed. Further, the position of the used vehicle may be the position of the current real-time used vehicle. In this case, there is an advantage that the parkable area can be understood including the positional relationship with the position of the vehicle to be used.

FIG. 7 is an example in which the user support unit displays information indicating a parkable area, information indicating the current position of the vehicle in use, and a display screen in the terminal device in the parking lot. The parkable area is indicated by diagonal lines 71, and the current position of the vehicle in use is indicated by ● mark 72. Needless to say, the parkable area and the current position of the vehicle used may be displayed by various information. Such a display has an advantage that the user can easily grasp the positional relationship between the position of the currently used vehicle and the parkable area.

When the parkable area is displayed by voice, the user does not need to check the user terminal while driving, and has an advantage that the user can safely move to the parkable area. In this case, the voice may provide information to sequentially guide in relation to the current position. For example, it may be a voice such as "Next is a right turn." "Please drive straight for 〇 meters and turn left."

Information about some or all of the parkable parking spaces may include information that can be directed to such parkable parking spaces. For example, such information may be information indicating an absolute position or information indicating a relative position in relation to the current vehicle. The information indicating the absolute position of the former may include, for example, information for identifying such a divided area when the parking lot is divided into a plurality of areas. For example, when there are parking lot areas A, B, and C as a plurality of areas, the information may be the B area among them. On the other hand, the latter information indicating the relative position may be information that can guide the position of the parkable parking space to be guided with reference to the current position of the vehicle to be used.

Further, the information about a part or all of the parking space that can be parked may include the above-mentioned information that can be guided to the parking space that can be parked, as well as the fullness information that is information indicating the status of the parking space.

The information that can be guided to the parking space may be generated in the user terminal or may be acquired from the management system.

In the former case where information that can be guided to the parking space is generated in the user terminal, it may be generated by using the fullness information of the parking lot that the user terminal has. The fullness information of such a parking lot may be obtained from the management system. In this case, the user terminal may transmit information for identifying the parking lot to be used to the management system when it is determined that the specific parking lot will be used. In response to such information, the management system may transmit the corresponding parking lot fullness information. The user support unit may generate information that can be guided to the parking space by various methods, and the specific processing thereof will be collectively described at a place in the management system described later.

On the other hand, in the latter case of acquiring information that can be guided to the parking space from the management system, information that can be guided to the parking space may be acquired from the management system. In this case, the user terminal may transmit information for identifying the parking lot to be used to the management system when it is determined that the specific parking lot will be used. In response to such information, the management system may transmit information that can be directed to the parking space for the corresponding parking lot.

Further, a part of the information that can be guided to the parking space may be generated by the management system, and the rest may be generated in the user terminal. The distribution of information processing between the management system and the terminal device may be performed by various methods.

The user support department may have a function of providing information on parking fees, which is a consideration for the act of using the parking space. The function to be provided may be displayed on the screen of the user terminal, or may be guided by voice using a voice generator provided in the user terminal. It may be other means. When displayed on the screen of the user terminal, when a plurality of parking fees are displayed in association with each parking space that can be parked, the user can determine the parking space while comparing the parking fees. There are advantages. In this case, spaces with the same parking fee may be displayed in the same manner. For example, they may have the same mark, the same color, or the same pattern.

When provided by voice, the user does not need to check the user terminal while driving, and has the advantage of being able to safely move to the parkable area. In this case, the voice may sequentially provide parking fee information in relation to the current position.

The user support department may have a function of providing monetary value corresponding to the use of the parking space in the parking lot. The monetary value may be coupons, electronic money, points, and the like. The coupon may be in the facility related to the parking lot. Coupons may reduce the price of goods or services. Points may be themselves or accumulated to reduce the price of goods or services.

2.2. Management system functions
2.2.1. User Information Management Department The user information management department has a function of managing user information. The user management unit may have a database having user information. In addition, the user management unit may be able to cooperate with an external database so that user information can be acquired, stored, and / or managed.

FIG. 9 is an example of user information data managed by the user management unit. This is an example in which user ID, name, authentication information, vehicle information, payment method information, etc. are stored and managed for each user.

It should be noted that a plurality of users may be individually set for each user, or the plurality of users may be set so as to have a certain relationship. For example, as an embodiment in which a relationship is created for a plurality of users so as to have a certain relationship, for example, for a plurality of users in one family, the same or overlapping vehicles for one or more users, the system of this example. This may be the case, or the case where one or more vehicles used by one business operator are used by a plurality of employees or a plurality of related parties within the one business operator. In this case, since the account payment means and the vehicle used are common within such a certain related party, information may be shared by those having such a certain relationship within a certain range. In order to realize such a situation, the user IDs of certain related parties are associated and stored, and if there is such an association, part or all of the user information related to other user IDs is viewed or changed. May be configured to allow In this case, in the user ID, for example, a user ID to be a representative and a user ID for normal use are separately input, recorded, and managed so that the ID related to the representative can be managed. It may be possible to set the authority to view or change the user information when the user information related to the representative is accessed by the user ID having a certain relationship with the user ID related to the representative.

The user information management unit may acquire, store, and manage a part or all of the user information input from the user terminal from the user terminal.

The user information management unit may cooperate with a third-party system in managing payment means for each user. For example, in the case of a car share or a rental car, the user information management department cooperates with a system that manages the car share or the rental car (sometimes referred to as a "third-party management system" in the documents of the present application). You may decide where to bill the parking fee.

For example, when acquiring information that identifies a user in a specific time zone in association with information that identifies one vehicle from a third-party management system that is a system such as car sharing or rental car, use the image described later. When the department acquires information that identifies a certain vehicle in a specific parking area at a specific time zone, it corresponds to the information that identifies such a vehicle, identifies the user in that time zone, and responds to such user. And user history information may be generated. In this case, in the system of one example, when the image utilization unit acquires the information for identifying a certain vehicle, the information for identifying the vehicle is transmitted to the third party management system, and in response to this, the first Information that identifies a user may be acquired from a third-party management system, and the usage history may be updated for such user. In this case, in the system of one example, in order to acquire only the information for identifying the vehicle indicating that the parking act has been performed from the third party management system, the information in the necessary range is acquired and the user's information is obtained. It has the advantage of improving privacy. In this case, the timing of transmitting information identifying one or more vehicles indicating that such parking has been performed from the system of one example to the third party management system may be various, and parking may be performed. It may be immediately after the act, every predetermined period, or at a predetermined time for specifying the payment method.

On the other hand, from such a third party management system, information for identifying a user associated with information for identifying one or more vehicles managed by such a third party management system may be transmitted in advance to an example system. ..

Further, as another aspect, the payment may be configured so that the system of the example requests the billing in the third party management system instead of billing each user in the system of the example. For example, when the image utilization unit of an example system acquires information for identifying one or more vehicles indicating parking behavior, the information for identifying such vehicles is transmitted to a third-party management system and each vehicle is used. Information on each parking fee corresponding to the above may be transmitted to a third-party management system. In this case, the example system may send a relevant image that estimates each parking activity, and may include time information indicating the duration of parking, such as, for example, parking start time, parking end time, parking lot entry time. , Parking lot entry time, etc. may be transmitted. It should be noted that such information may be sequentially transmitted at the stage when the parking action is detected. In this case, there is an advantage that the third-party management system can specify the parking fee by the end of the rental of the vehicle and can charge the rental user of the vehicle at the end of the rental. On the other hand, in the case where payment is made by credit card or account payment at predetermined intervals such as car sharing, the above-mentioned transmission may collectively send information at predetermined periods or at a predetermined time. In such a configuration, one example system has the advantage of being able to collect information about parking fees for vehicles managed by a third party management system. It should be noted that payments from a third-party management system such as a car share or a rental car to the administrator of one system may be processed by the system of this example as a part of the payment, or other payments. It may be processed by the management system.

The user information management unit may acquire, store, and manage image-based information from the image utilization unit. For example, the user information management unit acquires information for identifying a vehicle from the image utilization unit based on the captured image, and uses the information for identifying the vehicle to identify the vehicle in the user information management unit. You may specify the vehicle to be used that is associated with the information that identifies the vehicle in the user information management unit by comparing with the information to be used. In this case, the parking history information associated with the image may be stored in the parking history associated with the vehicle used.

The user information management unit may have a function of calculating a parking fee. The parking fee may be calculated for each predetermined period, or may be calculated for a period specified by the user terminal or the administrator terminal. In addition, the user information management unit may calculate the total parking fee for a predetermined period or a designated period in response to a new parking act.

The user information management unit may perform billing processing to the user in response to a predetermined time, a predetermined period, and a predetermined event. The billing process may be a billing of the parking fee or the total amount of the parking fee at a predetermined time, a predetermined period, or a predetermined event. Further, in the billing process, the system of this example has a billing processing function, and the billing process may be performed directly to the user, or the billing process may be performed in cooperation with another system. When linking with other systems, the system of this example may transmit information used for billing to such other systems. For example, it may be information that identifies the user, the total amount of charges charged by the user, part or all of the parking history information charged by the user, and the like. The other system may be an account payment relationship at a financial institution, a system related to a credit card, a debit card, or the like, and is not limited to these, and may be a system corresponding to a payment method. ..

2.2.2. Image utilization unit The image utilization unit has a function of utilizing an image. The image utilization unit may have a function of acquiring an captured image. The image utilization unit may acquire images captured from one or more image pickup devices installed in association with the parking lot. The management system including the image utilization unit may be directly or indirectly connectable to such one or more imaging devices. In the document of the present application, the image may be a term indicating a superordinate concept of a still image, or may be a term indicating a superordinate concept of a moving image. In addition, the image may be a superordinate concept of various imaging modes such as time lapse.

The image utilization unit may associate the captured image with information indicating the captured temporal information. The means related to the imaging time may be various methods. The imaging time point may be associated and stored as a database in association with the captured image, or may be associated by a method of embedding the imaging time point in the captured image. The information indicating the time information captured may be the date of the image, the time of the image, the date of the image, and the time of the image.

The image utilization unit may associate the captured image with information indicating the captured position. The information indicating the position of the image pickup may be information that can identify the parking lot of the image pickup device that captured the image, or information that can specify the location where the image pickup device is installed in the parking lot. It may be information that can specify the location of imaging by being associated with other information.

The image utilization unit may employ various means so that the captured image is not tampered with. For example, the image utilization unit may adopt a means for preventing falsification of the date and time information associated with the captured image. For example, the image utilization unit may adopt a method of acquiring a time stamp for information on the date and time associated with the captured image, or may adopt a method of storing it in the blockchain.

In the above description, the image utilization unit has a function of associating information indicating an imaging time point and an imaged position with the image utilization unit itself, but such an imaging time point and an imaged position are the captured image and the image utilization. It may be obtained that the image is associated with the outside, and the information indicating the time of imaging and / or the imaged position is associated with the image.

The image utilization unit has a machine-learned identification function. The machine-learned identification function may have a function capable of identifying a predetermined object.

Various artificial intelligence techniques may be used for machine learning. Artificial intelligence technologies include, for example, machine learning technologies such as neural networks, genetic programming, functional logic programming, support vector machines, clustering, regression, classification, Bayesian networks, reinforcement learning, expression learning, decision trees, and k-means clustering. May be used. In the following, an example using a neural network will be used, but the present invention is not necessarily limited to the neural network.

As the machine learning technique using the neural network, a deep learning (deep learning) technique may be used. This is a technique that makes it possible to generate an output corresponding to an input even for an unknown input by learning the relationship between the input and the output using a plurality of layers. There are supervised and unsupervised methods, either of which may be applied. In the case of the supervised method, the learning image and the attribute information related to the learning image are associated with each other, and machine learning is performed using these as learning data.

The learning algorithm itself when using the deep learning technique may be a known one. Further, as the program used in the deep learning technique, an open source program may be used, or a program obtained by modifying them as appropriate may be used.

Further, the machine-learned function may have a learning function. The learning function may be a function that can further learn about an identifiable object. For example, it may be a function capable of learning information related to the license plate and improving the identification function of the license plate. Such a learning function may be in a mode in which the program or its parameters are downloaded and installed.

The machine-learned identification function may have a function of acquiring information for identifying such a vehicle from an image including a vehicle. For example, information for identifying a vehicle may be acquired by acquiring information relating to such a license plate from an image including a license plate of the vehicle. In addition, instead of the information related to the license plate of the vehicle, or by acquiring the information on the vehicle type, color, and / or size of the vehicle in addition to the information on the license plate, there is a possibility of identifying the vehicle. You may obtain information that enhances the information.

For example, the image utilization unit may include two types of machine-learned identification functions. The identification device A, which is the first identification function, has a function of identifying and cutting out a portion corresponding to a license plate from an image. The identification device A inputs an image related to the vehicle. The image relating to the vehicle may be an image including the license plate as a part of the image. The image relating to the vehicle may be an image taken from the front of the vehicle and / or an image taken from the rear of the vehicle.

The identification device A may have an object detection model. The identification device A may be a machine-learned identification device specialized for a license plate. Examples of the machine-learned identification device include, but are not limited to, Yoro, SSD, and the like.

The identification device B, which is the second identification device, has a function of inputting a portion corresponding to the license plate specified by the identification device A and outputting information related to the license plate. The information related to the license plate may include, for example, the type of license plate such as a light vehicle, a motorcycle, a commercial vehicle, a passenger car, a foreign number, and the like. In addition, the information related to the license plate may include the area name, classification number, hiragana, and / or serial designation number of the license plate. For example, FIG. 9A is an example of the area name, classification number, hiragana, and / or serial designation number of these license plates.

The identification device B may specify the area name for each individual character, or may specify the area name with a plurality of characters. The former means each character such as tsu, ku, ba, and so on. The latter may be "Tsukuba", "Iwaki", or the like. In addition, the identification device B may specify the probabilities for possible candidates for these plurality of characters. For example, in the identification device B, from the image related to the license plate, as the area name, "Tsukuba" is 97%, "Iwaki" is 2%, etc., and the number plate is currently limited to 116 candidates as the area name. Therefore, it may be possible to identify a region name that is likely to be among these limited candidates. In this specifying process, probabilities may be generated for at least a part of the candidates.

In this way, by targeting a plurality of characters, compared to specifying by individual characters, for example, even if one character is unclear, a combination of three characters can be selected from candidates. By selecting it, there is an advantage that the information related to the license plate can be output with high accuracy. In particular, the license plate is provided in a position where it comes into contact with the outside in front of and behind the vehicle, so it is dirty due to dust and dirt, and it is also dirty due to mud splashes that occur as the vehicle moves, so it is completely different from a clean license plate. It may look different. Even if it can be recognized by humans, it has the advantage of being able to output information related to the license plate with high accuracy even for such irregular stains. In the above, the area name has been explained, but similarly, for the type, the probability for each of the light vehicle, motorcycle, commercial vehicle, passenger car, and foreign number is generated, and then the type is specified. You may. Similarly, the same processing may be performed for the classification number, hiragana, and serial designation number.

As a related technology, there is a technology using object detection (for example, https://monoist.atmarkit.co.jp/mn/articles/1908/28/news041.html), but such related technology is simply described above. The "object recognition" in the identification device A is used, and the "classification model" is not used. Since such a related technique uses object recognition, there are problems that the model size is generally large and the processing load is heavy, and it is difficult to improve the accuracy due to a detection error. Here, the detection error refers to, for example, the case where one of the three characters is determined to be unclear. On the other hand, in the above-mentioned identification device B, especially when a "classification" model for a region name, a classification number, etc. is used, the model size can be made smaller than the object recognition, and the model size can be made smaller than the object recognition, and the judgment is not unclear. There is an advantage that it is possible to generate probabilities such as area names and classification numbers that have similar character expressions. In particular, by applying the machine-learned identification device using the classification model in the machine learning model to the license plate containing Chinese characters in the area name by the above method, the license plate of the area name including Chinese characters can be applied. It is possible to classify.

As described above, the method of processing the information related to the license plate has been described separately for the identification device A and the identification device B, but the above may be processed by one identification device, or three or more different types. It may be realized by the identification device of.

An example system is
An acquisition unit that acquires an image including the captured license plate,
Using a machine-learned identification device based on a classification model, a generator that generates information related to the license plate from an image including the captured license plate, and a generator.
It may be a system including.

An example system is
An acquisition unit that acquires an image including the captured license plate,
Using a machine-learned identification device based on a classification model, the image including the license plate captured, including the area name, type, classification number, hiragana, and / or serial designation number related to the license plate. A generator that generates information related to the license plate,
It may be a system including.

An example system is
An acquisition unit that acquires an image including the captured license plate,
Using a machine-learned identification device based on a classification model capable of classifying a plurality of kanji, from the image including the license plate imaged, the license plate is related to the license plate including the area name including a plurality of kanji. A generator that generates information and
It may be a system including.

Further, the identification device that outputs the information related to the license plate may be machine-learned. Such machine learning is performed using the teacher data associated with the image containing the license plate and the type, area name, classification number, hiragana, and / or sequence designation number associated with the image. You can. Here, it may be difficult to collect image data including a large amount of license plates that enables machine learning. In particular, when collecting license plates by imaging a vehicle with a fixed imaging device, it is practically very difficult to collect learning data on a scale of several thousand for each of 116 area names, and it is tentatively collected. Even if it can be done, it is a heavy burden to give a correct answer label to the number of sheets of that scale. Therefore, prepare an "image generator" program that looks like a license plate, and generate an image according to the label input such as the desired area name or classification number. With this image generator, teacher data capable of machine learning can be created. However, the generated image does not always match the appearance of the actual license plate and is "too beautiful". The AI model trained with this overly clean image is not determined to be the same when applied to a dirty / blurred license plate in the real world.

Therefore, the inventors applied a method of artificially generating an image including a license plate. That is, first, 1) an image generator that generates a license plate image including the corresponding information by inputting information related to the license plate such as type, area name, classification number, hiragana, and / or serial designation number as input information. It is used to generate information associating such an image with information relating to the corresponding license plate. Since such generation is the execution of a program, the burden is small. Next, 2) prepare a certain number of images including actually captured license plates. In such an image, information related to the license plate such as type, area name, classification number, hiragana, and / or serial designation number may be biased. For example, a specific area name may be missing, or a specific hiragana may be missing. Such information may be, for example, an image generated by a fixed imaging device, a drive recorder, or the like. A technique for learning using these images 1) and 2) may be used. In particular, Japanese license plates have the advantage of solving the difficulty of collecting images due to the inclusion of region names by using both 1) and 2) above.

In addition, in the generation of the image of 1) described above, by generating an image closer to the license plate image actually captured (sometimes referred to as a "pseudo-captured image" in the document of the present application), further machine learning can be performed. It has the advantage of improving accuracy. The above-mentioned information related to the license plate such as the type, area name, classification number, hiragana, and / or serial designation number is used as the input information, and the input information corresponds to the uniform license plate appearance template. The image generated by the image generator that generates the image included in the position (sometimes referred to as "simple generated image" in the document of the present application) is the image of the license plate actually captured (in the document of the present application, the "real image". ”), For example, the following viewpoints can be mentioned.

The first is the dirt that is imaged. The stains to be imaged may be stains on the license plate itself to be imaged and / or those caused by imaging the license plate. In the latter case, the camera or lens of the image pickup apparatus itself for imaging may be dirty, or the window glass between the image pickup apparatus and the license plate to be imaged may be soiled. Further, in the latter case, for example, when an image pickup device installed inside the vehicle is used to image the outside of the vehicle, the license plate is imaged by the dirt on the window glass of the vehicle or the image pickup device installed inside the protective case. If it is, the protective case may be dirty.

The second is caused by movement. The vehicle to be photographed may be imaged while moving, or the vehicle equipped with an imaging device such as a drive recorder may be moving. As described above, what is caused by the movement may be caused by the movement of the vehicle including the license plate to be imaged or the movement of the imaging device. What is produced by such movement may be, for example, an image generated by blurring or out-of-focus.

The third is the influence of shadows. The number plate is often mounted on the lower part of the vehicle, in which case the bumper of the vehicle or the shadow of the lower edge of the vehicle may appear on the number plate. That is, the sunlight may partially cast a shadow on the number plate due to the bumper of the vehicle or the lower part of the vehicle. For example, the upper part of the number plate becomes dark due to the influence of the shadow, and the lower part of the number plate becomes bright due to the influence of the shadow. In addition, when the sunlight causes a partial shadow on the number plate due to the bumper of the vehicle or the lower part of the vehicle, the boundary between the shaded part and the non-shadowed part is the bumper of the vehicle or the low part of the vehicle. Since it is a boundary between departments, it is often in a straight line. The line indicating a substantially straight line of such a boundary may or may not be parallel to the lower edge of the number plate.

These are just examples and may differ for other reasons. An example of a real-life captured image is FIG. 9B. In this figure, although it is a generated figure, the last number is blacked out from the viewpoint of privacy. Since such license plates are real, the lower right part is painted black from the viewpoint of privacy. As a means for reducing the difference between the simply generated image and the actually captured image caused by these reasons, for example, with respect to the simply generated image or the above-mentioned uniform license plate appearance template, there are various methods. Image processing may be applied. For example, blurring and filtering may be applied. FIG. 9C is an example of a pseudo-captured image generated by the image generator created by the inventors. In this figure, although it is a generated figure, the last number is blacked out from the viewpoint of privacy. The pseudo-captured image may be an image that imitates a part of the three viewpoints as a difference between the above-mentioned simply generated image and the actual captured image, or an image that imitates all the viewpoints.

In the case of a pseudo-captured image including the above-mentioned third shadow imitation, the pseudo-captured image may include a substantially straight line. Further, such a substantially straight line may indicate a boundary between shadows. Further, the substantially straight line may be a line in the left-right direction of the number plate. Further, the substantially straight line may be one or more. Further, the brightness above the substantially straight line may be brighter than the brightness below the substantially straight line.

As described above, when the image generator generates a pseudo-captured image that looks similar to the actual captured image, there is an advantage that the burden of image generation is small and data for machine learning can be generated. Further, when the image generator generates data in which the generated pseudo-image is associated with the information related to the license plate in the pseudo-imaging, the data for machine learning as teacher data is generated with less burden. There are advantages that can be done.

An example system is
The first acquisition unit that acquires the information contained in the license plate including the type, area name, classification number, hiragana, and / or serial designation number,
The second acquisition part that acquires the image including the appearance of the license plate,
A first generation unit that generates a license plate information-containing image using the information contained in the license plate and the image.
It may be an image generation system relating to a license plate provided with.

Each piece of information in the type, area name, classification number, hiragana, and / or serial designation number in the information contained in the license plate may be selected based on a predetermined order. Instead of such a configuration, each information in the type, area name, classification number, hiragana, and / or serial designation number in the information contained in the license plate may be selected by using a random number function. Each information of the type, area name, classification number, hiragana, and / or serial designation number selected by using the random number function may be uniformly selected.

The image including the appearance of the license plate may be an image of a template of the appearance of a uniform license plate. Further, instead of such an image, the image including the appearance of the license plate may be an image imitating reality. The image that imitates reality may be an image generated because of the above-mentioned stains to be imaged and / or those caused by movement.

The first generation unit may process the image including the appearance of the license plate into an image imitating reality. The process for realizing such image contamination may be a process of various methods other than the above-mentioned blurring and filtering.

The license plate information-containing image generated by the first generation unit may be an image of learning data for machine learning.
In addition, the first generation unit may perform a process of associating the license plate information-containing image with the information contained in the license plate used for generating the license plate information-containing image. Such associated data may be data used for machine learning. Also, such associated data may be used for machine learning.

When processing is performed to reduce the difference between these simply generated images and the actual captured image in terms of the appearance of the image, the image as learning data used for machine learning becomes closer to the actual captured image, and the machine-learned identification device. It is possible to improve the accuracy by double the application of.

Further, in machine learning of the identification device that acquires the information related to the license plate, a method of dealing with the inclination of the license plate may be applied. The image of the license plate to which the machine-learned identification device is applied is rarely taken from the front of the license plate due to the installation location of the image pickup device. That is, the license plate is often installed in the lower part of the vehicle, but it is rare that the image pickup device is installed in a position where the lower part of the vehicle can be imaged from the front, which is common. In many cases, the image pickup device is installed at a height that is not affected by the traffic of people, and the image pickup device is installed at a predetermined depression angle so as to look down on the vehicle. Therefore, it is rare to image the license plate installed in the lower part of the vehicle from the front, and the image to which the machine-learned identification device is applied to acquire the information related to the license plate is the number. The image is often taken by the image pickup device at an angle equal to or higher than a predetermined angle from the perpendicular to the plane of the plate.

Therefore, the image utilization unit may perform machine learning using the information indicating the position of the license plate in the image obtained by capturing the license plate. For example, as data for machine learning, using an image, information indicating the position of the license plate, and information related to the license plate contained in the image, the image and information indicating the position of the corner of the license plate are used. And machine learning may be performed in relation to the information related to the license plate contained in the image. Here, the information indicating the position of the license plate may be information indicating the position of the license plate in the image obtained by capturing the license plate, and may be, for example, a function or coordinates indicating the position of the license plate. The function may indicate, for example, the upper and lower sides and the left and right sides of the license plate, and the coordinates may be, for example, the coordinates of the four corners of the license plate.

That is, in the machine learning identification device, as input of machine learning, one image, information indicating the position of the license plate in the one image, and information related to the license plate in the one image are related. You may also let them do machine learning. Then, the captured image to be the target for actually acquiring the information related to the license plate may be input to the machine-learned identification device to acquire the information related to the license plate. In this case, the machine-learned identification device may or may not acquire information indicating the position of the license plate in the captured image for which the information related to the license plate is actually acquired. Good. When the corner is acquired, the information indicating the position may be used for correction of the captured image. Further, even when the corner is not acquired, there is an advantage that the accuracy of the information related to the license plate to be output can be improved by inputting and learning the information in the image in machine learning. The information in the image to be input in the above-mentioned machine learning includes the above-mentioned information on the corners of the license plate (for example, information on the four corners as information indicating a part or all of the position of the license plate) and / or. , Information indicating the vehicle type of the license plate (for example, the color of the license plate, the shape of the license plate, etc.) may be included.

If the image utilization unit has a machine-learned identification function and can acquire information for identifying the vehicle from the captured image, there is an advantage that the vehicle can be identified. As described above, such information is stored in association with the position and / or time of the captured image, so that there is an advantage that information for specifying the position and / or time of a specific vehicle can be acquired. Therefore, for example, by acquiring information that identifies a specific vehicle in a specific parking lot, it can be shown that the vehicle has used the parking lot, and based on such information, the vehicle can be used. You may charge the parking fee to the person who used the parking lot.

In particular, when the information for identifying a vehicle for a certain vehicle can be acquired at the first time point and the second time point from the image captured by the image pickup device associated with the specific parking lot. From such a first time point to such a second time point, it is estimated that the vehicle has used the specific parking lot, and the image utilization unit tells the user related to the vehicle that the vehicle has used the first parking lot in the parking lot. It may have a function to add to the usage history of the parking lot from the time point to the second time point. In this case, as explained elsewhere in the documents of the present application, if a plurality of payment methods are associated with the user's vehicle, the conditions of each payment method are satisfied. Such parking fees may be associated with the means of payment.

In addition, in order to specify the time of entry to the parking lot and the time of entry, the image utilization unit is based on a predetermined image pickup device that captures an image that is a source for acquiring information for identifying the vehicle. May be specified.

For example, in a passage where the traffic direction of the vehicle is fixed, when the imaging device can acquire information for identifying the vehicle related to the vehicle in the passage, the entry of the vehicle is performed according to the traffic direction of the vehicle. Alternatively, participation may be determined. For example, when the traveling direction of the vehicle in the first passage is the entry direction to the parking lot, the information for identifying the vehicle is based on the image captured by the imaging device capable of imaging the vehicle on the first passage. If the vehicle can be obtained, it may be determined that the time when the image is captured is the time when the vehicle enters. Similarly, when the traveling direction of the vehicle in the second passage is the entry direction to the parking lot, the vehicle is identified based on the image captured by the imaging device capable of imaging the vehicle on the second passage. If the information can be obtained, the time when the image is captured may be determined as the time when the vehicle participates.

On the other hand, when the passage is a shared entrance / exit to the parking lot, the image utilization unit determines the entry or exit of the vehicle by using the traveling direction of the vehicle in the image and / or the image of the vehicle. Good. For example, when the imaging direction of the imaging device is a vector, the vector decomposed in the direction parallel to the ground is obtained in consideration of the case where the vector is decomposed into the direction perpendicular to the ground and the direction parallel to the ground. , In the case of a vector in which the direction of entry from the parking lot is positive, since the image is taken in the direction of entry from the parking lot, the height of the vehicle within a predetermined period in the time-series image of the imaging device. If the width and / or the width increases, it may be determined that the vehicle is approaching the parking lot and / or the time when the parking lot is entered. On the other hand, in the time-series images of the image pickup device, if the height and / or width of the vehicle decreases within a predetermined period, the vehicle will move away from the parking lot, and therefore, when the vehicle enters the parking lot. May be determined. Here, instead of or / or the width of the vehicle, the height and / or the width of a specific part or part of the vehicle may be used. Specific parts of some of such vehicles include, but are not limited to, headlamps, vehicle tires, steering wheels, the size of the person's face when there is a person in the vehicle, and the like. In short, it suffices that the size of the image of the vehicle captured in chronological order increases or decreases as the vehicle moves back and forth.

Further, the image utilization unit may use a specific vehicle in the image. For example, if the driver and / or the steering wheel are identified for the vehicle in the image, it can be recognized as being in front of the vehicle, and as described above, in the case of an imaging device facing the entry direction of the parking lot. , It may be determined that the vehicle is entering. In this case, there is an advantage that the direction of travel can be more specified by certifying a specific object such as a driver or a steering wheel, in addition to simply changing the size in the image.

When the imaging device captures the direction of entry into the parking lot, it may be the opposite of the above. That is, when the imaging direction of the imaging device is a vector, the vector decomposed in the direction parallel to the ground is obtained in consideration of the case where the vector is decomposed into the direction perpendicular to the ground and the direction parallel to the ground. In the case of a vector in which the direction of entry into the parking lot is positive, since the image is taken in the direction of entry from the parking lot, the height of the vehicle within a predetermined period in the time-series image of the image pickup device. If the width and / or the width increases, it may be determined that the vehicle is at the time of entry of the parking lot because it is far from the parking lot. On the other hand, in the time-series images of the image pickup device, if the height and / or width of the vehicle decreases within a predetermined period, the vehicle approaches the parking lot, and therefore, when the vehicle enters the parking lot. May be determined.

In this case as well, as described above, the image utilization unit may use a specific vehicle in the image. For example, if the driver and / or the steering wheel are identified for the vehicle in the image, it can be recognized as being in front of the vehicle, and as described above, in the case of an imaging device facing the entrance direction of the parking lot. , It may be judged that the vehicle participates.

On the other hand, the image utilization unit may use the machine-learned function to specify the traveling direction of the vehicle described above, and specify the entry time and / or exit time of the vehicle. In this case, the image utilization unit may perform machine learning on the image captured by the specific imaging device by using the supervised information of entry or exit. Then, the image utilization unit having such a machine-learned function can determine the entry or exit of the vehicle with respect to one or a plurality of still images on a time series or a moving image of a predetermined period captured by a specific imaging device. May have.

In addition, the image utilization unit determines the entry and / or entry of the vehicle based on the passage in which the traveling direction is defined, determines the entry and / or entry of the vehicle based on the fluctuation of the size of the captured image, and the machine. Part or all of the vehicle entry and / or entry determination using the learned function may be used. In this case, there is an advantage that the accuracy can be further improved by the plurality of determination mechanisms.

The image utilization unit has a function of managing information relating to one or more parking areas. The information relating to the parking area may include full vacancy information, parking lots relating to the parking area, information regarding the parking lot, parking fees, and / or other information. A management system having an image utilization unit may transmit information relating to a parking area to one or more terminal devices.

<Full sky information>
The image utilization unit may generate full sky information about the parking area from images captured by one or more image pickup devices. The image utilization unit has a data structure capable of storing information such as parking, empty, etc. for one or more parking spaces in the one parking area in advance corresponding to one parking area. Good. Depending on the presence or absence of a vehicle at a position corresponding to such one or more parking spaces and the presence of other objects in the captured image, the image utilization unit may perform parking, empty vehicles, or the like for each position or multiple parking spaces. , Etc. may be stored. Here, the determination of parking is determined when the image utilization unit can identify the vehicle at a position in the image corresponding to the parking space by using the machine-learned identification function capable of identifying the vehicle. You may do. Further, in the determination of being empty, the image utilization unit uses the machine-learned identification function capable of identifying the vehicle to determine that the vehicle is empty when the object is not identified at the position in the image corresponding to the parking space. You can. If the image utilization unit detects the presence of an object other than the vehicle at a position in the image corresponding to the parking space by using the machine-learned identification function that can identify the vehicle, other judgments are made. You can. This is because there is some object other than the vehicle and there is a possibility that it cannot be parked. In that case, it may be determined that the vehicle is not empty at least. If the person who parked the vehicle in the next parking space uses the vehicle for a short time, such as using a cart to move the purchased items in the parking lot of a commercial facility, the vehicle is vacant. You may judge that.

FIG. 10A is an example of an image of the parking area. Each parking space in the image captured by the image pickup device installed in the parking lot is schematically displayed by distinguishing between parking and empty vehicles using images. In this figure, vehicles parked in the parking space are surrounded by □ to indicate that they are identified, and the parking space in which the vehicle is parked is indicated by a red circle. On the other hand, the empty car space is indicated by a blue circle. Here, each of the four-digit numbers duplicated for each vehicle indicates an ID indicating each vehicle. The ID indicating such a vehicle may be a unique number that can uniquely identify the vehicle. As shown in this figure, it is not necessary to distinguish between parking and empty vehicles for a part of one or more parking spaces such as a place far from the imaging device. There may be a part that cannot be identified by an image due to a distant place or the presence of a shield. Further, FIG. 10B is another example of an image obtained by capturing the parking area. In this figure, a line surrounding the vehicle is displayed to indicate that the vehicle is identified. It should be noted that such a line identifies not only the front and rear of the vehicle but also the side of the vehicle. Such identification makes it possible to track a moving vehicle. Also, the truck identifies the driver's seat in front, not the loading platform.

Here, some techniques for the image utilization unit to acquire full / empty information from one or a plurality of images will be described. The image utilization unit may use one or more techniques described below. The image utilization unit may use only one technique described below, or may use a technique other than the technique described below.

First, the image utilization unit identifies and identifies each parking space for the parking area. The image utilization unit may automatically identify each parking space from one or more images, or may identify it with artificial support. In particular, when one or more images are captured by a fixed imaging device installed in the parking lot, the imaging field of view that is the target of the image is fixed and does not move, so the parking space is provided with artificial support. Even if it is specified, there is not much burden, and it is possible to record reliable information on each parking space. That is, an example system may have information associated with an image captured by a fixed imaging device, with one or more specific points in the image as parking spaces. For example, by associating one parking space with a plurality of specific coordinates in one image, it is associated with a plurality of parking spaces with a plurality of coordinates for each of a plurality of specific locations in one image. , You may remember.

Next, the image utilization unit identifies one or more vehicles in the image. The vehicle may use pattern matching technology or may use the identification function in machine learning. When the identification function in machine learning is used, there is an advantage that the identification function can be improved, particularly when the machine-learned ones for various vehicles are used.

Next, the image utilization unit compares one parking space specified in advance in the parking area with the specified vehicle. If a vehicle corresponding to one parking space is specified, the parking space may be determined to be parked, and if a vehicle corresponding to one parking space is not specified, the parking space may be determined to be empty.

In this case, the second number is less than the first number for the first number of parking spaces within a predetermined range, such as when only one vehicle is specified corresponding to a plurality of parking spaces. A number of vehicles may be identified. For example, when the imaging device is installed at a depression angle within a predetermined angle (an angle parallel to the ground toward the ground), the first number of parking spaces and the second number less than the first number. May be associated with as many vehicles as. In this case, among the first number of parking spaces, the vehicles may be associated with each other in a one-to-one manner in the order of proximity to the image pickup device, and the first number parking space is obtained by subtracting the second number from the first number. For the remaining parking space, it may be recorded that the parking situation is unknown. This is because such a place may simply not be visible to the vehicle from the image pickup device and may not be empty. In addition, such a place may be treated as parked when displaying the inside of the parking area as full vacancy information and displaying either parked or vacant car. On the other hand, the parking space recorded as unknown may or may not be treated as parked when statistical information is generated based on the full vacancy information. In the latter case, when the numerical information is such that the parked vehicle is treated as 1 and the empty vehicle is treated as 0, the unknown value may be 0.5 or the like. Therefore, for example, in a parking area with 100 parking spaces, when 42 is parked, 52 is empty, and 6 is unknown parking space, information that 52 is empty may be generated as full empty information. , 52 out of 100 may generate information that the vehicle is empty, or 55% may generate information that the vehicle is empty.

Further, the image utilization unit may generate full / empty information for a plurality of parking spaces in a predetermined area unit of the parking space, together with or in place of the above-mentioned technology. For example, a vector such as a straight line may be set for a plurality of parking spaces, and full / empty information may be generated using the number of identified vehicles that overlap with the vector. For example, the area units of 10C1 and 10C2 in FIG. 10C can be mentioned. In this case, the number of such a plurality of parking spaces may be specified in advance and compared with the number to generate full / empty information. Here, the number set in advance for each vector may be a number set in advance through artificial support as described above. In this case, the number of parking spaces in such a row is associated with a vector set to overlap for a row of parking spaces consisting of multiple adjacent parking spaces. Numbers may be used and compared to the number of vehicles recognized as vehicles obtained from the image.

In addition, regardless of whether the method using such a vector is used or not, the maximum number of vehicles that can be parked and the number of vehicles actually specified in a predetermined area unit of the parking space are used. , May be used to generate full vacancy information such as the number of parked vehicles and the vacancy rate. Since it is a predetermined area unit of the parking space, there is an advantage that the calculation can be performed with a wider width. Examples of a predetermined area unit of such a parking space include the area units of 10D1 and 10D2 in FIG. 10D. In this case, the maximum number of cars that can be parked may be the number set with artificial support as described above.

Further, the number of vehicles may be calculated by multiplying the number of vehicles specified in the predetermined area unit by a correction coefficient. That is, the number generated by multiplying the specified number of vehicles by a predetermined correction coefficient may be the number of vehicles parked in such a predetermined area unit. In this case, the correction coefficient may be artificially set according to an empirical rule, or may be mechanically set according to an actually measured value. When the image pickup device is fixed, the image pickup field of view is also fixed, so that the appearance of vehicles parked in the same parking area tends to be constant, and such a correction coefficient enables a constant prediction. There are advantages.

In addition, with or in place of the above techniques, the number of vehicles entering and exiting a predetermined parking area may be measured. That is, the number of vehicles traveling in a predetermined parking area is determined by using one or a plurality of images captured by an imaging device capable of capturing an image of a passage that can enter and exit the predetermined parking area. The number of vehicles in the parking area may be estimated by subtracting the number of exiting vehicles from the number of entering vehicles. This is because the aisle has a better view than the parking space, so that the number of vehicles may be calculated more accurately.

In addition, the vehicle may not be detected by such an imaging device, such as when a vehicle continuously passes through a passage that is imaged by an imaging device capable of imaging a passage that can enter and exit a predetermined parking area. The number of vehicles may be corrected in the early morning or at night. For example, when the parking area is locked at night to prevent parking, the number of parked vehicles may be reset to 0 at that time. Even if the parking area is not closed, if there are few vehicles parked at night or early in the morning, false detection of the number of vehicles will decrease. Therefore, depending on the number of vehicles in the parking area at a predetermined time such as at night or early morning, parking will be performed. The number of vehicles may be corrected. In such a case, for example, both the measurement of the number of parking lots based on the image of the inside of the parking area and the measurement of the number of entrances and exits based on the image of the passages that can enter and leave the parking area are used. And may generate full vacancy information about the parking area. Here, the image of the parking area and the image of the passage may be the same or different. Further, the image pickup device that images the inside of the parking area and the image pickup device that images the passage may be the same or different.

In the image pickup device for capturing an image as shown in this figure, when the resolution of the image pickup device is about a certain level, the image utilization unit may acquire information for identifying the vehicle for each vehicle from such an image. That is, for example, when the license plate information of each vehicle can be acquired from the high resolution, the information for identifying the vehicle is acquired from the image including the plurality of parking spaces in the imaging field of view, and the vehicle is parked in the parking space. You may identify the vehicle inside. In this case, for example, even if an image pickup device is installed at the entrance / exit of the parking lot and the vehicle entering and / or entering the parking lot is not specified, only the image pickup device that images the plurality of parking spaces is used for parking. It has the advantage of being able to identify the vehicle. In addition, there is an advantage that information for identifying the position of the parked vehicle in the parking area can be acquired.

In this case as well, an imaging device may be installed at the entrance / exit of the parking lot. When an imaging device can be installed at the entrance / exit of a parking lot to identify a vehicle entering and / or exiting the parking lot, the other imaging device can be used to identify a parked vehicle in the parking area. The use of information has the advantage of improving the accuracy of identifying parked vehicles.

The image utilization unit may generate full sky information about the parking area based on the captured image. For example, after generating information such as vacant cars and parked cars for each parking space, the vacant car space ratio can be calculated for one parking area including one or more parking spaces, and information such as the amount of vacant car spaces can be calculated. May be generated.

In addition, the image utilization unit may generate such full-sky information for one parking area by using images captured by a plurality of imaging devices. Here, when a plurality of imaging devices are used, the imaging fields of view may or may not overlap with each other. Further, when the imaging fields of view of a plurality of imaging devices overlap, the full / empty information generated for one or a plurality of overlapping parking spaces can be obtained by collating the information obtained from each image. It has the advantage of improving accuracy.

The image utilization unit may store the information related to the parking lot including the above-mentioned full / empty information in the same information processing device as the image utilization unit, or store it in an information processing device different from the image utilization unit. A configuration may be adopted in which information related to the parking lot is transmitted to different information processing devices, stored in the information processing device, and acquired from the information processing device according to management. The image utilization unit may store and manage the information related to the parking lot in a server, cloud, or the like.

The image utilization unit may generate full / empty information of such one parking area at any timing. Full / empty information may be generated at predetermined intervals, or full / empty information may be generated in response to some event. The predetermined period may be various, such as 5 minutes, 10 minutes, 30 minutes, 1 hour, etc., and the accuracy of the full / empty information is increased by shortening the predetermined period and increasing the frequency. There is an advantage that can be improved. Examples of the event include the case where the entry and / or entry of the vehicle is detected. As described above, from the viewpoint of the possibility of detecting a person other than the vehicle such as a person in the parking space, the full / empty information is generated by using the average value of the results of a plurality of detections within a predetermined period. May be good.

The image utilization unit may transmit the full / empty information about the parking area to the terminal device. The timing of transmitting the fullness information to the terminal device may be various. For example, the fullness information may be transmitted in response to a request from the terminal device, or the fullness information may be transmitted to the terminal device in response to a periodic or specific event without such a request. ..

As a request from the terminal device, for example, the user may request the fullness information at the timing corresponding to the activation of the parking application, or the user specifies a specific parking lot in the parking application. You may request full availability information at the timing. In the former case, the location information on which the parking application is located at the time when the parking application is activated is used to transmit the full / empty information about the parking area associated with the location information to the activated parking application. You can. Further, in the latter case, the full / empty information about the parking area related to the specific parking lot designated by the user may be transmitted to the designated parking application.

As a periodic one, for example, from the past parking usage history of the user, full vacancy information about the parking area where parking can be used may be transmitted at a timing based on the usage history.

Further, the specific event may be, for example, an event for detecting a specific vehicle to be used in the parking area. In such a case, the full / empty information may be transmitted to the parking application of the user ID related to the detected specific vehicle.

In addition, as another specific event, when the use and time of a specific parking area is reserved, in response to such reservation, full availability information about the reserved parking area at the corresponding time is provided. It may be transmitted to the user's parking application related to the reserved vehicle.

In addition, when the management system is directly or indirectly connected to the display device associated with the parking lot, the image utilization unit transfers the fullness information about the parking area to the display device related to the parking lot area. You may send it. The display device may be installed at the entrance of the parking lot and / or in the parking lot. Further, the display device may be one that can be seen from inside the vehicle, and may be a device that can display the situation in the parking lot. Such a display device may be capable of schematically showing a parking area associated with such a display device and displaying the fullness information of the parking area in association with the illustrated parking area. With such a display device, it is possible to provide full / empty information in the parking lot from the entrance to the parking lot and / or the inside of the parking lot.

<Waiting convoy detection function>
In addition, the image utilization unit may generate the number of vehicles in the waiting convoy on the aisle by using one or more images. For example, the image utilization unit may generate the number of vehicles on the aisle using one or more images captured by an imaging device capable of capturing the aisle in the field of view. In this case, the image utilization unit may set an area on the aisle, such as a parking area, and generate the number of vehicles on the aisle. In this case, the technique for generating full / empty information about the parking area described above may be used.

In addition, the image utilization unit may generate information related to the waiting time in the waiting convoy on the aisle. In this case, the image utilization unit may use the parking tracking function described later. For example, the image utilization unit may generate information on the time to wait in the waiting convoy on the aisle by using the distance traveled within the predetermined period on the aisle. In addition, the image utilization unit generates information related to the waiting time in the waiting convoy on the aisle by using the speed prediction described later, the prediction of the appearance position of the vehicle in the image after a predetermined period, and the Optical Flow technology. You can do it. Further, the image utilization unit may measure the moving time of a vehicle in the waiting convoy by measuring the moving time of a specific vehicle on the aisle. For example, the image utilization unit may measure the waiting time of the waiting vehicle line by using the movement time from the first predetermined position to the second predetermined position of the waiting vehicle line on the aisle. The first predetermined position and the second predetermined position may be arbitrarily determined. The first predetermined position and the second predetermined position may be a part in the middle of the waiting vehicle line, or may be the beginning and the end of the waiting vehicle line. For example, the movement time of a vehicle in the waiting vehicle line may be measured by the distance between the first predetermined position and the second predetermined position and a predetermined multiple thereof, or the first predetermined position may be the end of the vehicle line. It may be varied as a tail, and the second predetermined position may be the head of the waiting convoy, or a combination thereof may be used.

The information related to the waiting time may be the time from when one vehicle starts waiting in the waiting line to when it leaves the waiting line, or may be the average time taken for a plurality of vehicles. Moreover, it may be a part of the time taken. For example, the information may take at least 0 minutes.

When the image utilization unit can generate the number of vehicles in the waiting line on the aisle within a predetermined range from the entrance of the parking lot, it has an advantage that it can detect the degree of congestion until entering the parking lot. In this case, it is possible to recommend another parking lot. Therefore, the management system may transmit the information related to the waiting line at the entrance in one parking lot to the user application related to the user terminal related to the vehicle aiming at the parking lot. Such transmitted information may be the number of vehicles in the waiting convoy and / or the expected waiting time in the convoy.

Further, when the image utilization unit can generate the number of vehicles in the waiting line on the aisle within a predetermined range from the exit of the parking lot, there is an advantage that the degree of congestion from the exit of the parking lot can be detected. In this case, the management system may transmit the information related to the waiting line of the exit related to the parking lot to the user application in the commercial facility associated with the parking lot. Such transmitted information may be the number of vehicles in the waiting convoy and / or the expected waiting time in the convoy. In this case, there is an advantage that the convenience of the user in the commercial facility can be improved based on the congestion of the exit of the parking lot.

<Induction function>
The image utilization unit may have a vehicle guidance function. The image utilization unit may have a function of guiding the vehicle to the parkingable area by using the fullness information of the parking space in the parking lot. The image utilization unit may generate information related to vehicle guidance and transmit such information to the user terminal.

The image utilization unit may provide information on all parkable areas, may provide information on some parkable areas, or part of such information or as information related to vehicle guidance. It may provide useful information for moving to a specific predetermined range of parkable areas of all parkable areas.

When the image utilization unit provides information about all the parkable areas, the user has an advantage that the parking area desired to be parked can be selected from all the parking areas.

Further, when the image utilization unit provides information about a part of the parkable area, the user has an advantage that he / she does not have to hesitate from making an excessively large number of selections. The image utilization unit may specify such a part of the parkable area by various methods. For example, the image utilization unit may specify a part of the parkable area by using the fullness information of the parking area, the information related to the guidance, the distance between the parking area and the commercial facility, and the like.

For example, some parkingable areas include a parking area having full vacancy information with a vacancy rate of a predetermined value or more, a parking area having vacancy information having the highest vacancy rate, and an empty vehicle within a predetermined distance from a commercial facility. It may be a parking area having full / empty information whose rate is equal to or higher than a predetermined value. In addition, these partially parkable areas may be specified by a predetermined rule. Further, the application of such a predetermined rule may be specified based on the information obtained from the administrator terminal. In this case, there is an advantage that the manager terminal can include the parking area that the manager wants to park in the information related to the guidance.

Further, the information useful for moving to a specific predetermined range of the parkable area of a part or all of the parkable area may be information that can be used when the vehicle to be used moves to the empty space. For example, the direction of the empty car space such as the area where the empty car space is located in the parking lot, the floor where the empty car space is located in the parking lot having multiple floors such as a multi-story parking lot, or the specific position of the empty car space is shown. It may be information. In addition, information indicating the relative relationship with the empty vehicle space from the position of the vehicle used in real time may be generated. The relative relationship between the empty space and the position of the vehicle used in real time is obtained by the management system acquiring the position of the vehicle used in real time and generating the relative prefectural police in the management system. It may be transmitted to the user terminal, or information about the parkable area is transmitted to the user terminal, and the real-time position information of the vehicle used in the user terminal is used to generate such a relative relationship. You may.

In addition, when the management system is directly or indirectly connected to the display device associated with the parking lot, the image utilization unit relates the information related to the guidance of the vehicle about the parking area to the parking area. It may be transmitted to the display device. The display device may be installed at the entrance of the parking lot and / or in the parking lot. Further, the display device may be one that can be seen from inside the vehicle, and may be a device that can display the situation in the parking lot. Such a display device may be capable of schematically illustrating a parking area associated with such a display device and displaying information relating to vehicle guidance.

Further, when the management system is further directly or indirectly connected to the image pickup device associated with the display device, the image utilization unit provides information relating to the guidance of the vehicle to the vehicle imaged by the image pickup device. May be transmitted to such a display device. Here, the imaging device may include, for example, a vehicle at a position where the display device can be viewed in the imaging field of view. The image utilization unit may acquire information for identifying the vehicle viewing the display device by using the image captured by the image pickup device. The image utilization unit may display the information related to the guidance for the vehicle specified by using the image pickup device on the display device. In this case, the display device installed in the parking lot can display guidance information corresponding to the vehicle viewing the display device.

With such a configuration, for example, when a user inputs a desire to park in a specific parking area in the parking application, the parking area corresponding to the user vehicle is linked with the user vehicle used by the user. There is an advantage that the information related to the guidance to the vehicle can be displayed on the display device. In this case, the guidance information displayed on the display device may differ depending on the vehicle at a position where the display device can be viewed.

In the above description, the case where a certain user wants to park in a specific parking area has been described, but this is only an example, and even when the user does not want to park in a specific parking area. , The display device may display information related to different guidance for each user, and therefore, the image utilization unit generates and transmits information related to different guidance for each vehicle used. May be good.

For example, when a vehicle can travel to a plurality of parking areas from a location where the display device is installed, the plurality of parking areas are provided for a continuous series of vehicles in a position where the display device can be viewed. May be allocated sequentially. For example, when one vehicle is in a position where the display device can be viewed, the image utilization unit generates information related to guidance for one parking area of a plurality of parking areas, transmits the information to the display device, and sends the information to the display device. When displayed, when the next vehicle following one vehicle is in a position where the display device can be viewed, guidance for a parking area different from the one parking area among the plurality of parking areas. The image utilization unit may generate the information related to the above, transmit it to the display device, and display it on the display device. In addition, information related to guidance may be generated so as not to guide the same one parking area among the plurality of parking areas to both consecutive vehicles. In these cases, since continuous vehicles are not guided to the same parking area, there is an advantage that congestion in the parking lot can be alleviated.

<Vehicle tracking function>
The image utilization unit has a function of acquiring information on the movement of a vehicle determined to be the same, that is, the trajectory of the vehicle from a moving image or a plurality of still images on a time series by using a machine-learned identification function (in the document of the present application). , May be referred to as a "vehicle tracking function").

The image utilization unit may have a vehicle tracking function using images acquired from one imaging device, or may have a vehicle tracking function using each image acquired from a plurality of imaging devices. In the latter case, the determination of the identity of the object between the images captured by the plurality of imaging devices may use the prior art, or the identity of the vehicle, the overlap of the visual fields of the plurality of imaging devices, and the image. The relevance of the time when the vehicle in the vehicle was imaged, etc. may be used. Regarding the overlap of the visual fields, when the imaging visual fields of the imaging devices overlap, the identity of the vehicles may be determined by using the vehicles identified at the same position and at the same timing in the overlapping visual fields.

When the image utilization unit has a vehicle tracking function, the vehicle is identified from the image captured by another imaging device even when the information for identifying the vehicle cannot be acquired from the image itself to which the vehicle tracking function is applied. If the information to be used can be obtained, there is an advantage that the vehicle in the image to which the vehicle tracking function is applied can be identified in conjunction with the information of the image captured by the other imaging device.

As described above, the image used for identifying the vehicle and the image used for acquiring the information on the trajectory of the vehicle may be the same image or different images. Further, the imaging device that captures the image used to identify the vehicle and the imaging device that captures the image used to acquire the information on the trajectory of the vehicle may be the same imaging device. However, different imaging devices may be used. For example, the image used to identify the vehicle, the image used to obtain information on the trajectory of the vehicle as it moved into the parking space, and that the vehicle was parked in the parking space. The images used to determine the above may be all the same image, some images may be the same and other images may be different, or all images may be different. In addition, an imaging device that captures an image used to identify the vehicle, and an imaging device that captures an image used to acquire information on the trajectory of the vehicle as the vehicle moves to the parking space. The imaging device that captures the image used to determine that the vehicle is parked in the parking space may all be the same imaging device, or some imaging devices are the same and other imaging devices. May be different, or all imaging devices may be different.

The image utilization unit may use various methods as the vehicle tracking function, and for example, a prediction technique for a vehicle may be used. As the vehicle movement prediction technique, a technique of predicting the speed of the vehicle and predicting the position of the vehicle after a predetermined time based on the current position of the vehicle and such prediction may be used. As a technique for predicting the speed of the vehicle, for example, the speed of the vehicle predicted from a plurality of predetermined images on a time series in a moving image may be used, or a Kalman filter may be used. For example, the speed of the vehicle is specified from the position of the same vehicle in each image in a predetermined number of frames such as in the past 5 frames in the video, and the vehicle after a predetermined period when the speed is maintained. The vehicle may be tracked by predicting the position of appearance.

In addition, the image utilization unit may use the Optical Flow technology as the vehicle tracking function together with or in place of the above-mentioned functions. Even in this case, since it is possible to predict the movement of the vehicle after a predetermined period of time by expressing the movement of the vehicle by a vector, the position of the vehicle after a predetermined time can be predicted and the accuracy of the vehicle tracking function can be predicted. There is an advantage that can be improved.

In addition, the image utilization unit may realize the vehicle tracking function by using the color of the vehicle together with the above-mentioned functions, together with other technologies, or independently. The image utilization unit may also use vehicle color to improve the accuracy of the vehicle tracking function, in combination with the above functions or with other techniques. For example, the image utilization unit stores the color X information of the vehicle to be tracked at a certain time point or within a certain frame, and stores the vehicle to be tracked within a frame after a predetermined time elapses or a predetermined number of moving images. In the specifying process, it may be determined whether the color of the candidate vehicle to be tracked matches the color X of the vehicle, and the candidate vehicles to be tracked having different vehicle colors may be excluded from the tracking target. Since each vehicle has various body colors, there is an advantage that the accuracy of the vehicle tracking function can be improved by excluding clearly different vehicles by using such a color.

Further, the image utilization unit may determine the identity of the vehicle by using the deep learning machine learning function together with the above-mentioned function, together with other techniques, or independently. In addition, the image utilization unit may improve the accuracy of the vehicle tracking function by using the deep learning machine learning function together with the above-mentioned functions or in combination with other techniques. For example, the image utilization unit stores the vehicle type information X identified by using the deep learning machine learning function for the vehicle to be tracked at a certain time point or within a certain frame, and after a predetermined time elapses or in a predetermined video. In the process of identifying the vehicle to be tracked in the frame after the number of sheets, it is determined whether the vehicle type information of the candidate vehicle to be tracked matches the vehicle type information X of the vehicle, and the candidate vehicle to be tracked with a different vehicle type is determined. , You may remove it from the tracking target. The vehicle type information may include the color of the vehicle, the vehicle type of the vehicle, the type of the vehicle, and the like. Since there are vehicles of the same color and model number in the parking area, there is an advantage that tracking of different vehicles can be prevented.

The image utilization unit may have a function of providing the captured image to the user. By being provided to the user, the user has the advantage of being able to obtain information related to parking.

As described above, the image utilization unit has a function of acquiring information for identifying a vehicle including a license plate from an image, and an imaging device that has captured such an image enters and / or exits a specific parking lot. If it is installed in a reasonably necessary place, if the information indicating a specific vehicle is obtained by such imaging, it means that the specific vehicle in such parking lot existed at the time when such an image was taken. Meaning, since such existence leads to the billing of the parking fee, the image from which the information for identifying the vehicle is obtained indicates the existence of the vehicle to the user. Therefore, the image that is the source of the information for identifying the vehicle is stored and managed in the image utilization unit, and by providing the image to the user, the user can also understand the parking behavior. is there. As the information that can be provided to such a user, the image may capture only the license plate, but may also capture information around the license plate. For example, the whole and / or part or all of the rear part of the vehicle including the license plate may be imaged, or the person who gets in the vehicle may be included in the image. In the case of an image including a person getting in the vehicle, the user has an advantage that the memory of the parking act can be more aroused.

Further, as shown in FIG. 10 described above, in the case of an imaging device capable of imaging a plurality of vehicles from a certain distance, the entire image may be used. In this case as well, the user has an advantage that it is easy to recall the memory of parking in the parking lot. In such a case, the place other than the target vehicle to be used may be provided to the user after being blurred. Such processing has an advantage that the privacy of users other than the target user can be protected. Especially when the blurring process is irreversible, there is an advantage that the privacy of users other than the target user can be protected.

In addition, only images within a predetermined range of the parked vehicle space may be provided to the user. In this case, for example, a cropped image may be provided corresponding to each parking space. In this case, since there is no image about other parking spaces, there is an advantage that the privacy of other than the target user can be protected.

Further, when the information including at least the periphery of the used vehicle is provided to the user, the user has an advantage that he / she can obtain information about some incident on the used vehicle. For example, when such an image is provided by a moving image, the user has an advantage that he / she can collect information about an incident such as mischief or theft of the vehicle used. When such moving images are provided, they may have various start and end times. For example, an example of the start time and the end time may be a video from the time of entering the parking lot to the time of leaving the parking lot, or from the time of parking in the parking lot to parking in the parking lot. It may be a video from the time of departure to the time of departure.

Further, the image that is the source of the information indicating a specific vehicle may be an image having various resolutions. That is, as long as the resolution of the image pickup device is higher than a predetermined value, the distance of the captured vehicle may be smaller than the predetermined length or longer than the predetermined length.

<Parking fee>
In addition, the image utilization unit may set a parking fee for the parking space. The parking fee may be set by acquiring parking fee information from the administrator terminal, or may be set based on a predetermined rule.

The former may be set based on the information input by the administrator via the administrator terminal. For example, the manager inputs each parking fee for each parking space in the parking lot, receives the entered parking fee at the manager terminal, sends the input parking fee to the management system, and receives the management. The parking fee charged in the system may be stored in association with the parking space.

In the latter case, the predetermined rule may be a rule in which the administrator sets a part or all from the beginning, or a rule in which the administrator sets a part or all with respect to a predetermined rule. It may be a predetermined rule that cannot be set by the administrator.

As for the parking fee, the parking fee may be set for each one parking space, or the parking fee may be set for a plurality of parking spaces. In the latter, for example, a plurality of parking spaces may be set as one parking area, and the same parking fee per parking space may be set for such one parking area, or one including a plurality of parking spaces. For parking lots, the same parking fee per parking space may be set. Further, the same parking fee may be set for all the parking spaces in one parking lot, or different parking fees may be set. In addition, even if the same parking fee can be set for the same type of parking space (for example, a type of vehicle that requires almost the same area as a parking space for passenger cars, buses, motorcycles, etc.). It may be possible to set different parking fees.

The predetermined rule may be a rule in which the parking fee of the same parking space changes. For the change of the parking fee, time information, parking space fullness information, and / or parking space location information may be used.

The time information may include the day and time of parking. For example, in a commercial facility whose main target is families, it may be possible to promote weekday visits and decentralize visits by making weekday parking fees cheaper than holiday parking fees. .. Similarly, by reducing the parking fee in the morning and the parking fee in the afternoon for the time zone on holidays, it is possible to promote the visit in the morning and decentralize the visit. Also, in this case, a commercial facility having a lunch facility may be able to increase the number of lunch visitors. In this way, by using the past usage history or by making the parking fee of the date and time B, which has a smaller number of visitors than the date and time A, lower than the date and time A, the number of visitors is dispersed. There is an advantage that it may be possible.

Further, the parking space fullness information may include only the current parking space fullness information, the past parking space fullness information only, or both the current and past parking space fullness information. It may include empty information. When the current fullness information is used, there is an advantage that the charge can be changed and set in real time based on the current fullness information. Further, when using the past full / empty information, there is an advantage that the charge can be set by using the tendency of the past users. Further, by using the current and past full / empty information, there is an advantage that the charge can be set based on the current user's situation while taking into account the past user's tendency.

Further, the fullness information of the parking space may include only the fullness information of the parking space in the same parking area as the parking space for which the parking fee is set and the set parking area, or the parking fee is charged. Only the fullness information of the parking space in the same parking lot as the target parking space to be set may be included, or both of them may be included. By using the fullness information of the same parking area, there is an advantage that the charge can be set in relation to the vicinity of the parking space for which the parking charge is set. For example, if the parking space is the same as the parking space for which the parking fee is set and the fullness information of the parking space in the set parking area is lower than a predetermined threshold value, the parking area can be reduced to such a parking area. It may be possible to promote parking in. In this case, for example, in a parking lot laid in a commercial facility, there is a possibility that parking in the far parking area or parking lot can be promoted by reducing the parking fee of the parking area or parking lot far from the commercial facility. As a result, there is an advantage that the congestion in the parking area or the parking lot can be reduced. That is, in the parking area A and the parking area B associated with one commercial facility, the image utilization unit sets the distance between the predetermined location of the commercial facility and the predetermined location in the parking area A of the commercial facility. When the distance between the predetermined location and the predetermined location in the parking area B is shorter, the parking fee of the parking area A may be set lower than the parking fee of the parking area B.

Further, the parking fee may be set by using the past usage history of the parking area. When the frequency of use of the parking space A is less than the frequency of use of the parking space B different from A in a predetermined period, the parking fee of the parking space A is set lower than the parking fee of the parking space B. There is an advantage that the use of space A can be promoted.

A monetary value may be set in place of the above-mentioned parking fee or in addition to the parking fee . The monetary value may also be set based on various factors similar to the parking fee described above. The monetary value setting may have the same function as the reduction of the parking fee. For example, the monetary value may be set without changing the parking fee while maintaining a predetermined standard fee. In this case, the parking space in which the monetary value is set may have a function of encouraging the user to use the parking space. Such monetary value may be of various types, as described above.

The management system may be composed of a plurality of management systems. For example, a local management system connected to one or more imaging devices and a central management system connectable to such one or more management systems may be provided. The local management system and the central management system may have the same function or may have different functions. For example, a local management system may have an image utilization unit that manages one or more parking lots managed by one facility manager. On the other hand, the central management system may have an image utilization unit that manages one or more parking lots managed by one or more local management systems.

For example, FIG. 11 shows an example of the relationship between the local management system and the central management system. In this figure, the local management systems 1102a and 1103a are directly or indirectly connected to the central management system 1101a via a network. An administrator terminal 1101b is connected to the central management system 1101a. The image pickup devices 1102c and 1102d are connected to the local management system 1102a, and the administrator terminal 1102b is also connected. The local management system 1103a may have the same configuration, or may be different as shown in this figure. The user terminals 1104a and 1104b may be connectable to the management system. As described above, the number of imaging devices, the number of administrator terminals, the number of management systems, the number of central management systems, the number of local management systems, and / or the number of user terminals may be one or more. ..

2.3. Functions in the administrator terminal The administrator terminal has a function of managing the management system.

The administrator terminal may be able to manage the user management unit and / or the image utilization unit. For example, the administrator terminal may be able to manage information about the user, such as checking and correcting the information in the user management unit. For example, when an incident such as theft or vehicle damage occurs in a parking lot, the administrator terminal has a function of acquiring information necessary for investigating the incident such as theft or vehicle damage through the image utilization unit. May have. In addition, the administrator terminal may have a function of correspondingly acquiring information about the user who should be responsible for such an incident.

The administrator terminal may also have a statistical information unit that confirms the usage status of the user and generates statistical information for improving the parking lot. For example, it may have a function of acquiring information such as usage time zone, usage vehicle, usage frequency, parking fee, etc. for the use of each parking space, and generating and displaying statistical information about these.

Further, when the system of this example is in a rented format, the administrator terminal may be able to generate and display information related to the management fee. The information related to the management fee may include the fee related to the use of this system. In addition, the information related to the management fee may include the total amount of the management fee in a predetermined period.

The administrator terminal may have two types of administrator terminals: a local administrator terminal that can manage the local management system and a central administrator terminal that can manage the central management system.

The local manager terminal may be able to manage only the local management system. The central administrator terminal may be able to manage only the central management system, may be able to manage only the local management system, or may be able to manage both the central management system and the local management system. Such management may include setting, viewing, and viewing information relating to management fees. For example, the central administrator terminal may be able to browse information related to the management fee by the local management system. In addition, the central administrator terminal may be able to browse the information managed in each local management system. For example, the central administrator terminal may be able to acquire all or a part of the information managed in each local management system. In addition, statistical information generated using such managed information may be generated and can be viewed in a central management system. The central administrator terminal may be able to manage only some local management systems and may not be able to manage other local management systems.

3. 3. Example
3.1. Example 1
The system example according to the first embodiment uses some or all of the above-mentioned functions to facilitate payment by the user.

In the system example according to the first embodiment, an imaging device (in the document of the present application) that includes a passage (sometimes referred to as a “parking passage” in the document of the present application) that the vehicle can reach in the parking space in which the vehicle is parked. , Sometimes referred to as "imaging device on parking passage") can be used in situations where it is installed. Here, FIG. 12 is an example of such a parking passage.

The image pickup device on the parking passage may be installed near the entrance / exit of the parking lot (within a predetermined distance from the entrance / exit) that enters the parking lot from a public road, or a place (entrance / exit) that is a certain distance away from the vicinity of the entrance / exit. It may be installed outside the range of a predetermined distance from. The latter may be, for example, when it is placed somewhere in the passage from the roadway to the parking lot.

Further, the image pickup device on the parking aisle may be one image pickup device for one parking lot, or may be provided with a plurality of image pickup devices. For example, for one parking lot, a plurality of image pickup devices on a parking passage are installed, and at least one image pickup device on a parking passage is installed on a passage leading to any one parking space in the parking lot. An imaging device on one parking passage may be provided that includes a passage leading to any one parking space in the imaging field. For example, FIG. 13 shows such a case.

Further, when the area of one parking lot is divided into a plurality of partial areas, the image pickup device on the parking passage may be an image pickup device installed on the passage leading to the plurality of partial areas. For example, FIG. 14 shows such a case.

In short, the on-parking passage imaging device is an imaging device installed on a passage where vehicles can reach all parking spaces in one parking lot or all parking spaces in a partial area of one parking lot. May include. This has the advantage that all vehicles that use all the parking spaces in one parking lot or all the parking spaces in a partial area of one parking lot can be imaged by the image pickup device on the parking passage. ..

FIG. 15 is an example of a system using an image captured by an image pickup device on a parking passage. An example system may include a management system 1501. Further, the management system 1501 may be connected to the image pickup devices 1502a and 1502b. Further, the management system 1501 may be connected to a manageable administrator terminal 1501a. Further, the user terminals 1503a and 1503b may be connectable to the management system.

Next, a method in which the management system of one example specifies a payment means for the parking act of the user will be described with reference to FIG.

First, the management system records each relationship between the information that identifies the vehicle and the information that identifies the payment instrument (step 1). Such identification may be based on the information input from the user terminal or may be based on the input from the administrator terminal. Information that identifies one or more payment instruments may be associated and stored with information that identifies one or more vehicles. In the information, the information for identifying the vehicle may include the information obtained from the image obtained by capturing the license plate, the image itself obtained by capturing the license plate, or the user regardless of the image. Alternatively, the information of the character string related to the license plate input by the administrator may be included.

Next, the management system acquires information for identifying the vehicle from the image captured by the image pickup device (step 2). In the case of a parking lot where the parking price fluctuates depending on the start and end of the parking lot, the management system uses a plurality of images obtained from the acquired information to identify the vehicle, and uses time information indicating the start of parking and the end of parking. You may acquire the temporal information indicating that.

Next, the management system identifies the user information related to the information that identifies the vehicle and updates the parking history information (step 3). The update of the parking history information may be information added to the parking history information for one parking, for example, information on parking time, information on parking lot, and imaged at the time of parking for one parking. It may be an image, parking fee related information, or the like.

Next, the management system performs billing processing using the target payment means based on the user history information (step 4). When the management system performs billing processing for the parking use act, it is not necessary to add to the history information in step 3 described above or to use the history information.

Next, the processing in the parking application will be described including the usage mode of the user.

The user downloads and installs the parking application on the user terminal (step 1).

The user inputs user information in the parking application on the user terminal, the parking application records such user information, and transmits a part or all of the user information to the management system (step 2). .. Such user information may be the one described above, including information identifying the vehicle used by the user and payment instrument information for parking.

When a user uses a parking lot equipped with an image pickup device capable of capturing an image used by an example system using a vehicle, when entering the parking lot, a ticket indicating the time of entry is issued. There is no need to acquire it and parking is done (step 3).

When leaving the parking lot, the user exits the parking lot without paying the parking fee (step 4). In this way, when an image pickup device capable of capturing an image capable of acquiring information identifying the vehicle is installed in association with the parking lot, one system can use the image captured by the image pickup device. In this case, the user has an advantage that there is no burden related to the parking ticket when entering the parking lot and no burden related to payment when leaving the parking lot. The parking lot also has an advantage that the congestion until such parking can be reduced.

The management system acquires information for identifying the vehicle to be used from the image of the vehicle to be used, further identifies the means of payment for the parking act of the vehicle to be used, and pays the parking fee for the parking act by the means of payment. Associate (step 5). If the time information in steps 3 and 4 above affects the parking fee, the management system may calculate the parking fee using the time information.

The user pays for the parking act using the payment means information input via the parking application after the parking act (step 6). In addition, the payment request processing may be performed by the example system itself, or the payment request processing may be performed in cooperation with another system. The user can pay at a different time than when parking. Further, when the payment means can be set, there is an advantage that payment can be made from an appropriate payment means according to the mode of use of the vehicle to be used.

In the above flow, the parking application is used, but instead, the same applies to the page on the WEB by identifying the user by using the user ID and authentication on the Internet. Functions may be processed.

An example management system is
A database that stores information that identifies a vehicle using a license plate and information that identifies a payment method that is paid for parking the vehicle in association with each other.
A means of acquiring information that identifies the first vehicle,
A means for searching the database for information that identifies the first vehicle, and
The specific means for specifying the information for specifying the first payment means corresponding to the information for identifying the first vehicle, and the specific means for specifying the information.
It may be a system including.

The database that stores the information that identifies the vehicle and the payment means information that is paid for the parking action of the vehicle in association with each other is the information that identifies the vehicle and the payment means information that is paid for the parking action of the vehicle. And, as long as they are stored in association with each other, and may be in any format. Even when a database is used, any database technical method may be used. Examples include, but are not limited to, relational databases, hierarchical databases, network databases, and NoSQL. Further, the number of databases may be one or more, and may be an information processing device on the cloud.

Further, the mode of association is not limited, and for example, one or more payment means information may be associated with the information identifying one vehicle. When a plurality of payment method information is associated with each other, a plurality of payment method information may be specified for a specific vehicle, and one of the plurality of payment method information is a predetermined rule or. By identifying by information indicating that the user has selected the parking activity, it is possible to identify the means for paying the fee for the parking act.

In addition, by storing a plurality of payment means information in association with the information identifying one vehicle, for example, when one vehicle is used by a plurality of entities, which entity bears the charge. There is an advantage that you can choose. For example, it is possible to distinguish between the use of a vehicle as a business or public activity such as a corporation or sole proprietor, and the personal use, and for a vehicle owned by a family. It is possible for the user to bear the burden according to the user in the family, and in the case of so-called car sharing where one vehicle is shared by people related to multiple families, it is linked with the car sharing system. And may be associated with payment instrument information.

Another example of a management system is
A database that stores information that identifies a vehicle using a license plate in association with information that identifies a payer who pays for the parking of the vehicle or a user who has registered for use.
A means of acquiring information that identifies the first vehicle,
A means for searching the database for information that identifies the first vehicle, and
A specific means for specifying information that identifies the first payer or the first user corresponding to the information that identifies the first vehicle, and
It may be a system having.

The database that stores the information that identifies the vehicle using the license plate and the information that identifies the payer who pays for the parking act of the vehicle in association with each other pays instead of the information that identifies the payment method. It may be the information that identifies the payer who performs the above, and the others may be the same database as described above. By identifying the payer, there is an advantage that payment for parking can be pursued even when the payment method is not specified. After specifying the payer, it may be registered as a debt to the payer, or the payment method to be paid by the payer described in a separate database or the like may be specified.

In addition, the database that stores the information that identifies the vehicle using the license plate and the information that identifies the user who has registered for the parking of the vehicle in association with each other is the information that identifies the payment method. Instead, it may be information that identifies the user who has registered for use, and other information may be the same database as described above. By identifying the user, there is an advantage that payment for parking can be pursued even when the payment method is not specified. After specifying the user, it may be registered as a debt to the user, or the payment method to be paid by the payer described in a separate database or the like may be specified.

In the documents of the present application, information capable of pursuing payment for parking activities such as "payment means", "user", and "payer" may be referred to as "payment pursuitable information".

Another example system is
A database that stores information that identifies a vehicle using a license plate and information that identifies information that can be paid for parking of the vehicle in association with each other.
A means of acquiring information that identifies the first vehicle,
A means for searching the database for information that identifies the first vehicle, and
A specific means for specifying information for specifying payment pursuitable information corresponding to the information for identifying the first vehicle, and
It may be a system having.

3.2. Example 2
The second embodiment related to this system is an example in which the image utilization unit can generate full / empty information. Next, such a process will be described with reference to FIG.

First, the system of one example generates full sky information from images captured by one or more imaging devices (step 1). Such information may be generated at various timings such as every predetermined period and the timing of entering and exiting the vehicle.

An example system provides such full / empty information so that the user can view it (step 2). In particular, it may be provided to the user related to the vehicle to be parked in the parking area associated with the fullness information. Various information processing devices may be used, for example, a display device installed in association with the parking lot, a display device of a user terminal related to the user, and the like. The timing of such provision may be various, such as a timing corresponding to a request from the user terminal, a timing in which the user vehicle has a predetermined relationship with the corresponding parking area, and the like. With such a configuration, the user has an advantage that he / she can understand the full / empty information of the parking area.

Next, the user parks in the parking space located in the parking area related to the fullness information (step 3).

Next, the system of the example changes the corresponding parking space to parked information based on the image captured by the imaging device including the parking space in the imaging field of view (step 4). As an example of the full / empty information, for example, the full / empty information as shown in FIG. 22 may be generated for the parking lot of FIG.

3.3. Example 3
The system according to the third embodiment is an example in which the image utilization unit has a function of guiding the vehicle, and will be described with reference to FIG.

First, the system of one example generates full sky information from images captured by one or more imaging devices (step 1).

The system of one example presents an empty car area for a part or all of the parking area related to the fullness information, and provides the user with information useful for moving to the empty car area (step 2). As described above, the mode provided to the user may be various. Further, the system of the example may provide all the vacant vehicle areas among the parking areas related to the full vacancy information, but may not provide all the vacant vehicle areas and may provide some vacant vehicle areas. In the latter case, an example system may determine one or more empty vehicle areas to guide prior to such offer. The determination of such an empty vehicle area may be determined from various viewpoints. For example, it may be determined for the purpose of preventing confusion in the parking lot, for the purpose of equalizing the vacancy rate in the parking lot, for the purpose of realizing the user's wishes, and these. It may be decided for the purpose of a part or all of. The specific means for achieving each purpose may be as described above.

Next, the vehicle to be used approaches the empty vehicle area (step 3). An example system may provide the user with information useful to move to the empty vehicle area correspondingly as the vehicle being guided approaches the empty vehicle area.

Next, the vehicle to be used parks in the empty vehicle area (step 4).

An example system changes a parked empty car area to parked (step 5). Here, an example system may acquire such information from images captured by one or more imaging devices.

3.4. Example 4
The system according to the fourth embodiment is an example in which the image utilization unit changes the parking fee for the parking area, and will be described with reference to FIG. Examples 1 to 3 are reference examples of the invention described in the claims of the present application.

First, one example system sets a charge for one parking area (step 1). Such charges may be set based on various information. Further, the parking area where the charge is set may be different or the same for each parking space.

Next, the system of the example associates the parking fee with the parking area and presents it to the user (step 2). The mode of use may be various methods, and may be displayed on a display associated with the parking area or displayed on a user terminal.

Next, the user parks in one parking space in the parking area (step 3).

In the case where the system of one example adopts the rule of changing the parking fee due to the change of the full vacancy information due to the current parking action, the parking fee is newly set by using the change of the full vacancy information. (Step 4). If the system of one example does not set the parking fee due to the change in the current full vacancy information, it is not necessary to set the parking fee by using the change in the current full vacancy information.

The system of one embodiment of the first aspect is
A database that stores information that identifies a vehicle using a license plate and information that can be paid for parking of the vehicle in association with each other.
An acquisition unit that acquires information that identifies the first vehicle,
A search unit that searches the database for information that identifies the first vehicle, and
A specific unit that specifies the first payment pursuitable information corresponding to the information that identifies the first vehicle, and
System with.

The system of one embodiment of the second aspect is
The information for identifying the first vehicle includes information generated by applying a machine learning function to an image including a license plate.
The system of the first embodiment.

The system of one embodiment of the third aspect is
The image includes a plurality of parking spaces in a parking lot.
The system of the first or second embodiment.

The system of one embodiment of the fourth aspect is
The image was taken by an imaging device capable of capturing a plurality of parking spaces in the parking lot.
The system of any one of the first to third embodiments.

The system of one embodiment of the fifth aspect is
Each parking space in the parking lot is provided with an image utilization unit that manages the presence or absence of parking using the image.
The system of any one of the first to fourth embodiments.

The system of one embodiment of the sixth aspect is
Using the image, the trajectory of a vehicle moving to each parking space in the parking lot is tracked.
The system of any one of the first to fifth embodiments.

The system of one embodiment of the seventh aspect is
The image utilization unit identifies at least one parking space to be guided to the vehicle to be parked by using the presence or absence of parking.
The system of any one of the first to third embodiments.

The system of one embodiment of the eighth aspect is
The image utilization unit stores each of the parking spaces in association with information indicating a parking fee.
The system of any one of the first to seventh embodiments.

The system of one embodiment of the ninth aspect is
The image utilization unit has a first parking fee associated with the first parking space and a second parking space different from the first parking space for vehicles of the same type as the vehicle parked in the first parking space. Set a different fee for the second parking fee associated with,
The system of any one of the first to eighth embodiments.

The system of one embodiment of the tenth aspect is
The image utilization unit changes the parking fee associated with the first parking space according to whether or not the parking space other than the first parking space is parked.
The system of any one of the first to ninth embodiments.

The system of one embodiment of the eleventh aspect is
The information for identifying the first vehicle and the parking time of the first vehicle are stored in association with each other.
The parking time includes a parking start time and a parking end time, or a parking duration.
The system of any one of the first to tenth embodiments.

The system of one embodiment of the twelfth aspect is
A means of acquiring information that identifies the second vehicle,
Using the information for identifying the second vehicle and the information for identifying the first vehicle, it is determined whether the first vehicle and the second vehicle are the same vehicle.
When the first vehicle and the second vehicle are the same vehicle,
The parking fee for the first vehicle is calculated using the parking fee for the second vehicle.
The system of any one of the first to eleventh embodiments.

The system of one embodiment of the thirteenth aspect is
The acquisition unit that acquires the information that identifies the first vehicle is
A machine learning unit that has machine-learned the relationship between an image including a license plate and numbers and character strings in the license plate is provided.
The system of any one of the first to twelfth embodiments.

The system of one embodiment of the fourteenth aspect is
The first payment pursuitable information includes at least one of information relating to a credit card, a debit card, electronic money, account information, a user, and a payer.
The system of any one of the first to thirteenth embodiments.

The mobile terminal device of the embodiment of the fifteenth aspect is
The acquisition department that acquires information that identifies the vehicle that is the target of parking,
An acquisition unit that can acquire information on payment means for the parking of the vehicle,
A display unit that displays information indicating that the fee for parking the vehicle is paid by the payment means information.
Mobile terminal device having.

The mobile terminal device of the embodiment of the 16th aspect is
An image pickup unit that can image the license plate of the vehicle subject to parking,
An acquisition unit that can acquire information on payment means for the parking of the vehicle,
A display unit that displays information indicating that the fee for parking the vehicle is paid by the payment means information.
Mobile terminal device having.

The program of one embodiment of the 17th aspect is
Computer,
A storage means for associating and storing information for identifying a vehicle using a license plate and information for pursuing payment for parking of the vehicle.
Acquisition means for acquiring information that identifies the first vehicle,
A search means for searching the database for information identifying the first vehicle,
A specific means for identifying the first payment pursuitable information corresponding to the information for identifying the first vehicle,
A program that operates as.

The method of one embodiment of the eighteenth aspect is
The computer
A step of associating and storing information for identifying a vehicle using a license plate and information for pursuing payment for parking of the vehicle.
Steps to obtain information that identifies the first vehicle,
A step of searching the database for information identifying the first vehicle,
The step of identifying the first payment pursuitable information corresponding to the information identifying the first vehicle,
How to do.

The program of one embodiment of the nineteenth aspect is
A program for operating a computer as a system according to any one of the first to fourteenth embodiments.

4. Hardware Configuration of System The system according to the present invention may be composed of one or a plurality of information processing devices. As shown in FIG. 23, the information processing device 10 according to the present invention may include a bus 15, an arithmetic device 11, a storage device 12, and a communication device 16. Further, the information processing device 10 in one embodiment may include an input device 13 and a display device 14. It is also directly or indirectly connected to the network 17.

The bus 15 may have a function of transmitting information between the arithmetic unit 11, the storage device 12, the input device 13, the display device 14, and the communication device 16.

An example of the arithmetic unit 11 is a processor. This may be a CPU or an MPU. Further, the arithmetic unit in one embodiment may include a graphics processing unit, a digital signal processor, and the like. In short, the arithmetic unit 12 may be any device capable of executing program instructions.

The storage device 12 is a device that records information. This may be either an external memory or an internal memory, and may be either a main storage device or an auxiliary storage device. Further, a magnetic disk (hard disk), an optical disk, a magnetic tape, a semiconductor memory, or the like may be used. Further, it may have a storage device via a network or a storage device on the cloud via a network.

In this block diagram, the registers, L1 cache, L2 cache, etc. that store information at a position physically close to the arithmetic unit may be included in the arithmetic unit 11, but in the design of the computer architecture, As a device for recording information, the storage device 12 may include these. In short, the arithmetic unit 11, the storage device 12, and the bus 11 may be configured to cooperate with each other to execute information processing.

The storage device 12 may include a part or all of a program capable of executing the process according to the present invention. In addition, data necessary for executing the process according to the present invention can be appropriately recorded. The storage device 12 in one embodiment may also include a database.

Also, one of the above, the arithmetic unit 11, has been described the case where it is performed based on a provided program in a storage device 12, the above-mentioned bus 15, the arithmetic unit 11 and the storage device 12 are combined format As a result, part or all of the information processing according to the present invention may be realized by a programmable logic device capable of changing the hardware circuit itself or a dedicated circuit in which the information processing to be executed is determined.

The input device 13 inputs information, but may have other functions. Examples of the input device 14 include an input device such as a keyboard, a mouse, a touch panel, or a pen-type instruction device.

The display device 14 has a function of displaying information. For example, a liquid crystal display, a plasma display, an organic EL display, and the like can be mentioned, but in short, any device capable of displaying information may be used. Further, the input device 13 may be partially provided like a touch panel.

The network 17 transmits information together with the communication device 16. That is, it has a function of enabling information of 10 information processing devices to be transmitted to other information terminals (not shown) via the network 17. The communication device 16 may use any connection type, such as IEEE1394, Ethernet (registered trademark), PCI, SCSI, USB, 2G, 3G, 4G, 5G, and the like. The connection to the network 17 may be either wired or wireless.

The information processing device according to the present invention may be a general-purpose type or a dedicated type. Further, the information processing device may be a workstation, a desktop personal computer, a laptop personal computer, a laptop computer, a PDA, a mobile phone, a smartphone or the like.

Although described as one information processing device 10 in this figure, the system according to the present invention may be composed of a plurality of information processing devices. The plurality of information processing devices may be internally connected or may be externally connected.

Further, the system according to the present invention may be of various types of devices. For example, the system according to the present invention may be a stand-alone system, a server-client system, a peer-to-peer system, or a cloud system. The system according to the present invention may be a stand-alone information processing device, may be composed of a part or all of the information processing device of the server-client type, or may be a part or all of the information processing of the peer-to-peer type. It may be composed of devices, or may be composed of some or all information processing devices in cloud format.

Further, when the system according to the present invention is composed of a plurality of information processing devices, the owner and the manager of each information processing device may be different.

Further, the information processing device 10 may be a physical existence or a virtual one. For example, the information processing device 10 may be virtually realized by using cloud computing.

In the above description, the configurations implemented by the system of this example have been described, but these may be configurations implemented by one or more information processing devices in the system. Further, the information processing device described above as the portable information processing device may be an information processing device that is appropriately installed and fixed.

It goes without saying that the invention examples described in the examples of the documents of the present application are not limited to those described in the documents of the present application, and can be applied to various examples within the scope of the technical idea. For example, in the embodiment of the document of the present application, the information presented on the screen of the information processing device can be displayed on the screen of the other information processing device, so that the information can be transmitted to the other information processing device. The system may be configured. In addition, the indication of ◯ in various drawings may contain appropriate values according to each context, and may be the same or different.

In addition, the processes and procedures described in the documents of the present application may be feasible not only by those explicitly described in the embodiments, but also by software, hardware or a combination thereof. Further, the processes and procedures described in the documents of the present application may be able to be implemented by various computers by implementing the processes and procedures as a computer program. Further, these computer programs may be stored in a storage medium. Also, these programs may be stored on a non-transient or temporary storage medium.

10 Information processing device 11 Computing device 12 Storage device 13 Input device 14 Display device 15 Bus 16 Communication device 71 Display screen 72 Number plate 73 Line indicating outer frame 81 Parkable area 82 Current position 1101a Central management system 1101b Administrator terminal 1102a 1103a Local management system 1102b 1103b 1103c Administrator terminal 1102c 1102d 1103c 1103d 1103e Imaging device 1104a 1104b User terminal 02 Parking space 03 Imaging device 04 Imaging field 03A 03B Entrance / exit 04A 04B Imaging device 05A 01 1501a Administrator terminal 1502a 1502b Imaging device 1503a 1503b User terminal

Claims (9)

An image acquisition unit that acquires an image from the one or more image pickup devices, in which one or more image pickup devices each have a plurality of parking spaces related to one commercial facility as an image pickup field of view.
A charge setting unit that sets a charge for the first parking space in the first area based on the availability information for the first area including a plurality of parking spaces in the image.
It is a system equipped with
The charge setting unit of the second parking space in the second area is based on the fullness information about the second area including the plurality of parking spaces, which is different from the first area in the one or more images. The charge is set to a charge different from the charge of the first parking space.
system. When the charge for the first parking space related to the one commercial facility is the first charge at the first predetermined time, the charge setting unit performs the first predetermined time at a second predetermined time different from the first predetermined time. The charge for one parking space is set to a second charge different from the first charge based on the fullness information for the first area .
The system according to claim 1. A monetary value setting unit that sets a monetary value for parking in the first parking space in the plurality of parking spaces based on the fullness information for the first area .
With more
The monetary value includes the monetary value available in the one commercial facility pertaining to the parking space.
The system according to any one of claims 1 or 2. The charge setting unit
Full sky information about the second area including the first area and the plurality of parking spaces comprises a plurality of parking spaces, both when it is congestion degree, full air information according to the first region If the information is higher than the full / empty information related to the second area, the charge for the first parking space is set to be higher than the charge for the second parking space.
Full sky information about the second area including the first area and the plurality of parking spaces comprises a plurality of parking spaces, both when it is unladen rate, is fully empty information according to the first region If the information is higher than the full / empty information related to the second area, the charge for the first parking space is set to be lower than the charge for the second parking space.
The system according to any one of claims 1 to 3. The vacancy information is based on a number generated by multiplying the number of vehicles specified in the parking area including the plurality of parking spaces by a predetermined correction coefficient.
The system according to any one of claims 1 to 4. The fullness information is based on an association between a parking space and a vehicle in the parking area including the plurality of parking spaces in the order of proximity to the imaging device including the parking area in the imaging range.
The system according to any one of claims 1 to 5. The computers that make up the system
An image acquisition means for acquiring an image from the one or more image pickup devices, wherein each one or more image pickup devices has a plurality of parking spaces related to one commercial facility as an image pickup field.
A charge setting means for setting a charge for a first parking space in the first area based on fullness information about a first area including a plurality of parking spaces in the image.
It is a program to function as
The charge setting means of the second parking space in the second area is based on the fullness information about the second area including the plurality of parking spaces, which is different from the first area in the one or more images. The charge is set to a charge different from the charge of the first parking space.
program. Image acquiring section according to the system, and the imaging field of view several parking spaces according to each one commercial establishment one or more imaging devices, from the one or more imaging devices, an image acquisition step of acquiring an image,
The charge setting unit related to the system sets the charge of the first parking space in the first area based on the fullness information about the first area including the plurality of parking spaces in the image. Pricing step,
Is a way to do
The rate setting step is based on the availability information of the second area including the plurality of parking spaces, which is different from the first area in the one or more images, and is based on the availability information of the second parking space in the second area. The charge is set to a charge different from the charge of the first parking space.
Method. A program for operating a computer as the system according to any one of claims 1 to 6 .