KR20200079181A - Electronic apparatus and operating method thereof, and server and operation method thereof in system for providing service according to position based on object detected from video - Google Patents

Abstract

According to various embodiments of the present invention, in a system in which a first electronic device, a server, and a second electronic device wirelessly communicate, the first electronic device detects object information related to at least one object from a captured image and transmits the object information to the server, the server identifies a location of the object based on the object information and transmits service information related to the location of the object to the second electronic device, and the second electronic device provides the service information to a user.

Description

An electronic device and its operation method and a server and its operation method in a system for providing service according to a location based on an object detected from an image. POSITION BASED ON OBJECT DETECTED FROM VIDEO}

Various embodiments relate to an electronic device and an operation method thereof and a server and an operation method thereof in a system for providing a service according to a location based on an object detected from an image.

In general, an electronic device provides various services by performing various functions in combination. For example, the electronic device provides a location based service (LBS). For example, the electronic device may measure the current location and guide the travel route from the current location to the destination. As another example, the electronic device may download and provide service information related to a location specified by the user.

However, the electronic device as described above only provides a location-based service based on a user input. For example, as the electronic device directly measures the current location or directly inputs a location desired by a user of the electronic device, the electronic device may search for service information through a server. Due to this, it is difficult for a user of the electronic device to use the location-based service.

The electronic device according to various embodiments of the present disclosure may include a camera module configured to capture an image, a communication module configured to wirelessly communicate with a server, and a processor connected to the camera module and the communication module. According to various embodiments, the processor may be configured to detect object information related to at least one object existing in the captured image, and to transmit the object information to the server.

An operation method of an electronic device according to various embodiments includes an operation of photographing an image, an operation of detecting object information related to at least one object existing in the image, and wirelessly transmitting the object information to a server. Action.

A server according to various embodiments may include a communication module configured to wirelessly communicate with at least one electronic device, and a processor operatively connected to the communication module. According to various embodiments, the processor, through the communication module, receives object information related to at least one object, and based on the object information, identifies the location of the object, and through the communication module , May be configured to transmit service information related to the location.

An operation method of a server according to various embodiments of the present disclosure includes receiving object information related to at least one object from a first electronic device, identifying an object location based on the object information, and the location and And transmitting related service information to the second electronic device.

According to various embodiments of the present disclosure, an operation method of an electronic device wirelessly receives service information related to the location of the object determined based on at least one object from a server, and based on a pre-installed application. It may include an operation to provide.

According to various embodiments, a server may provide an appropriate location-based service for a user without user input to the electronic device. That is, the server may grasp the user's location based on an object detected during image capture in the electronic device. For this reason, a separate user input is unnecessary in the electronic device, and the electronic device does not need to grasp the user location. And since the server determines the service information based on the user's location, the user can use the appropriate service information according to the time and place in response to the current location. Therefore, the user can efficiently use the location-based service.

1 is a diagram illustrating a system in accordance with various embodiments.
2A is a diagram illustrating signal flow in a system according to various embodiments.
2B is a diagram for describing a system according to some embodiments.
3 is a diagram illustrating an electronic device according to various embodiments of the present disclosure.
4 is a diagram illustrating a method of operating an electronic device according to various embodiments of the present disclosure.
5 is a diagram illustrating a server according to various embodiments.
6 is a diagram illustrating a method of operating a server according to various embodiments.
7 is a diagram illustrating an electronic device according to various embodiments of the present disclosure.
8 is a diagram illustrating a method of operating an electronic device according to various embodiments of the present disclosure.
9A, 9B, and 9C are diagrams illustrating an example of a screen displayed when providing service information in FIG. 8.
10 is a diagram illustrating a vehicle equipped with an electronic device according to various embodiments of the present disclosure.
11 is a diagram illustrating the electronic device of FIG. 10.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings.

It should be understood that various embodiments of the document and terms used therein are not intended to limit the technology described in this document to specific embodiments, and include various modifications, equivalents, and/or replacements of the embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar elements. Singular expressions may include plural expressions, unless the context clearly indicates otherwise. In this document, expressions such as “A or B”, “at least one of A and/or B”, “A, B or C” or “at least one of A, B and/or C”, etc. are all of the items listed together. Possible combinations may be included. Expressions such as "first", "second", "first" or "second" can modify the corresponding components, regardless of order or importance, and are used only to distinguish one component from another component The components are not limited. When one (eg, first) component is said to be “connected (functionally or communicatively)” or “connected” to another (eg, second) component, some of the components are It may be directly connected to the component, or may be connected through another component (eg, the third component).

As used herein, the term "module" includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic blocks, components, or circuits. The module may be an integrally configured component or a minimum unit performing one or more functions or a part thereof. For example, the module may be configured with an application-specific integrated circuit (ASIC).

1 is a diagram illustrating a system in accordance with various embodiments.

Referring to FIG. 1, the system 100 according to various embodiments may include at least one electronic device 110 and a server 120. The electronic device 110 may communicate with at least one of the other electronic device 110 or the server 120 through at least one of the first network 130 or the second network 140. For example, the first network 130 may support short-range wireless communication, and the second network 140 may support long-range wireless communication. According to an embodiment, the electronic device 110 may communicate with another electronic device 110 through the server 120. The electronic device 110 may be various types of devices. For example, the electronic device 110 may be a user terminal, such as a portable communication device (eg, a smart phone), an image acquisition device such as a dashboard camera, a computer device, a portable multimedia device, a portable medical device, or a camera. , A wearable device, a point of sales terminal, or at least one of a home appliance.

2A is a diagram illustrating signal flow in a system according to various embodiments.

Referring to FIG. 2A, a system according to various embodiments (eg, the system 100 of FIG. 1) includes a first electronic device 210 (eg, the electronic device 110 of FIG. 1 ), a server 220 ( For example, the server 120 of FIG. 1 and the second electronic device 230 (eg, the electronic device 110 of FIG. 1) may be included. According to an embodiment, the first electronic device 210 may be configured in the same manner as the second electronic device 230. According to another embodiment, the first electronic device 210 may be configured differently from the second electronic device 230. For example, the first electronic device 210 is a black box mounted on a vehicle or a V2X-enabled communication module installed in the vehicle, and the second electronic device 230 is a user terminal, a computer device or a force carried by the user. It can be a terminal.

According to various embodiments, the second electronic device 230 may install an application determined in operation 211. For example, the predetermined application may be an application related to location-based services. The second electronic device 230 may download and install a predetermined application from the application store.

According to various embodiments, the server 220 may store location information and service information in operation 213. The location information may indicate information related to at least one location where at least one object is disposed. The object may be formed in various forms, and the shape of the object may vary depending on the position where the object is placed. Objects can be placed on the walls of buildings, signs, signs, large stickers or banners, or in the form of a hologram at the entrance to a parking lot. The shape of the object may include at least one of a barcode, QR code, image, text, or color. In the location information, an object and a location are mutually mapped and stored, and the location information may include object information for each object and location identification information for each location. According to an embodiment, when an object is placed at a predetermined location, the server 220 may store location information of the object in correspondence with the predetermined location. The service information may indicate information related to a service available at at least one location where at least one object is disposed. For example, the service information may include guide information, advertising information, coupon information, event information such as coupons, payment information, or a user's visit frequency and time of visit (eg, season, date, time point) for at least one area corresponding to each location. ), residence time, contact information, and vehicle information (eg, vehicle type, year, vehicle number), and the like. Through this, the server 220 may use the service information, perform a business for product sale or service provision for a corresponding company under a contract with a specific company, and obtain a profit for the business. According to various embodiments, the server 220 generates and stores service information based on data input by an operator of the server 220, or communicates with another server (eg, the server 120 of FIG. 1). Through, service information can be defined and stored.

According to various embodiments, the first electronic device 210 may take an image in operation 215. According to an embodiment, while the first electronic device 210 is turned on, the first electronic device 210 may continuously take an image. For example, if the first electronic device 210 is mounted on the vehicle, the first electronic device 210 may continuously or periodically collect image information related to the vehicle while photographing the image. The status information may include, for example, at least one of the current location of the vehicle, the amount of fueling, the battery status, or the engine oil status. Here, the first electronic device 210 may take an image regardless of an operation state of the vehicle, for example, whether it is parked or not. According to another embodiment, while the first electronic device 210 is on, the first electronic device 210 may take an image temporarily or for a time specified by the user based on the user's request. have.

According to various embodiments, the first electronic device 210 may detect object information from an image captured in operation 217. If the captured image includes at least one object, the first electronic device 210 may detect object information from the captured image. For example, the first electronic device 210 may analyze the captured image in units of frames. For example, the first electronic device 210 may analyze all frames of the image being photographed. As another example, the first electronic device 210 may analyze frames spaced apart at regular intervals from among all frames of the captured image. Through this, the first electronic device 210 may recognize an object in at least one of consecutive frames and detect object information. Thereafter, the first electronic device 210 may transmit the object information to the server 220 in operation 219. In some embodiments, when the same object information is repeatedly detected a predetermined number of times, the first electronic device 210 may transmit the object information to the server 220.

According to various embodiments, the server 220 may determine location information based on object information received from the first electronic device 210 in operation 221. The server 220 may identify a location where an object corresponding to the object information is placed, based on the object information. In some embodiments, the server 220 may manage statistical data such as the number of visits, preferences, and usage status of an area corresponding to each location, based on the number of times the location information is determined. In addition, the server 220 may determine service information based on the location information in operation 223. That is, the server 220 may determine service information for transmission to the second electronic device 230.

In one embodiment, the server 220 may identify object information according to a location where the object is placed. For example, when an object is disposed at a gas station entrance or the like, the object information may be gas information or the like, and when the object is disposed at a store entrance or the like, the object information may be a sale item or coupon information. If the object is placed in a workshop, the object information may be a maintenance target item of a vehicle equipped with the first electronic device 210, a maintenance manual, software version information such as a vehicle operating system, update information, and the like. When arranged in a parking lot, the object information may be information such as a parking fee and parking hours. Also, the location of the object may be mapped to information uniquely assigned to each object. The server 220 may determine service information related to a location corresponding to the location information.

In another embodiment, if the object is placed in an access point (AP) that provides high-speed wireless communication such as a mobile station or 4G/5G-based cellular or wireless fidelity (WIFI), the object information is the first electronic device 210 Defects in the software of the vehicle to check the safety of the vehicle equipped with the first electronic device 210 or information for installing or updating the software of the vehicle equipped with) (software version information, error information, whether an update is necessary) It can be information to check your back. In this embodiment, the server 220 may determine the location of the vehicle to perform software installation, update, and safety inspection procedures through the location information of the object, and the first electronic device 210 through the identified location Alternatively, the software installation, update, and safety inspection of the vehicle may be performed through OTA (Over the air). If the server 220 installs the software of the first electronic device 210 or the vehicle, updates the software, and checks the safety, the second electronic device 230 performs software installation, software update, and safety. Payment information for inspection may be determined as service information.

In another embodiment, when the object is disposed in a distribution warehouse or a delivery location of the goods, the object information may be transportation information, delivery information (delivery completion information, delivery location information, etc.), and the second electronic device 230 The object information varies depending on whether the computer or the recipient of the logistics management system is a portable electronic device. Specifically, if the second electronic device 230 is a computer of the logistics management system, the first electronic device 210 may be mounted on a transportation means such as a truck delivering goods, and in this case, the object information is It may be whether or not the transport means to be transmitted to the server 220 arrives at the destination of the goods, and the service information 225 for the server 220 to transmit to the second electronic device 230 is the recipient who received the goods. Payment information for shipping charges charged to the customer or logistics information to notify the payment information about the freight to be paid to the first electronic device 210 mounted on the transportation means transporting the goods to the location where the object is located, or the operation server of the transportation means It can be information. On the other hand, when the second electronic device 230 is an electronic device carried by the recipient, the service information for the server 220 to transmit to the second electronic device 230 includes information on the completion of arrival of goods, arrival time, and shipping charges. It can be payment information.

According to various embodiments, the server 220 may transmit service information to the second electronic device 230 in operation 225. At this time, in operation 225, the service information transmitted by the server 220 to the second electronic device 230 may be service information related to at least one object obtained by the first electronic device 210. The server 220 may detect the second electronic device 230 set corresponding to the first electronic device 210 and transmit service information to the second electronic device 230. In some embodiments, the server 220 transmits service information to another server (eg, the server 120 in FIG. 1 ), thereby causing the other server (eg, the server 120 in FIG. 1) to transmit the second electronic device ( 230).

The second electronic device 230 set corresponding to the above-described first electronic device 210 means an electronic device capable of receiving service information related to object information acquired by the first electronic device 210. In this case, the second electronic device 230 may use not only a service information user but also a service information provider.

Here, the second electronic device 230 may be an electronic device owned by the same user as the first electronic device 210 or an electronic device owned by another user. If the second electronic device 230 is an electronic device owned by the same user as the first electronic device 210, the server 220 may include the second electronic device 230 and the first electronic device 210. It is possible to determine that the electronic device is owned by the same user through an authentication procedure performed when accessing the communication network. If the first electronic device 210 and the second electronic device 230 are electronic devices owned by the same user, the server 220 corresponds to object information transmitted by the first electronic device 210. The service information is transmitted to the second electronic device 230.

Specifically, if the second electronic device 230 is an electronic device owned by the same user as the first electronic device 210, the server 220 includes object information transmitted by the first electronic device 210. As related service information, guide information, advertisement information, event information such as coupons, payment information, and the like may be transmitted to the second electronic device 230. In addition, a user who receives service information through the second electronic device 230 may perform an additional transaction through various information corresponding to the service information. In this case, the user of the second electronic device 230 It will mainly be a service user. For example, if the service information received through the second electronic device 230 is guide information of the facility where the object information is located, driving indoor navigating so that the user can grasp the location of a desired point according to the guide information You could do it. In addition, if the service information received through the second electronic device 230 is payment information such as a parking lot or a gas station where the object information is located, the user may perform a transaction for payment using the received payment information.

On the other hand, if the user of the first electronic device 210 and the user of the second electronic device 230 are different from each other, the server 220 is transmitted by the first electronic device 210 among electronic devices connected to the network. A second electronic device 230 capable of providing service information related to one object information is detected, and service information related to the object information is provided to the detected second electronic device 230. Specifically, the server 220 includes a user's visit frequency, visit time (eg, season, date, time point), residence time, contact information, and vehicle information (eg, information related to object information received from the first electronic device 210) : Transmission information, such as a vehicle model, year, and vehicle number) to the second electronic device 230 as service information. In this case, the user of the second electronic device 230 will be a service provider such as a store owner.

According to various embodiments, the second electronic device 230 may provide the service information received from the server 220 to the user in operation 227. According to an embodiment, the second electronic device 230 may provide service information through a predetermined application. According to another embodiment, when the service information is received from the server 220, the second electronic device 230 generates a notification related to the service information, and based on a user request, through a predetermined application Service information can be provided. For example, the second electronic device 230 may display a graphic user interface (GUI) related to service information. In some embodiments, the second electronic device 230 may provide service information in connection with the server 220 or another server such as a payment server (eg, the server 120 of FIG. 1 ).

According to some embodiments, the server 220 may determine that the location corresponding to the location information is the parking lot in operation 221. Here, the server 220 may determine location information related to the object 231 based on the object information received from the first electronic device 210. However, in the modified examples, the first electronic device 210 directly determines the location based on GPS information and the like, and transmits it to the server 220, so that the server 220 locates the first electronic device 210. You can also decide as a parking lot. In this case, although not shown, the server 220 may request monitoring of the parking lot from the first electronic device 210. In response to this, the first electronic device 210 may analyze the image captured as shown in FIG. 2B to determine the presence (parking) of the vehicle in the parking lot. For example, the first electronic device 210 may detect the solid line 233 in response to the vehicle and the dotted line 235 in response to the empty parking space. In addition, the first electronic device 210 may transmit the determination result to the server 220 in real time or periodically with time information. The server 220 may analyze the determination result received from the first electronic device 210 based on the time information, and update the status data for the corresponding parking lot in real time or periodically.

Based on this, the server 220 may determine service information for transmission to the second electronic device 230 in operation 223. For example, the server 220 may determine whether there is a parking space in the corresponding parking lot or a parking position or a parking state for a specific vehicle in the parking lot (for example, a vehicle owned by the user of the second electronic device 230) as service information. . In addition, the server 220 transmits service information to the second electronic device 230 in operation 225, and the second electronic device 230 provides service information to the user in operation 227.

That is, the server 220 may operate as a parking control system through linkage with the first electronic device 210. Therefore, even if there is no CCTV in the corresponding parking lot, the server 220 can secure the current status data for the corresponding parking lot based on the image taken through the first electronic device 210.

In addition, the server 220 may operate as a parking fee management system. Accordingly, the parking fee payment information for the vehicle may be transmitted to the first electronic device 210 or the second electronic device 230 mounted in the vehicle parked in the corresponding parking lot, and the user may request payment.

3 is a diagram illustrating an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 3, an electronic device 300 according to various embodiments (eg, the electronic device 110 of FIG. 1 and the first electronic device 210 of FIG. 2) includes a camera module 310 and a sensor module It may include at least one of the 320, the input module 330, the output module 340, the interface module 350, the communication module 360, the battery 370, the memory 380 or the processor 390. have. In some embodiments, the electronic device 300 may be mounted on a vehicle and driven.

The camera module 310 may capture still images and videos. For example, the camera module 310 may include at least one of one or more lenses, an image sensor, an image signal processor, or flash.

The sensor module 320 may generate an electrical signal or data value corresponding to an internal operating state (eg, power or temperature) of the electronic device 300 or an external environmental state. For example, the sensor module 320 may include a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an infrared (IR) sensor, a biological sensor, a temperature sensor, a humidity sensor, or It may include at least one of the illuminance sensor.

The input module 330 may input a signal received from the outside. The input module 330 may convert an audio signal into an electrical signal and input it. Alternatively, the input module 330 may receive an instruction or data to be used in the processor 390 from a user, and generate input data. For example, the input module 330 may include at least one of a microphone, button, keypad, keyboard, or mouse.

The output module 340 may output a reproduction signal to the outside. The output module 340 may convert and output an electrical signal into an audio signal. Alternatively, the output module 340 may visually provide output data. For example, the output module 340 may include at least one of a speaker, a receiver, a display, a hologram device, or a projector and a control circuit for controlling the at least one. As an example, the display may include a touch screen display.

The interface module 350 may support a designated protocol for connecting the electronic device 300 to an external device through wired or wireless communication. For example, the interface module 350 may include a connector capable of physically connecting the electronic device 300 to an external device.

The communication module 360 may include an electronic device 300 and a server (eg, the server 120 of FIG. 1, the server 220 of FIG. 2) or other electronic devices (eg, the electronic device 110 of FIG. 1, FIG. 2) It may support performing communication through a communication channel between at least one of the second electronic device (230). The communication module 360 may include at least one of a wireless communication module 361 or a wired communication module 363. For example, the wireless communication module 361 may include at least one of a remote wireless communication module, a short-range wireless communication module, or a location receiving module. To this end, the wireless communication module 361 includes at least one of at least one antenna or a subscriber identification module in which subscriber identification information is stored, and may transmit or receive a radio signal through the antenna using the subscriber identification information. Through this, the communication module 360 may be connected to a server (eg, through a first network (eg, the first network 130 of FIG. 1) or a second network (eg, the second network 140)). To communicate with at least one of the server 120 of FIG. 1, the server 220 of FIG. 2, or other electronic devices (eg, the electronic device 110 of FIG. 1, the second electronic device 230 of FIG. 2 ). Can.

The battery 370 may supply power to at least one component of the electronic device 300. For example, the battery 370 may include a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The memory 380 may store various data used by at least one component of the electronic device 300. The memory 380 may store at least one of input data or output data for software such as a program and related commands. The program may include at least one of an operating system, middleware, or application. For example, the memory 380 may include at least one of volatile memory and nonvolatile memory.

The processor 390 may drive software to control at least one component of the electronic device 300 and perform various data processing and calculations. The processor 390 may include at least one of the main processor 391 or the coprocessor 393. For example, the main processor 391 may include at least one of a central processing unit or an application processor. The coprocessor 393 may use a lower power than the main processor 391 or specialized in a designated function, so that the coprocessor 393 can operate independently of the main processor 391. For example, the coprocessor 393 may include at least one of a graphic processing device, an image signal processor, a sensor hub processor, or a communication processor. According to an embodiment, the coprocessor 393 may replace the main processor 391 while the main processor 391 is in an inactive (eg, sleep) state, and at least one component of the electronic device 300 Can be controlled. According to another embodiment, the coprocessor 393 may control at least one component of the electronic device 300 together with the main processor 391 while the main processor 391 is in an active state. According to another embodiment, the coprocessor 393 may be implemented as other functionally related components, such as some components of the camera module 310 or the communication module 360.

4 is a diagram illustrating a method of operating an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 4, the processor 390 may take an image through the camera module 310 in operation 411. According to an embodiment, while the first electronic device 210 is turned on, the processor 390 may continuously photograph an image. For example, when the first electronic device 210 is mounted on the vehicle, the processor 390 may continuously or periodically collect image information related to the vehicle while photographing the image. The status information may include, for example, at least one of the current location of the vehicle, the amount of fueling, the battery status, or the engine oil status. Here, the processor 390 may take an image regardless of an operation state of the vehicle, for example, whether it is parked or not. According to another embodiment, while the first electronic device 210 is on, the processor 390 may capture an image temporarily or for a time designated by the user based on a user's request.

In operation 413, the processor 390 may detect object information from an image captured through the camera module 310. If the captured image includes at least one object, the processor 390 may detect object information related to the object from the captured image. The object may be formed in various forms, and the shape of the object may vary depending on the position where the object is placed. Objects can be placed on the walls of buildings, signs, signs, large stickers or banners. The shape of the object may include at least one of a barcode, QR code, image, text, color, or pattern. The object information may include at least one of feature information according to at least one feature point recognizable from the object or object identification information for the object.

For example, the processor 390 may analyze the captured image in units of frames. For example, the processor 390 may analyze all frames of an image to be photographed. As another example, the processor 390 may analyze frames spaced at regular intervals among all frames of the image to be photographed. Through this, the processor 390 may extract at least one feature point from at least one of consecutive frames. Also, the processor 390 may detect object information based on the feature point. According to an embodiment, the processor 390 may detect feature information according to the feature point. The feature information may include at least one of an attribute for each feature point or a relationship between a plurality of feature points. For example, the attribute for the feature point may include at least one of the size or color of the feature point, and the relationship between the feature points may include at least one of an arrangement structure, spacing, pattern, or color combination of the feature points. According to another embodiment, the processor 390 may detect object identification information together with feature information. To this end, the memory 380 may store object information. For example, when manufacturing the electronic device 300, object information may be stored in the memory 380. As another example, the processor 390 may download object information from a server (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2 ), and store the object information in the memory 380. As another example, the processor 390 may download object information from a server (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2 ), and update the object information of the memory 380. Through this, the processor 390 may detect object identification information from the stored object information based on the detected feature information.

The processor 390 may transmit object information to the server (eg, the server 120 in FIG. 1 and the server 220 in FIG. 2) through the communication module 360 in operation 415. In some embodiments, when the same object information is repeatedly detected a predetermined number of times, the processor 390 may transmit the object information to the server (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2 ). . According to an embodiment, the processor 390 may transmit feature information according to the feature point to the server (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2 ). According to another embodiment, the processor 390 may transmit object identification information to a server (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2 ).

5 is a diagram illustrating a server according to various embodiments.

Referring to FIG. 5, the server 500 according to various embodiments (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2) includes an input module 510, an output module 520, communication It may include at least one of the module 530, the memory 540, or the processor 550.

The input module 510 may input a signal received from the outside. The input module 510 may convert an audio signal into an electrical signal and input it. Alternatively, the input module 510 may receive commands or data to be used in the processor 550 from an operator of the server 500 and generate input data. For example, the input module 510 may include at least one of a microphone, button, keypad, keyboard, or mouse.

The output module 520 may output a reproduction signal to the outside. The output module 520 may convert and output an electrical signal into an audio signal. Alternatively, the output module 520 may visually provide output data. For example, the output module 520 may include at least one of a speaker, a receiver, a display, a hologram device, or a projector and a control circuit for controlling the same. As an example, the display may include a touch screen display.

The communication module 530 includes a server 500 and an electronic device (eg, the electronic device 110 of FIG. 1, the first electronic device 210 and the second electronic device 230 of FIG. 2, and the electronic device of FIG. 3) 300)) or other electronic devices (eg, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) through the communication channel. The communication module 530 is an electronic device (eg, FIG. 1) through at least one of a first network (eg, the first network 130 of FIG. 1) or a second network (eg, the second network 140 ). Of the electronic device 110, the first electronic device 210 and the second electronic device 230 of FIG. 2, or other electronic device (eg, the electronic device 110 of FIG. 1, the second electronic device of FIG. 2) 230), the electronic device 300 of FIG. 3.

The memory 540 can store various data used by at least one component of the server 500. The memory 540 may store at least one of input data or output data for software such as a program and related commands.

The processor 550 may drive software to control at least one component of the server 500 and perform various data processing and calculations.

6 is a diagram illustrating a method of operating a server according to various embodiments.

Referring to FIG. 6, the processor 550 may store location information 541 and service information 543 in the memory 540 in operation 611. The location information 541 may indicate information related to at least one location where at least one object is disposed. The object may be formed in various forms, and the shape of the object may vary depending on the position where the object is placed. The object may be disposed on a wall surface of a building, a signboard, a display panel, a large sticker, a banner, or the like, or may be disposed in a hologram form at the entrance of the building. The shape of the object may include at least one of a barcode, QR code, image, text, pattern, or color. In the location information 541, an object and a location are mutually mapped and stored, and the location information 541 may include object information for each object and location identification information for each location. For example, the object information may include at least one of feature information according to at least one feature point recognizable from the object or object identification information for the object. According to an embodiment, when an object is disposed at a predetermined location, the processor 550 may store location information 541 in correspondence with the predetermined location. The service information 543 may indicate information related to a service available in at least one location where at least one object is disposed. For example, the service information 543 may include guide information for at least one area corresponding to each location, advertisement information, event information such as coupons, payment information, or a user's frequency of visit, time of visit (eg, season, It may include at least one of relationship information with a user, such as date and time), residence time, contact information, and vehicle information (eg, vehicle model, year, vehicle number). According to various embodiments, the processor 550 generates and stores service information 543 based on data input by an operator of the server 500, or another server (eg, the server 120 of FIG. 1) , Through communication with the server 220 of FIG. 2, the service information 543 may be defined and stored.

The processor 550 may receive object information from the first electronic device (eg, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3) through the communication module 530 in operation 613. . For example, the processor 550 may receive at least one of feature information or object identification information from the first electronic device (eg, the first electronic device 210 of FIG. 2 or the electronic device 300 of FIG. 3 ). Can. In addition, the processor 550 may determine the location information 541 from the memory 540 based on the object information in operation 615. The processor 550 may identify a location where an object corresponding to the object information is placed based on the object information. According to an embodiment, when feature information is received from a first electronic device (eg, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3 ), the processor 550 analyzes the feature information The object identification information may be determined, and the location information 541 may be determined based on the object identification information. According to another embodiment, when the object identification information is received from the first electronic device (eg, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3 ), the processor 550 may transmit the object identification information. Based on this, the location information 541 may be determined. In some embodiments, the processor 550 may manage statistical data such as the number of visits, preferences, and usage status of an area corresponding to each location, based on the number of times the location information 541 is determined.

In addition, the processor 550 may identify the current location in the GPS shaded area using the location information 541 determined based on the object identification information. Through this, the object identification information may be used as a reference point for route guidance in a GPS shaded area or as a reference point for route guidance in indoor navigation. For example, the first electronic device may store indoor data for indoor navigation, and based on the indoor data, the location information 541 may be grasped through object recognition, and thus may be used for route guidance or service provision.

The processor 550 may determine the service information 543 from the memory 540 based on the location information 541 in operation 617. The processor 550 may determine service information 543 related to the location corresponding to the location information 541. For example, when the location where the object is placed corresponds to a department store, mart, accommodation, event place, festival place, gas station, parking lot, car wash, etc., the processor 550 may provide guidance information such as location-related map information or event information, or At least one of event information such as a coupon may be determined as the service information 543. As another example, when the location where the object is disposed corresponds to a toll gate, a drive-through, a gas station, a parking lot, a car wash, the processor 550 provides payment information for payment of an order or billing fee, etc. 543).

In one embodiment, the processor 550 may identify object information according to a location where the object is disposed. For example, when an object is disposed at a gas station entrance or the like, the object information may be gas information or the like, and when the object is disposed at a store entrance or the like, the object information may be a sale item or coupon information. Also, the location of the object may be mapped to information uniquely assigned to each object. The processor 550 may determine service information related to a location corresponding to the location information.

In another embodiment, if the object is placed in an access point (AP) that provides high-speed wireless communication such as a mobile station or 4G/5G-based cellular or wireless fidelity (WIFI), the object information is the first electronic device (eg : Information for installing or updating software of a vehicle equipped with the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3 (software version information, error information, whether an update is necessary) or the first electronic Information for checking the safety level of a vehicle equipped with a device (for example, the first electronic device 210 in FIG. 2 and the electronic device 300 in FIG. 3 ), for example, information for detecting defects in software of the vehicle, etc. Can be In this embodiment, the processor 550 may identify the location of the vehicle to perform software installation, update, and safety inspection procedures through the location information of the object, and the first electronic device (eg, the identified location) The first electronic device 210 of FIG. 2, the electronic device 300 of FIG. 3 ), or software installation, update, and safety inspection of the vehicle may be performed through OTA (Over the air). If the processor 550 is the first electronic device (for example, the first electronic device 210 of FIG. 2, the electronic device 300 of FIG. 3) or software installation of the vehicle, software update, safety inspection is performed for a fee, Determining payment information for the software installation, software update, and safety inspection performed on the second electronic device (for example, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) as service information Can.

In another embodiment, when the object is placed in a distribution warehouse or a delivery place of goods, the object information may be transportation information, delivery information (delivery completion information, delivery place information, etc.), and a second electronic device (for example, FIG. The object information is changed according to whether the electronic device 110 of 1 or the second electronic device 230 of FIG. 2 is a computer or an electronic device carried by a recipient of the logistics management system. Specifically, if the second electronic device (eg, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) is a computer of the logistics management system, the first electronic device (eg, the first electronic device of FIG. 2) The electronic device 210 and the electronic device 300 of FIG. 3 may be mounted on a transportation means such as a truck for delivering the goods, and in this case, the object information of the transportation means to be transmitted to the processor 550 of the logistics system Service information for transmitting to the destination of goods, etc., and being transmitted by the processor 550 to the second electronic device (for example, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) (225) is the first electronic device (for example, the first electronic device of FIG. 2) mounted on a transportation means that transports the goods to the location where the object is located, or payment information for the shipping fee charged to the recipient who received the goods. 210), it may be the payment information for the fare to be paid to the electronic device 300 of FIG. 3 or the logistics information to be notified to the operation server of the transportation means. On the other hand, when the second electronic device (for example, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) is an electronic device carried by the recipient, the processor 550 may display the second electronic device (eg : The service information for transmission to the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2 may be payment information for arrival of goods, arrival time, and transportation charges.

The processor 550 transmits the service information 543 to the second electronic device (for example, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) through the communication module 530 in operation 619. Can. The processor 550 may be configured to correspond to the first electronic device (eg, the electronic device 110 of FIG. 1, the first electronic device 210 of FIG. 2, and the electronic device 300 of FIG. 3) ( Example: The electronic device 110 of FIG. 1, the second electronic device of FIG. 2 (eg, the electronic device 110 of FIG. 1, the second electronic device 230 of FIG. 2) are detected, and the service information is Electronic device (for example, the electronic device 110 of FIG. 1, the second electronic device 230 of FIG. 2) In some embodiments, the processor 550 may be another server (eg, the server of FIG. 1 ( 120). As an example, the processor 550 may cause another server (eg, the server 120 of FIG. 1) to transmit a second electronic device (eg, the electronic device of FIG. 1). 110), it is possible to transmit the service information 543 to the second electronic device 230 of Figure 2. As another example, when the service information 543 includes payment information, the processor 550 is a payment server The service information 543 may be transmitted to another server (eg, the server 120 of FIG. 1) to enable the user's payment. Through this, a server (eg, the server 120 of FIG. 1), The server 220 of FIG. 2 may use the service information to perform a business for selling a product or providing a service for a corresponding company under a contract with a specific company, and obtain profits therefrom.

According to some embodiments, the processor 550 may determine that the location corresponding to the location information is a parking lot in operation 615. Here, the processor 550 is based on the object information received from the first electronic device (eg, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3 ), and the location associated with the object 231 Information can be determined. In this case, although not shown, the processor 550 may request monitoring of the parking lot from the first electronic device (eg, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3 ). In response to this, the first electronic device (for example, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3) analyzes an image photographed as shown in FIG. 2B, and in the parking lot. It is possible to judge the presence or absence of a vehicle (parking). For example, the processor 550 may detect the solid line 233 in response to the vehicle, and detect the dotted line 235 in response to the empty parking space. In addition, the first electronic device (for example, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3) displays the determination result together with time information in real time or periodically (eg, the server of FIG. 1 ). (120), the server 220 of FIG. 2). The processor 550 analyzes the determination result received from the first electronic device (for example, the first electronic device 210 of FIG. 2 and the electronic device 300 of FIG. 3) based on time information, so that the Status data can be updated in real time or periodically.

Based on this, the processor 550 may determine service information for transmission to the second electronic device 230 in operation 617. For example, the processor 550 may determine whether there is a parking space in the corresponding parking lot or a parking location or parking state or payment for a specific vehicle in the corresponding parking lot as service information. In addition, the processor 550 may transmit service information to the second electronic device (eg, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) in operation 619.

That is, the server (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2) is the first electronic device (eg, the first electronic device 210 of FIG. 2, the electronic device 300 of FIG. 3) Through linkage with, it can operate as a parking control system. Therefore, even if there is no CCTV in the corresponding parking lot, a server (for example, the server 120 of FIG. 1 and the server 220 of FIG. 2) is based on the image taken through the first electronic device 210, and for the corresponding parking lot. Status data can be secured.

7 is a diagram illustrating an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 7, the electronic device 700 according to various embodiments (eg, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) includes a camera module 710 and a sensor module 720, an input module 730, an output module 740, an interface module 750, a communication module 760, a battery 770, a memory 780, or a processor 790. have. Here, the components of the electronic device 700 may be the same or similar to those of the electronic device 300 of FIG. 3, and overlapping descriptions will be omitted below.

8 is a diagram illustrating a method of operating an electronic device according to various embodiments of the present disclosure. 9A, 9B, and 9C are diagrams illustrating an example of a screen displayed when providing service information in FIG. 8.

Referring to FIG. 8, a predetermined application may be installed in the electronic device (eg, the electronic device 110 of FIG. 1 and the second electronic device 230 of FIG. 2) 700 in operation 811. For example, the predetermined application may be an application related to location-based services. To this end, the processor 790 may download and install a predetermined application from the application store through the communication module 760.

The processor 790 may receive service information from a server (eg, the server 120 of FIG. 1, the server 220 of FIG. 2, and the server 500 of FIG. 5) through the communication module 760 in operation 813. have. The service information may indicate information related to a service available at a specific location, such as a user location. For example, the service information may include at least one of guide information, advertisement information, event information such as coupons, or payment information for at least one area corresponding to each location. The service information may be determined based on at least one object disposed in a specific location, for example, a user location in a server (eg, the server 120 in FIG. 1, the server 220 in FIG. 2, and the server 500 in FIG. 5). Can. The object may be formed in various forms, and the shape of the object may vary depending on the position where the object is placed. The object may be disposed on a wall surface of a building, a signboard, a display panel, a large sticker, a banner, or the like, or may be arranged in a hologram form at the entrance of the building. The shape of the object may include at least one of a barcode, QR code, image, text, pattern, or color.

For example, if the location where the object is placed corresponds to a department store, mart, lodging facility, event place, festival place, gas station, parking lot, car wash, etc., the service information may include information or coupons such as map information related to the location or event information. At least one of the same event information may be included.

As another example, when the location where the object is disposed corresponds to a toll gate, a drive-through, a gas station, a parking lot, a car wash, etc., the service information may include payment information for payment of an order or billing cost.

As another example, if the location where the object is located is an access point (AP) that provides high-speed wireless communication such as a workshop or a 4G/5G-based cellular or wireless fidelity (WIFI) service information is provided to other electronic devices ( Example: The first electronic device 2101 of FIG. 2, the electronic device 300 of FIG. 3, or another electronic device (eg, the first electronic device 210 of FIG. 2, the electronic device 300 of FIG. 3) It may include payment information for software installation, software update, and safety inspection performed on the mounted vehicle.

As another example, when the location where the object is disposed is a distribution warehouse or a delivery location of goods, the service information may be different depending on whether the electronic device 700 is a computer in the logistics management system or an electronic device carried by the recipient. If the electronic device 700 is a computer of the logistics management system, the service information is payment information for a shipping fee charged to the recipient who received the goods or another electronic device mounted on a transportation means that transports the goods to the location where the object is located (for example, : The first electronic device 210 in FIG. 2 and the electronic device 300 in FIG. 3 may be payment information for freight to be paid or logistics information to be notified to the operation server of the transportation means. On the other hand, when the electronic device 700 is an electronic device carried by the recipient, the service information may be payment information for the arrival of the goods, arrival time, and transportation charges.

The processor 790 may provide service information to the user in operation 815. According to an embodiment, the processor 790 may provide service information through a predetermined application. According to another embodiment, when service information is received from a server (eg, the server 120 of FIG. 1, the server 220 of FIG. 2, and the server 500 of FIG. 5 ), the processor 790 is associated with the service information. A notification may be generated, and service information may be provided through a predetermined application based on a user's request. In some embodiments, the processor 790 may communicate with another server, such as a server (eg, the server 120 in FIG. 1, the server 220 in FIG. 2, the server 500 in FIG. 5) or a payment server through the communication module 760. In connection with a server (for example, the server 120 of FIG. 1 and the server 220 of FIG. 2 ), service information may be provided.

For example, the processor 790 may provide service information using at least one of text, image, map, or voice. The processor 790 may display a graphic user interface (GUI) related to service information as illustrated in FIGS. 9A, 9B, or 9C. For example, when the location where the object is placed corresponds to the AA department store, the processor 790 provides guidance information such as map information for each floor in the AA department store and event information such as coupons as service information as illustrated in FIG. 9A. Can. As another example, when the location where the object is placed corresponds to a toll gate in the XX region, the processor 790 bills tolls for the toll gate in the XX region and bills information to guide the payment of the toll, as illustrated in FIG. 9B. Information such as local event information can be provided as service information. As another example, when the location where the object is placed corresponds to the drive-through of the YY burger ZZ point, the processor 790 provides payment information for guiding an order and payment of a desired menu as shown in FIG. 9C as service information. can do. 9B or 9C, when the service information includes payment information, the processor 790 is a server (eg, the server 120 of FIG. 1, the server 220 of FIG. 2, the server of FIG. 5) 500)) or other servers such as a payment server (eg, the server 120 of FIG. 1 and the server 220 of FIG. 2), and perform electronic payment.

According to various embodiments, a server (eg, an electronic device 110 of FIG. 1, a first electronic device 210 of FIG. 2, and an electronic device 300 of FIG. 3) does not require a user input. : The server 120 of FIG. 1, the server 220 of FIG. 2, and the server 500 of FIG. 5 may provide an appropriate location-based service for the user. That is, an electronic device (eg, the electronic device 110 of FIG. 1, the first electronic device 210 of FIG. 2, and the electronic device 300 of FIG. 3) based on an object detected during image capture, a server (eg: The server 120 of FIG. 1, the server 220 of FIG. 2, and the server 500 of FIG. 5 may grasp the user location. For this reason, a separate user input is unnecessary in the electronic device (for example, the electronic device 110 of FIG. 1, the first electronic device 210 of FIG. 2, and the electronic device 300 of FIG. 3 ). And since the server (for example, the server 120 in FIG. 1, the server 220 in FIG. 2, and the server 500 in FIG. 5) determines service information based on the user's location, the user may respond to the current location in time. Appropriate service information is available according to the location and location. Therefore, the user can efficiently use the location-based service.

10 is a diagram illustrating a vehicle equipped with an electronic device according to various embodiments of the present disclosure. 11 is a diagram illustrating the electronic device of FIG. 10.

10 and 11, a control device 2100 according to various embodiments (eg, the electronic device 110 of FIG. 1, the first electronic device 210 of FIG. 2A, and the second electronic device of FIG. 2A) The electronic device 300 of FIG. 3, the electronic device 300 of FIG. 3, and the electronic device 700 of FIG. 7 may be mounted on a vehicle. At this time, the vehicle may be an autonomous vehicle 2000.

In the present embodiment, the control device 2100 includes a memory (eg, the memory 380 of FIG. 3, the memory 780 of FIG. 7) 2122 and a processor (eg, the processor 390, the processor 790 of FIG. 7). ) 2124 includes a controller 2120, a sensor (e.g., sensor module 320 of FIG. 3, sensor module 720 of FIG. 7) 2110, a wireless communication device (e.g., communication module of FIG. 3 ( 360), communication module 760 of FIG. 7, 2130, LIDAR 2140, and a camera module (eg, camera module 310 of FIG. 3, camera module 710 of FIG. 7) 2150. Can.

In this embodiment, the controller 2120 may be configured at the time of manufacture by the manufacturer of the vehicle, or additionally configured to perform the function of autonomous driving after manufacturing. Alternatively, a configuration for continuously performing additional functions may be included through an upgrade of the controller 2120 configured at the time of manufacture.

The controller 2120 includes a sensor 2110, an engine 2006, a user interface 2008, a wireless communication device 2130, a LIDAR 2140, and a camera module 2150, which include control signals in other components in the vehicle. Can be delivered to. In addition, although not shown, a control signal may be transmitted to an acceleration device, a braking system, a steering device, or a navigation device related to driving of a vehicle.

In this embodiment, the controller 2120 may control the engine 2006, for example, the engine 2006 so that the autonomous vehicle 2000 detects the speed limit of the road on which it is driving and the driving speed does not exceed the speed limit. ), or may control the engine 2006 to accelerate the driving speed of the autonomous vehicle 2000 within a range not exceeding the speed limit. In addition, when the sensing module (2004a, 2004b, 2004c, 2004d) detects the environment outside the vehicle and transmits it to the sensor 2110, the controller 2120 receives it and receives an engine 2006 or a steering device (not shown). The driving of the vehicle can be controlled by generating a control signal.

The controller 2120 may control the engine 2006 or the braking system to decelerate the driving vehicle when other vehicles or obstacles are present in front of the vehicle, and may control trajectory, driving path, and steering angle in addition to speed. . Alternatively, the controller 2120 may control driving of the vehicle by generating a necessary control signal according to recognition information of other external environments such as the driving lane of the vehicle, a driving signal, and the like.

In addition to the generation of its own control signal, the controller 2120 can also control the driving of the vehicle by performing communication with the surrounding vehicle or the central server and transmitting a command for controlling the peripheral devices through the received information.

In addition, since the controller 2120 may be difficult to recognize an accurate vehicle or lane according to the present embodiment when the position of the camera module 2150 is changed or the angle of view is changed, calibration of the camera module 2150 to prevent this ( It is also possible to generate a control signal that controls to perform calibration. Therefore, in this embodiment, the controller 2120 generates a calibration control signal to the camera module 2150, so that the mounting position of the camera module 2150 is caused by vibration or shock generated according to the movement of the autonomous vehicle 2000. Even if changed, it is possible to continuously maintain the normal mounting position, direction, angle of view, etc. of the camera module 2150. In the controller 2120, the initial mounting position, direction, and angle of view information of the previously stored camera module 2120 and the initial mounting position, direction, and angle of view information of the camera module 2120 measured during driving of the autonomous vehicle 2000 are critical. If it is more than the value, a control signal may be generated to perform calibration of the camera module 2120.

In this embodiment, the controller 2120 may include a memory 2122 and a processor 2124. The processor 2124 may execute software stored in the memory 2122 according to the control signal of the controller 2120. Specifically, the controller 2120 stores data and instructions for performing the lane display method or the lane departure guidance method according to the present invention in the memory 2122, and the instructions 2124 to implement one or more methods disclosed herein. ).

At this time, the memory 2122 may be stored in a recording medium executable on the nonvolatile processor 2124. The memory 2122 can store software and data through an appropriate internal or external device. The memory 2122 may include a random access memory (RAM), a read only memory (ROM), a hard disk, and a memory 2122 device connected to a dongle.

The memory 2122 may store at least operating system (OS), user application, and executable commands. The memory 2122 may also store application data and array data structures.

The processor 2124 may be a microprocessor or a suitable electronic processor, which may be a controller, microcontroller, or state machine.

The processor 2124 may be implemented as a combination of computing devices, and the computing device may be a digital signal processor, a microprocessor, or an appropriate combination thereof.

In addition, in the present embodiment, the control device 2100 may monitor characteristics of the inside and outside of the autonomous vehicle 2000 and detect a state with at least one sensor 2110.

The sensor 2110 may be composed of at least one sensing module 2004, and the sensing module 2004 may be implemented at a specific location of the autonomous vehicle 2000 according to the detection purpose. The autonomous vehicle 2000 may be located at a lower portion, a rear end, a front end, an upper end, or a side end of the autonomous vehicle, and may also be located in an internal part of a vehicle or a tire.

Through this, the sensing module 2004 can detect information related to driving such as engine 2006, tires, steering angle, speed, and weight of the vehicle as internal information of the vehicle. In addition, the at least one sensing module 2004 may be configured as an acceleration sensor 2110, a gyroscope, an image sensor 2110, a RADAR, an ultrasonic sensor, a LiDAR sensor, and the like, and display motion information of the autonomous vehicle 2000. Can be detected.

The sensing module 2004 may receive specific data about the external environment state, such as the state information of the road on which the autonomous vehicle 2000 is located, the surrounding vehicle information, the weather as external information, and detect the vehicle parameters accordingly. Do. The detected information may be temporarily or long-term stored in the memory 2122 according to the purpose.

In this embodiment, the sensor 2110 may collect and collect information of sensing modules 2004 for collecting information generated from inside and outside of the autonomous vehicle 2000.

The control device 2100 may further include a wireless communication device 2130.

The wireless communication device 2130 is configured to implement wireless communication between the autonomous vehicles 2000. For example, the user's mobile phone or other wireless communication device 2130, another vehicle, a central device (traffic control device), a server, and the like, enables the autonomous vehicle 2000 to communicate. The wireless communication device 2130 may transmit and receive a wireless signal according to an access wireless protocol. The wireless communication protocol may be Wi-Fi, Bluetooth, Long-Term Evolution (LTE), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Global Systems for Mobile Communications (GSM), and the communication protocol is It is not limited to this.

In addition, in the present embodiment, the autonomous vehicle 2000 may also implement communication between vehicles through the wireless communication device 2130. That is, the wireless communication device 2130 may communicate with other vehicles and other vehicles on the road through vehicle-to-vehicle communication. The autonomous vehicle 2000 may transmit and receive information such as driving warnings and traffic information through communication between vehicles, and may request or receive information from other vehicles. For example, the wireless communication device 2130 may perform V2V communication as a designated short-range communication (DSRC) device or a C-V2V (Celluar-V2V) device. In addition, communication between the vehicle and another object (for example, an electronic device carried by a pedestrian, etc.) in addition to communication between vehicles (V2X, Vehicle to Everything communication) may also be implemented through the wireless communication device 2130.

Also, the control device 2100 may include a LIDAR device 2140. The LIDAR device 2140 may detect an object around the autonomous vehicle 2000 during operation using data sensed through the LIDAR sensor. The LIDAR device 2140 may transmit the detected information to the controller 2120, and the controller 2120 may operate the autonomous vehicle 2000 according to the detection information. For example, the controller 2120 may instruct the vehicle to reduce the speed through the engine 2006 when there is a forward vehicle traveling at a low speed in the detection information. Or, the vehicle may be commanded to reduce the entry speed according to the curvature of the entering curve.

The control device 2100 may further include a camera module 2150. The controller 2120 may extract object information from an external image photographed by the camera module 2150 and allow the controller 2120 to process the information.

In addition, the control device 2100 may further include imaging devices for recognizing an external environment. In addition to the LIDAR 2140, a RADAR, GPS device, odometer, and other computer vision devices can be used, and these devices can be selected or operated as needed to enable more precise detection.

The autonomous vehicle 2000 may further include a user interface 2008 for user input to the above-described control device 2100. The user interface 2008 may allow the user to enter information through appropriate interaction. For example, it may be implemented with a touch screen, a keypad, and operation buttons. The user interface 2008 may transmit an input or command to the controller 2120, and the controller 2120 may perform a control operation of the vehicle in response to the input or command.

In addition, the user interface 2008 may be configured to communicate with the autonomous vehicle 2000 through a wireless communication device 2130 as a device external to the autonomous vehicle 2000. For example, the user interface 2008 may enable interworking with a mobile phone, tablet, or other computer device.

Furthermore, although the autonomous vehicle 2000 in this embodiment has been described as including the engine 2006, it is also possible to include other types of propulsion systems. For example, the vehicle may be driven by electric energy, and may be driven by hydrogen energy or a hybrid system combining them. Accordingly, the controller 2120 may include a propulsion mechanism according to the propulsion system of the autonomous vehicle 2000, and provide a control signal accordingly to the components of each propulsion mechanism.

Hereinafter, a detailed configuration of the control device 2100 for performing object information detection and object information guidance method according to this embodiment will be described in more detail with reference to FIG. 11.

The control device 2100 includes a processor 2124. The processor 2124 may be a general purpose single or multi-chip microprocessor, dedicated microprocessor, microcontroller, programmable gate array, or the like. The processor may also be referred to as a central processing unit (CPU). Also, in the present embodiment, the processor 2124 may be used as a combination of a plurality of processors.

The control device 2100 also includes a memory 2122. The memory 2122 may be any electronic component capable of storing electronic information. Memory 2122 may also include a combination of memories 2122 in addition to a single memory.

Data and instructions 2122a for performing object information detection and object information guidance method according to the present invention may be stored in the memory 2122. When the processor 2124 executes the instructions 2122a, all or part of the instructions 2122a and data 2122b required for the execution of the instructions may be loaded 2124a, 2124b onto the processor 2124.

The control device 2100 may include a transmitter 2130a, a receiver 2130b, or a transceiver 2130c to allow transmission and reception of signals. The one or more antennas 2132a and 2132b may be electrically connected to the transmitter 2130a, the receiver 2130b, or each transceiver 2130c, and may additionally include antennas.

The control device 2100 may include a digital signal processor (DSP) 2170. Through the DSP 2170, a digital signal can be quickly processed by the vehicle.

The control device 2100 may also include a communication interface 2180. The communication interface 2180 may include one or more ports and/or communication modules for connecting other devices to the control device 2100. The communication interface 2180 may enable the user and the control device 2100 to interact.

Various configurations of the control device 2100 may be connected together by one or more buses 2190, and the buses 2190 may include a power bus, a control signal bus, a status signal bus, a data bus, and the like. Under the control of the processor 2124, the components may transfer mutual information through the bus 2190 and perform a desired function.

Although various embodiments of the present document have been described, various modifications are possible without departing from the scope of the various embodiments of the present document. Therefore, the scope of various embodiments of the present document should not be limited to the described embodiments, but should be determined not only by the scope of the claims described below, but also by the scope and equivalents of the claims.

Claims (30)

In the electronic device,
A camera module configured to shoot an image;
A communication module configured to wirelessly communicate with the server; And
It includes a processor connected to the camera module and the communication module,
The processor,
Detecting object information related to at least one object present in the captured image,
An electronic device configured to transmit the object information to the server.
According to claim 1,
The object is an electronic device including at least one of a barcode, a QR code, an image, text, or color.
According to claim 1,
The object information includes at least one of feature information representing at least one feature point recognized from the object or identification information about the object.
The method of claim 3, wherein the processor,
The feature point is extracted from the captured image,
Based on the feature point, the feature information is detected,
An electronic device configured to transmit the feature information to the server.
The method of claim 3,
Further comprising a memory configured to store the object information,
The processor,
The feature point is extracted from the captured image,
Based on the feature point, the identification information is detected,
And an electronic device configured to transmit the identification information to the server.
The method of claim 1, wherein the processor,
An electronic device configured to transmit the object information to the server when the object information is repeatedly detected a predetermined number of times within a predetermined time.
According to claim 1,
The server stores at least one of location information related to a location corresponding to the object information or service information related to a service available at the location.
In the operation method of the electronic device,
Shooting an image;
Detecting object information related to at least one object present in the captured image; And
And wirelessly transmitting the object information to a server.
The method of claim 8,
The object comprises at least one of a barcode, QR code, image, text or color.
The method of claim 8,
The object information includes at least one of feature information representing at least one feature point recognized from the object or identification information about the object.
11. The method of claim 10, The object information detection operation,
Extracting the feature points from the captured image; And
And detecting the feature information based on the feature point.
11. The method of claim 10, The object information detection operation,
Extracting the feature points from the captured image; And
And detecting the identification information based on the feature point.
The method of claim 8, wherein the object information transmission operation,
And if the object information is repeatedly detected a predetermined number of times within a predetermined time, transmitting the object information to the server.
The method of claim 8,
The server stores at least one of location information related to a location corresponding to the object information or service information related to a service available at the location.
On the server,
A communication module configured to wirelessly communicate with the at least one electronic device; And
And a processor operatively connected to the communication module,
The processor,
Through the communication module, receives object information related to at least one object,
Based on the object information, identify the location of the object,
A server configured to transmit service information related to the location through the communication module.
The method of claim 15,
And a memory configured to store the location information and the service information in which the object and location are mutually mapped.
The method of claim 15,
The object is a server comprising at least one of a barcode, QR code, image, text or color.
The method of claim 16,
The service information may include information such as guide information, advertisement information, coupon information for at least one area corresponding to the location information, payment information, or visit frequency, visit time, residence time, contact information, and vehicle information. Server comprising at least one of the relationship information.
The method of claim 15, wherein the processor,
When the object information is received from a first electronic device, a server configured to transmit the service information to a second electronic device.
The method of claim 19,
The second electronic device is a server configured or configured differently from the first electronic device.
In the server operation method,
Receiving object information related to at least one object from a first electronic device;
Identifying a location of the object based on the object information; And
And transmitting service information related to the location to a second electronic device.
The method of claim 21,
A method in which the location information in which the object and the location are mutually mapped and the service information are previously stored in the server.
The method of claim 21,
The object comprises at least one of a barcode, QR code, image, text or color.
The method of claim 22,
The service information includes at least one of guide information, advertisement information, event information such as coupons, or payment information for at least one area corresponding to the location information.
The method of claim 21,
The second electronic device is configured to be the same as or different from the first electronic device.
In the operation method of the electronic device,
Wirelessly receiving service information related to a location of the object determined based on at least one object from a server; And
And providing the service information based on a pre-installed application.
The method of claim 26,
The object comprises at least one of a barcode, QR code, image, text or color.
The method of claim 26,
The service information is information on at least one area corresponding to the location, advertisement information, event information such as coupons, payment information or visit frequency, visit time, residence time, contact information, relationship with a user such as vehicle information How to include at least one of the information.
27. The method of claim 26, wherein the service information providing operation,
And providing the service information using at least one of text, image, map, or voice.
27. The method of claim 26, wherein the service information providing operation,
And performing an electronic payment based on the service information.

Images