WO2019024010A1 - 一种图像处理方法、系统以及智能导盲装置 - Google Patents

一种图像处理方法、系统以及智能导盲装置 Download PDF

Info

Publication number
WO2019024010A1
WO2019024010A1 PCT/CN2017/095676 CN2017095676W WO2019024010A1 WO 2019024010 A1 WO2019024010 A1 WO 2019024010A1 CN 2017095676 W CN2017095676 W CN 2017095676W WO 2019024010 A1 WO2019024010 A1 WO 2019024010A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
module
terminal
user
quality standard
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/095676
Other languages
English (en)
French (fr)
Inventor
王宁
刘兆祥
廉士国
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Cloudminds Shenzhen Robotics Systems Co Ltd
Original Assignee
Cloudminds Shenzhen Robotics Systems Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Cloudminds Shenzhen Robotics Systems Co Ltd filed Critical Cloudminds Shenzhen Robotics Systems Co Ltd
Priority to CN201780003265.9A priority Critical patent/CN108366899A/zh
Priority to PCT/CN2017/095676 priority patent/WO2019024010A1/zh
Publication of WO2019024010A1 publication Critical patent/WO2019024010A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61H—PHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H3/00—Appliances for aiding patients or disabled persons to walk about
    • A61H3/06—Walking aids for blind persons
    • A61H3/061—Walking aids for blind persons with electronic detecting or guiding means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence

Definitions

  • the present application relates to the field of intelligent guiding blindness, and in particular to an image processing method, system and intelligent guiding device.
  • the blind are vulnerable groups who need the help of the society to help them improve their ability to live independently and have a better quality of life. Therefore, products that assist blind people to travel freely and cognitively are a trend in blind aids and blinding software research.
  • Chinese Patent Application No. 201310711816.0 discloses an intelligent guide stick based on image recognition, which belongs to the technical field of electronic information engineering.
  • the intelligent guide stick is composed of a hand stick handle, a smart stick body, a roller stick tip and a Bluetooth earphone.
  • the human-machine interaction subsystem, the road condition recognition subsystem, the navigation subsystem and the wheel system are embedded in the guide stick.
  • the smart wand body is located in the middle of the guide wand, is hollow inside, and is embedded in the road condition recognition subsystem.
  • the road condition recognition subsystem includes a microprocessor; the camera and the LED lighting device are connected to the microprocessor;
  • the road condition recognition subsystem is connected with the navigation subsystem of the roller stick tip; it adopts the image recognition processing method to effectively identify blind roads and public transportation information, and assists the blind person to travel safely and independently.
  • the existing intelligent guide sticks help blind people to travel safely and independently through image recognition of blind roads and public transportation information. Can not assist navigation and more comprehensive help blind people understand the surrounding environment.
  • the obstacle avoidance effect is not good enough, and if the blind guide device helps the blind person to understand the surrounding environment through image recognition, the amount of data processed is large, there is a delayed response phenomenon, and there is a safety hazard in a specific environment.
  • the embodiment of the present application provides an image processing method, a system, and an intelligent guide blind device, which are used to reduce the consumption of network resources, and at the same time, can achieve accurate obstacle avoidance.
  • an embodiment of the present application provides an image processing method, including the following steps: at a terminal, when a terminal is in a motion state, acquiring a video image around the terminal and uploading; when the terminal is in a static state, acquiring the terminal Image of the image and upload it.
  • an embodiment of the present application provides an image processing system, including a terminal and a cloud server that wirelessly communicates with the terminal, where the terminal includes a video acquisition module, a picture acquisition module, and a sending module.
  • the video acquisition module acquires a video image around the terminal and uploads it to the cloud server by the sending module.
  • the image acquiring module acquires a picture image around the terminal and is uploaded by the sending module to the cloud. server.
  • the application of the intelligent guide blind device is provided.
  • the embodiment of the present application provides an intelligent guide blind device, including: a video acquisition module, a picture acquisition module, and a sending module.
  • the video acquisition module acquires a video image around the terminal and is uploaded by the sending module; when the smart guide device is in a stationary state, the image acquisition module acquires a picture image around the terminal and is sent by the sending module. Upload.
  • the embodiment of the present application provides an image processing method, including the following steps:
  • Generating real-time obstacle avoidance information which is based on the video image and/or the image image, identifying a navigation state and environment information of the user, and generating the navigation state and the environment information based on the navigation state and the environment information;
  • the embodiment of the present application further provides an electronic device, including:
  • At least one processor and,
  • the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to establish a data channel through the communication component to enable the at least one processor to perform the method as described above.
  • the embodiment of the present application further provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions, when When the program instructions are executed by the computer, the computer is caused to perform the method as described above.
  • the image processing method and system and the intelligent guide blind device provided by the embodiments of the present application automatically acquire an appropriate image according to the state and environment of the user, and when the user is in a motion state, the user is acquired.
  • the video image when the user is in a stationary state, the image of the picture around the user is acquired to reduce the consumption of network resources.
  • the real-time obstacle avoidance information includes the identified barrier-free route and the obstacle avoidance travel instruction, providing the blind person with more real-time, more convenient and more accurate obstacle avoidance information and realizing safety guide. Blind navigation.
  • FIG. 1 is a system architecture diagram of an image processing system according to an embodiment of the present application.
  • FIG 2 is another system architecture diagram of an image processing system provided by an embodiment of the present application.
  • FIG. 3 is a main flowchart of an image processing system provided by an embodiment of the present application.
  • FIG. 5 is a block diagram of an embodiment of an intelligent guide blind device according to an embodiment of the present application.
  • FIG. 6 is a block diagram of another embodiment of an intelligent guide blind device according to an embodiment of the present application.
  • FIG. 7 is a flowchart of processing an intelligent guide blind device cooperated with a cloud server according to an embodiment of the present application
  • FIG. 8 is a flowchart of a cloud server provided by an embodiment of the present application.
  • FIG. 9 is a block diagram of a cloud server provided by an embodiment of the present application.
  • FIG. 10 is a hardware framework diagram of an image processing method provided by an embodiment of the present application.
  • the image processing method and system, the intelligent guide blind device, and the cloud server provided by the embodiment of the present application acquire the state and environment of the user in real time, select different types of images according to the state and the environment, and perform image processing to reduce the consumption of network resources.
  • the navigation state and environment information can be identified according to the image and the real-time obstacle avoidance information for guiding the blind interaction can be provided, and no more manual operation is needed to provide more real-time and accurate obstacle avoidance suggestions and navigation feedback.
  • the intelligent guide blind device of the present embodiment can be applied to the guide blind helmet 10 as an intelligent guide blind terminal, and can be applied to the guide blind glasses 20 , and can also be applied to the guide blind cane, and can also be applied to other guides.
  • Blind wearable device As shown in FIG. 1 , the intelligent guide blind device of the present embodiment can be applied to the guide blind helmet 10 as an intelligent guide blind terminal, and can be applied to the guide blind glasses 20 , and can also be applied to the guide blind cane, and can also be applied to other guides.
  • Blind wearable device can be applied to the guide blind helmet 10 as an intelligent guide blind terminal, and can be applied to the guide blind glasses 20 , and can also be applied to the guide blind cane, and can also be applied to other guides.
  • the smart guide device sets one or more CPUs, and adds a GPU to complete data analysis and image recognition functions when necessary to meet image recognition requirements.
  • the smart guide device can be equipped with an android operating system, or an iOS operating system or a Windows Phone operating system applied to the mobile terminal.
  • the cloud server 51-53 receives the image information sent by the smart guide device in a big data processing manner.
  • the intelligent guide device system collects user travel status and environment information through sensors such as a GPS module or a Beidou positioning module, a gyroscope, and a camera, and monitors the state of the user.
  • the video image around the video capture user is activated while the user is in the mobile state.
  • the captured video image is degraded, and then encoded and compressed and transmitted to the cloud server 50.
  • the cloud server 50 uses an algorithm such as obstacle detection to perform image recognition to calculate a barrier-free route and identify the navigation of the user.
  • Status and environmental information and based on the navigation status and environmental information to provide obstacle avoidance information, and returned to the user's intelligent guide device in real time.
  • the picture image When the user is in a static state, the picture image is collected. In order to reduce the load of the data transmission, the collected picture image is degraded, and then encoded and compressed and transmitted to the cloud server 50, and the cloud server 50 calls an object recognition algorithm to perform the object. Identification and environmental information identification, providing obstacle avoidance information based on the identified object and environmental information, and returning the obstacle avoidance information to the user's intelligent guide blind device in real time.
  • the intelligent guide blind system of the embodiment can be applied to a guide blind helmet. Products such as mobile terminal guide applications or other wearable guide blind devices.
  • this embodiment provides an image processing system.
  • the image processing system generally includes a plurality of terminals and a cloud server 50 that wirelessly communicates with the terminal.
  • the terminal includes a video acquisition module 32b, a picture acquisition module 32a, and a transmission module 35.
  • the video acquisition module 32b acquires a video image around the terminal and uploads it to the cloud server 50 by the sending module 35.
  • the image acquiring module 32a acquires a picture image around the terminal and uploads it to the cloud server 50 by the sending module 35.
  • the terminal of the image processing system is a plurality of intelligent guide blind devices 10, 20, wherein each smart guide blind device 10, 20 is in wireless communication with the cloud server 50.
  • the cloud server 50 is a big data processing center, and the server cluster can be formed by a plurality of server networkings 51-53.
  • the smart guide device can be a guide blind helmet 10, a mobile terminal guide application or a wearable guide blind glasses. In the present embodiment, the guide blind helmet 10 will be described as an example.
  • the smart guide device includes a detection module 31, an image acquisition module 32, an adjustment module 33, a transmission module 35, a receiving module 36, and a voice interaction module. 37 and a generation module 39.
  • the image acquisition module 32 includes the video acquisition module 32b and a picture acquisition module 32a.
  • the detection module 31 monitors the user's travel status and receives the user status parameters.
  • the image obtaining module 32 is configured to determine a user traveling state based on the state parameter, and when the user is in a motion state, the video acquiring module 32b acquires a video image around the user; when the user is in a stationary state, the image acquiring module 32a acquires the user. Image of the surrounding picture.
  • the adjustment module 33 adjusts the file size of the video image and/or the picture image to reduce the data processing amount while ensuring the image recognition quality, and reduces the data transmission amount and reduces the consumption of network resources.
  • the sending module 35 is configured to transmit data to the cloud server.
  • the adjustment module 33 includes a compression module 34.
  • the compression module 34 adjusts the acquired video image to a first image quality standard that is less than a quality standard of the image image.
  • the video image under the first image quality standard is used for obstacle recognition, and the detection module 31 determines the navigation state based on the identified obstacle.
  • the adjustment module 33 adjusts the acquired picture image to a second image quality standard.
  • a second image quality standard can be used for the identification of environmental information.
  • the generating module 39 is configured to identify the navigation state and environment information of the user according to the adjusted video image and/or picture image, and provide obstacle avoidance information based on the navigation state and the environment information.
  • the generating module 39 is further configured to determine the confidence of the navigation state and the environment information.
  • the obstacle avoidance information includes an obstacle avoidance travel instruction such as an obstacle avoidance travel instruction such as stop/forward/left turn/right turn.
  • the voice interaction module 37 plays the obstacle avoidance information to the user in a voice manner, and recognizes the voice feedback of the user by voice to complete the voice interaction control.
  • the video image and/or the image image is sent to the manual server 60 by the sending module 35, and the feedback of the feedback sent by the manual server 60 is obtained by the receiving module 36.
  • Barrier information If the confidence of the navigation state and the environment information is lower than the threshold, the video image and/or the image image is sent to the manual server 60 by the sending module 35, and the feedback of the feedback sent by the manual server 60 is obtained by the receiving module 36. Barrier information.
  • the generating module 42 is disposed on the cloud server 50 in order to reduce local data processing and facilitate centralized upgrade unified management.
  • the generating module 42 is further configured to determine the confidence of the navigation state and the environment information.
  • the obstacle avoidance information includes an obstacle avoidance travel instruction such as an obstacle avoidance travel instruction such as stop/forward/left turn/right turn.
  • the cloud server 50 sends the generated obstacle avoidance travel instruction to the intelligent guide device 10 of the data source, and the voice interaction module 37 of the smart guide device 10 plays the obstacle avoidance information to the user in a voice manner, and recognizes the voice of the user by voice. Feedback to complete voice interaction control.
  • the manual server 60 is connected, and the manually transmitted obstacle avoidance information is acquired from the manual server 60, and then sent to the corresponding smart guide device 10.
  • the image processing method comprises the following steps:
  • Step 101 At the terminal, when the terminal is in a motion state, acquire a video image around the terminal and upload it;
  • Step 102 When the terminal is in a stationary state, acquire a picture image around the terminal and upload it.
  • the terminal may be a smart guide device. So it also includes the following steps:
  • Determining a user traveling state according to the state parameter acquiring a video image around the user when the user is in a motion state; acquiring a picture image around the user when the user is in a stationary state;
  • the navigation state and environment information of the user are identified according to the adjusted video image and/or picture image, and obstacle avoidance information is provided based on the navigation state and environment information.
  • the image processing method further includes:
  • Step 202 Determine a confidence level of the navigation state and environment information.
  • Step 203 Determine whether the confidence is lower than a threshold
  • Step 204 When the confidence level of the navigation state and the environment information is higher than the threshold, continue to provide obstacle avoidance information based on the navigation state and the environment information.
  • Step 205 When the confidence of the navigation state and the environment information is lower than the threshold, connect to the manual server to obtain the obstacle avoidance information sent by the manual server.
  • the image processing method and system are applied to the intelligent guide blind device.
  • the technical solution of the present application will be described below from the side of the smart guide device 10.
  • the smart guide device includes: a video acquisition module 32b, a picture acquisition module 32a, and a transmission module 35, from the perspective of completing a technical solution for reducing consumption of network resources.
  • the video acquisition module 32b acquires a video image around the terminal and uploads it by the sending module 35; when the smart guide device is in a stationary state, the image acquisition module 32a acquires a picture image around the terminal and Uploaded by the sending module 35.
  • the intelligent guide blind device further includes an adjustment module 33 and a detection module 31.
  • the adjustment module 33 includes a compression module 34.
  • the compression module 34 adjusts the acquired video image to a first image quality standard that is less than a quality standard of the image image.
  • the video image under the first image quality standard is used for obstacle recognition, and the detection module 31 determines the navigation state based on the identified obstacle.
  • the adjustment module 33 adjusts the acquired picture image to a second image quality standard. Images under the second image quality standard can be used for the identification of environmental information.
  • the adjustment module 33 adjusts the image in many ways, except that the compressed image reduces the amount of image data processing.
  • the adjustment module 33 may further include a single-channel grayscale image conversion module or an image binarization module or an extraction edge image module or a combination of the above modules to complete image adjustment and processing.
  • FIG. 5 an embodiment of the smart guiding device is shown.
  • a generating module is provided. Placed on the smart guide device.
  • the intelligent guide blind device includes a detection module 31, an image acquisition module 32, an adjustment module 33, a transmission module 35, a receiving module 36, a voice interaction module 37, and a generation module 39.
  • the image acquisition module 32 includes the video acquisition module 32b and a picture acquisition module 32a.
  • the detection module 31 monitors the user's travel status and receives the user status parameters.
  • the image acquisition module 32 is configured to determine a user travel state according to the state parameter.
  • the video acquisition module 32b acquires a video image around the user;
  • the image acquisition module 32a obtains Image image around the user.
  • the generating module 39 is configured to identify the navigation state and environment information of the user according to the adjusted video image and/or picture image, and provide obstacle avoidance information based on the navigation state and the environment information.
  • the generating module 39 is further configured to determine the confidence of the navigation state and the environment information.
  • the obstacle avoidance information includes an obstacle avoidance travel instruction such as an obstacle avoidance travel instruction such as stop/forward/left turn/right turn.
  • the voice interaction module 37 plays the obstacle avoidance information to the user in a voice manner, and recognizes the voice feedback of the user by voice to complete the voice interaction control.
  • the receiving module 36 receives data from the cloud server.
  • the detecting module 31 is configured to monitor a user traveling state and receive a user state parameter. There are many implementations of the detection module 31, which are explained below.
  • the detection module 31 is always in the detection working state after the system of the intelligent guide blind device is started, and determines and switches the image acquisition mode in time.
  • the detecting module 31 monitors sensor signals such as an inertial sensor and a GPS module, and the inertial sensor is a gyroscope.
  • the duration value is 5 seconds
  • the camera image is activated to acquire a video image; if the sensor signals such as the inertial sensor and the GPS module are monitored, it is determined that the user enters the stop mode from the sport mode and continues.
  • the duration value is 5 seconds
  • the camera is started to take a picture to obtain a picture image.
  • the user status is monitored by an image sensor.
  • the image is collected by an image sensor, and the difference between the frames is calculated based on an algorithm to estimate the motion state of the user:
  • the detection module 31 includes an optical flow motion analysis module for monitoring the state of travel of the user.
  • the speed of the mode movement in the image is also instantaneously changed, and the speed of the user is estimated.
  • a set threshold such as 1.5 km/hour, and lasts for several seconds.
  • the duration value is 5 seconds
  • the user is considered to be in a motion state
  • the guide application enters the path finding guide mode, and the system starts video capture to obtain a video image around the user; if the motion speed is less than a set threshold, and If the user is in a static state for a few seconds, the smart guide device is required to enter the environment recognition mode to recognize the object, and then the camera is activated to take a picture image.
  • the detection module 31 includes a visual navigation module for monitoring the user's travel status.
  • the detection is implemented by Visual Simultaneous Localization and Mapping (vSLAM), which is referred to as a visual navigation module in this embodiment.
  • vSLAM Visual Simultaneous Localization and Mapping
  • the user's motion speed and angular velocity are obtained. If the motion speed and angular velocity exceed a set threshold and last for several seconds, such as a duration value of 5 seconds, the user is considered to be in motion.
  • the blind application enters the path finding guide mode, and the system starts video capture to obtain the video image around the user; if the motion speed and the angular velocity are less than the set threshold and lasts for a few seconds, for example, the duration value is 5 seconds, the user is considered to be in a stationary state, and needs to be
  • the intelligent guide blind device enters the environmental recognition mode to recognize the object, the camera is activated to take a picture and obtain a picture image.
  • the detection module includes means for monitoring the state of travel of the user, and monitoring the state of the user by a combination of sensors.
  • the sensor is combined into one of an inertial sensor, an electronic compass sensor, a GPS module, or a combination thereof.
  • the following examples estimate the user's motion state through inertial sensors, electronic compass sensors, GPS modules, and combinations thereof.
  • the linear acceleration and angular velocity of the user are measured in real time by an inertial sensor, and the motion speed and angular velocity of the user are estimated.
  • the absolute position of the user is measured by the electronic compass sensor, and the change of the time can be used to indirectly estimate the user's rotation speed. If the user's motion speed and angular velocity exceed the set threshold for a few seconds, such as a duration value of 5 seconds, the user is considered to be in motion; if the motion speed and angular velocity are less than the set threshold and lasts for a few seconds, the user is considered to be at rest. .
  • the absolute position of the user is measured by the GPS module, and the movement speed and the steering speed of the user can be estimated indirectly by the change of the time. If the user's motion speed and angular velocity exceed the set threshold for a few seconds, such as a duration value of 5 seconds, the user is considered to be in motion; if the motion speed and angular velocity are less than the set threshold and lasts for a few seconds, the user is considered to be at rest. .
  • a more accurate movement speed and angular velocity of the user can be obtained. If the user's motion speed and angular velocity exceed the set threshold for a few seconds, such as a duration value of 5 seconds, the user is considered to be in motion; if the motion speed and angular velocity are less than the set threshold and lasts for a few seconds, The user is considered to be at a standstill.
  • a visual real-time positioning modeling and sensor combination is combined for more accurate detection of user status.
  • the detection module 31 includes a visual navigation module and sensor combination for accurately monitoring the user's travel status.
  • the image-based estimation result can be integrated with the estimation results of the inertial sensor, the electronic compass sensor, and the GPS module through the Kalman filter, and finally a more accurate user state is output.
  • the generating module is disposed on the cloud server 50.
  • the smart guide blind device includes a detection module 31, an image acquisition module 32, an adjustment module 33, a transmission module 35, a receiving module 36, and a voice interaction module 37.
  • the image acquisition module 32 includes the video acquisition module 32b and a picture acquisition module 32a.
  • the obstacle avoidance information is generated by the cloud server 50, wherein the obstacle avoidance information is to identify the navigation state and environment information of the user according to the video image and/or the picture image, and based on the navigation state and the environment information. generate.
  • the adjustment module 33 adjusts the file size of the video image and/or the picture image.
  • the adjustment module 33 also includes a compression module 34.
  • the compression module 34 compresses the adjusted video image and/or picture image and sends it to the cloud server 50.
  • the adjustment module 33 may adopt a single-channel grayscale image conversion module or an image binarization module or an extraction edge image module or a combination of the above modules when adjusting the file size of the video image and/or the image image.
  • the embodiment further provides an implementation method of the smart guide device.
  • the method on one side of the smart guide device includes the following steps:
  • Step 301 Monitor user travel status, and receive user status parameters.
  • Step 302 Determine a user traveling state according to the state parameter, acquire a video image around the user when the user is in a motion state, and acquire a picture image around the user when the user is in a static state;
  • Step 303 Adjust and send the video image and/or picture image.
  • Step 304 Receive obstacle avoidance information, wherein the obstacle avoidance information is to identify the navigation state and environment information of the user according to the video image and/or the picture image, and generate the navigation state and the environment information based on the navigation state and the environment information.
  • the receiving the obstacle avoidance information may be the obstacle avoidance information calculated by the local generating module 39, or may receive the obstacle avoidance information from the cloud server 50.
  • the method of adjusting the file size of the video image and/or the picture image includes single-channel grayscale image conversion or image binarization or extraction of edge images or a combination of the above methods.
  • the smart guide device After receiving the obstacle avoidance information, the smart guide device performs voice interaction with the user through the voice interaction module 37.
  • FIG. 8 illustrates the technical solution of the present application from the cloud server 50 side.
  • the cloud server 62 completes the generation of obstacle avoidance information as an example.
  • the cloud server 50 includes a receiving module 41, a generating module 42, a sending module 44, and a voice interaction module 45.
  • the generation module 42 includes an image recognition module 43.
  • the cloud server 62 does not include the generation module 42 and the image recognition module 43.
  • the receiving module 41 receives video images and/or picture images from the smart guide blind device 10.
  • the video image is a guide blind device that monitors a user's travel state to receive a user state parameter, and determines a user travel state according to the state parameter, and a video image around the user acquired when the user is in a motion state; the image image is a guide blind device at the user A picture image around the user acquired when it is at rest.
  • the file size of the video image and/or picture image is first adjusted to a reasonable size.
  • a single-channel grayscale image conversion module or an image binarization module or an edge image module or a combination of the above modules may be used.
  • the adjusted video image and/or picture image also needs to be compressed to further reduce the amount of data transmitted, and then sent to the cloud server 50.
  • the cloud server 50 After determining the obstacle avoidance information, the cloud server 50 sends the obstacle avoidance information back to the source smart guide blind device by the sending module 44.
  • the cloud server 50 also sets the voice interaction module 45.
  • the voice interaction module 45 is configured to play the audio message transmitted by the user and collect the voice response of the manual server 60 when the cloud server 50 is selected as the manual server 60. Then, the sending module 44 sends the signal to the source smart guide device 10 for real-time voice communication.
  • FIG. 8 is a flowchart of a method for implementing intelligent guide blind from the cloud server side.
  • the intelligent guiding method includes the following steps:
  • Step 401 Receive a video image, where the video image is a video image around the user acquired when the terminal is in a motion state, and the video image is used by the guiding device to monitor the user's traveling state to receive the user state parameter, and determine the user traveling state according to the state parameter. a video image around the user acquired when the user is in motion;
  • Step 402 Receive a picture image, where the picture image is a picture image around the user acquired when the terminal is in a stationary state;
  • Step 403 Generate real-time obstacle avoidance information, which is to identify a navigation state and environment information of the user according to the video image and/or the image image, and generate the navigation state and the environment information based on the navigation state and the environment information;
  • Step 404 Send the obstacle avoidance information.
  • the method further includes determining a confidence level of the navigation state and the environment information.
  • the connection to the manual server 60 is performed, and the obstacle avoidance information sent by the manual server 60 is acquired.
  • the image processing method and system and the intelligent guide blind device provided by the embodiments of the present application automatically select an appropriate image acquisition device to capture an image according to a state and an environment in which the user is located, and acquire an image around the user when the user is in a motion state. a video image; when the user is in a stationary state, a picture image around the user is acquired to reduce consumption of network resources.
  • the image processing method and system and the intelligent guide blind device provided by the embodiments of the present application generate real-time obstacle avoidance information according to the acquired image, and the real-time obstacle avoidance information includes the identified barrier-free route and the obstacle avoidance travel instruction, and provides the blind person with the obstacle More real-time, more convenient, and more accurate obstacle avoidance information and safe navigation guidance.
  • the detecting module of the present application can accurately and quickly detect the current traveling state of the user, and provide accurate navigation state and environment information for real-time blind contact interaction.
  • the technical solution performs reasonable size adjustment on the image before the image information is recognized, thereby reducing the system data processing burden.
  • the image is compressed before image data transmission to reduce the data transmission load.
  • the technical solution of the present application sets a confidence level for the navigation state and the environment information, so as to improve the accuracy of the blind obstacle avoidance decision.
  • FIG. 10 is a schematic diagram showing the hardware structure of an electronic device 600 according to an image processing method according to an embodiment of the present disclosure. As shown in FIG. 10, the electronic device 600 includes:
  • processors 610 One or more processors 610, memory 620, one or more graphics processing units (GPUs) 630, and communication components 650 are illustrated in FIG. 10 by a processor 610 and a graphics processor 630.
  • the memory 620 stores instructions executable by the at least one processor 610, the instructions being executed by the at least one processor to establish a data channel through the communication component 650 to enable the at least one processor to perform the image processing method.
  • the processor 610, the memory 620, the graphics processor 630, and the communication component 650 may be connected by a bus or other means, as exemplified by a bus connection in FIG.
  • the memory 620 is a non-volatile computer readable storage medium and can be used for storing a non-volatile software program, a non-volatile computer-executable program, and a module, such as a program instruction corresponding to the image processing method in the embodiment of the present application.
  • / Module for example, the detection module 31, the image recognition module 32, the adjustment module 33, the generation module 39, the generation module 45, the image recognition module 27, and the voice interaction module 45 shown in FIG. 6) shown in FIG.
  • the processor 610 executes various functional applications and data processing of the server by executing non-volatile software programs, instructions, and modules stored in the memory 620, that is, implementing the image processing method in the above method embodiments.
  • the memory 620 can include a storage program area and a storage data area, wherein the storage program area can store an operating system, an application required for at least one function; and the storage data area can store data created according to the use of the terminal, such as a smart guide device. Wait.
  • memory 620 can include high speed random access memory, and can also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device.
  • memory 620 can optionally include memory remotely located relative to processor 610, which can be connected to the robotic interactive electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
  • the one or more modules are stored in the memory 620, and when executed by the one or more processors 610, perform an image processing method in any of the above method embodiments, for example, performing the above described FIG. Method steps 101 to 102, performing the method steps 202 to 205 in FIG. 4 described above and performing the method steps 301 to 304 in FIG. 7 described above, implementing the detection module 31 and the image shown in FIG.
  • Embodiments of the present application provide a non-transitory computer readable storage medium storing computer-executable instructions that are executed by one or more processors, for example, to perform the above
  • the method steps 101 to 102 in FIG. 3 described above, the method steps 202 to 205 in FIG. 4 described above, and the method steps 301 to 304 in FIG. 7 described above are performed to implement the method shown in FIG.
  • the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, ie may be located A place, or it can be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Rehabilitation Therapy (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Epidemiology (AREA)
  • Pain & Pain Management (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Traffic Control Systems (AREA)
  • Navigation (AREA)

Abstract

一种图像处理方法，包括以下步骤：在终端处，当终端为运动状态时，获取终端周围的视频图像并上传；当终端为静止状态时，获取终端周围的图片图像并上传。

Description

一种图像处理方法、系统以及智能导盲装置 技术领域
本申请涉及智能导盲领域，具体涉及一种图像处理方法、系统和智能导盲装置。
背景技术
随着网络技术的发展，图像处理技术以及大数据处理能力的提升以及移动终端的普及，盲人辅助设备和导盲软件的研究获得越来越多的关注。
盲人是弱势群体，需要得到社会的帮助，帮助他们提高独立生活的能力，可以拥有更好的生活质量。因此辅助盲人自由出行和认知物品的产品是盲人辅助设备和导盲软件研究的一个趋势。
目前，常用的导盲辅助软件大多采用盲人手动拍照，再上传到服务器检测，然后向盲人返回识别结果的方式进行导航和认知环境。这种导盲方式需要盲人自己操作拍摄照片，因为盲人看不见环境状况无法及时准确获取环境图像，而且手动获取图像的方式比较繁琐，而且盲人操作极为不便。对于一般的要求不高的物体检测，基本可以满足需求，但在需要路径导盲等要求较高的场合，手动的图片检测方式无法满足导盲实时性的要求。
中国专利申请第201310711816.0号披露了一种基于图像识别的智能导盲杖，属于电子信息工程技术领域。所述的智能导盲杖由手型杖柄、智能杖体、滚轮杖尖和蓝牙耳机四部分组成，在导盲杖内嵌入人机交互子系统、路况识别子系统、导航子系统和滚轮制动系统。该智能杖体，位于导盲杖中间段，内部中空，嵌入路况识别子系统，所述的路况识别子系统内包含了微处理器；摄像头和LED照明装置与所述的微处理器连接；所述的路况识别子系统与滚轮杖尖的导航子系统相连；其采用图像识别处理的方法，有效识别盲道与公交信息，协助盲人安全独立的出行。
但是，现有的智能导盲杖通过图像识别盲道与公交信息，协助盲人安全独立的出行。不能辅助导航和更全面的帮助盲人了解周围环境。避障效果不够好，并且，导盲设备如果通过图像识别方式帮助盲人了解周围环境，处理的数据量很大，有延迟响应现象，在特定环境下存在安全隐患。
因此，现有技术的智能避障技术还有待于改进。
发明内容
本申请实施例提供一种图像处理方法、系统以及智能导盲装置，用以降低对网络资源的消耗，同时能够实现准确避障。
第一方面，本申请实施例提供了一种图像处理方法，包括以下步骤：在终端处，当终端为运动状态时，获取终端周围的视频图像并上传；当终端为静止状态时，获取终端周围的图片图像并上传。
第二方面，本申请实施例提供了一种图像处理系统，包括终端以及与终端无线通信的云端服务器，该终端包括视频获取模块、图片获取模块以及发送模块，
当终端为运动状态时，该视频获取模块获取终端周围的视频图像并由发送模块上传至云端服务器；当终端为静止状态时，该图片获取模块获取终端周围的图片图像并由发送模块上传至云端服务器。
第三方面，从智能导盲装置的应用阐述，本申请实施例提供了一种智能导盲装置，包括：视频获取模块、图片获取模块以及发送模块，
当智能导盲装置为运动状态时，该视频获取模块获取终端周围的视频图像并由发送模块上传；当智能导盲装置为静止状态时，该图片获取模块获取终端周围的图片图像并由发送模块上传。
第四方面，从云端服务器的角度阐述，本申请实施例提供了一种图像处理方法，包括以下步骤：
接收视频图像，该视频图像为当终端为运动状态时，获取的用户周围的视频图像；
接收图片图像，该图片图像为当终端为静止状态时，获取的用户周围的图片图像；
生成实时避障信息，该避障信息为根据该视频图像和/或图片图像，识别用户的导航状态和环境信息，并基于该导航状态和环境信息而生成；以及
发送该避障信息。
第五方面，本申请实施例还提供了一种电子设备，包括：
至少一个处理器；以及，
与该至少一个处理器通信连接的存储器以及通信组件；其中，
该存储器存储有可被该至少一个处理器执行的指令，该指令被该至少一个处理器执行时，通过通信组件建立数据通道，以使该至少一个处理器能够执行如上所述的方法。
第六方面，本申请实施例还提供了一种计算机程序产品，所述计算机程序产品包括存储在非易失性计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行如上所述的方法。
本申请的有益效果在于，本申请实施例提供的图像处理方法、系统和智能导盲装置，根据用户所处的状态和环境自动获取相适应的图像，当该用户为运动状态时，获取用户周围的视频图像；当该用户为静止状态时，获取用户周围的图片图像，用以降低对网络资源的消耗。并根据获取的图像，生成实时避障信息，该实时避障信息包括识别出的无障碍路线和避障行进指令，为盲人提供更实时、更方便、更准确的避障信息以及实现安全的导盲导航。
附图说明
一个或多个实施例通过与之对应的附图中的图片进行示例性说明，这些示例性说明并不构成对实施例的限定，附图中具有相同参考数字标号的元件表示为类似的元件，除非有特别申明，附图中的图不构成比例限制。
图1是本申请实施例提供的图像处理系统的系统架构图；
图2是本申请实施例提供的图像处理系统的另一系统架构图；
图3是本申请实施例提供的图像处理系统的主要流程图；
图4是本申请实施例提供的图像处理系统的流程图；
图5是本申请实施例提供的智能导盲装置一实施例的模块图；
图6是本申请实施例提供的智能导盲装置另一实施例的模块图；
图7是本申请实施例提供的与云端服务器配合的智能导盲装置处理流程图；
图8是本申请实施例提供的云端服务器的流程图；
图9是本申请实施例提供的云端服务器的模块图；以及
图10是本申请实施例提供的为实现图像处理方法的硬件框架图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅用以解释本申请，并不用于限定本申请。
本申请实施例提供的图像处理方法和系统、智能导盲装置以及云端服务器实时获取用户所处的状态和环境，根据状态和环境选择不同类型的图像进行图像处理，在降低对网络资源的消耗的同时，可根据所述图像识别出导航状态和环境信息并提供实时的用于导航导盲交互的避障信息，无需手动操作，提供更实时和准确的避障建议和导航反馈。
如图1所示，本实施例的智能导盲装置，作为智能导盲终端可以应用于导盲头盔10，可以应用于导盲眼镜20，也可以应用于导盲手杖，也可以应用于其他导盲可穿戴设备。
在主要硬件上，该智能导盲装置设置一个或者多个CPU，为了满足图像识别要求必要的时候增设GPU以完成数据分析和图像识别等功能。
在软件上，该智能导盲装置可以搭载android操作系统，或者iOS操作系统或者WindowsPhone等应用于移动端的操作系统。
该云端服务器51-53，以大数据处理方式接收智能导盲装置发送的图像信息。
在一实施例中，该智能导盲装置系统通过GPS模块或者北斗定位模块、陀螺仪以及摄像头等传感器采集用户行进状态和环境信息，监测用户所处的状态。在用户处于移动状态下，启动视频采集用户周围的视频图像。为了减少数据传输的负荷，对采集的视频图像进行降质处理，再经过编码和压缩传输到云端服务器50，云端服务器50运用障碍检测等算法进行图像识别计算出无障碍路线，识别该用户的导航状态和环境信息，并基于该导航状态和环境信息提供避障信息，并实时返回给用户的智能导盲装置。在用户处于静止状态时，则采集图片图像，为了减少数据传输的负荷，对采集的图片图像进行降质处理，再经过编码和压缩传输到云端服务器50，云端服务器50调用物体识别等算法进行物体识别和环境信息识别，基于识别的物体和环境信息提供避障信息，并将避障信息实时返回给用户的智能导盲装置。本实施例的智能导盲系统可应用于导盲头盔， 移动终端导盲应用或其它可穿戴导盲设备等产品。
实施例1
请参阅图1，本实施例提供图像处理系统。
该图像处理系统概括来讲包括多个终端以及与该终端无线通信的云端服务器50，该终端包括视频获取模块32b、图片获取模块32a以及发送模块35，当终端为运动状态时，该视频获取模块32b获取终端周围的视频图像并由发送模块35上传至云端服务器50；当终端为静止状态时，该图片获取模块32a获取终端周围的图片图像并由发送模块35上传至云端服务器50。
在具体实施时，该图像处理系统的终端为若干智能导盲装置10、20，其中，每一智能导盲装置10、20与云端服务器50无线通信。云端服务器50为大数据处理中心，可为服务器集群由若干服务器组网51-53而成。该智能导盲装置可为导盲头盔10，移动终端导盲应用或可穿戴导盲眼镜等产品。本实施例中以导盲头盔10为例加以说明。
请参考图5，所示为该图像处理系统的导盲装置的实施例，该智能导盲装置包括检测模块31、图像获取模块32、调整模块33、发送模块35、接收模块36、语音交互模块37以及生成模块39。该图像获取模块32包括该视频获取模块32b以及图片获取模块32a。
该检测模块31监测用户行进状态，接收用户状态参数。该图像获取模块32用于基于该状态参数，判断用户行进状态，当该用户为运动状态时，视频获取模块32b获取用户周围的视频图像；当该用户为静止状态时，图片获取模块32a获取用户周围的图片图像。该调整模块33调整该视频图像和/或图片图像的文件大小，在保证图像识别质量的同时，降低数据处理量，另一方面降低数据传输量，降低对网络资源的消耗。
该发送模块35用于将数据传送给云端服务器。
该调整模块33包括压缩模块34。该压缩模块34将获取的到的视频图像调整至第一图像质量标准，该第一图像质量标准小于该图片图像的质量标准。
在第一图像质量标准下的视频图像用于障碍识别，该检测模块31根据识别的障碍确定导航状态。
该调整模块33将获取的到的图片图像调整至第二图像质量标准。在第二图 像质量标准下的图像能够用于环境信息的识别。
该生成模块39用于根据经过调整的该视频图像和/或图片图像，识别该用户的导航状态和环境信息，并基于该导航状态和环境信息提供避障信息。
该生成模块39还用于确定该导航状态和环境信息的置信度。
该避障信息包括避障行进指令，诸如停止/前进/左转/右转等避障行进指令。该语音交互模块37将避障信息以语音方式播放给用户，并语音识别用户的语音反馈，以完成语音交互控制。
如果该导航状态和环境信息的置信度低于阈值时，则通过发送模块35将该视频图像和/或图片图像发送至人工服务端60，并通过接收模块36获取人工服务端60反馈发送的避障信息。
请参考图6，所示为该智能导盲系统的一个实施例，本实施例中，为了减少本地数据处理量和方便集中升级统一管理，该生成模块42设置在云端服务器50上。本实施例中该智能导盲系统中，该生成模块42还用于确定该导航状态和环境信息的置信度。
同样的，该避障信息包括避障行进指令，诸如停止/前进/左转/右转等避障行进指令。该云端服务器50将生产的避障行进指令，发送给数据来源的智能导盲装置10，智能导盲装置10的语音交互模块37将避障信息以语音方式播放给用户，并语音识别用户的语音反馈，以完成语音交互控制。本实施例中，该导航状态和环境信息的置信度低于阈值时，连接至人工服务端60，从人工服务端60获取人工发送的避障信息，再发送给对应的智能导盲装置10。
请参考图3，所示为该图像处理系统的流程图。该图像处理方法，包括以下步骤：
步骤101：在终端处，当终端为运动状态时，获取终端周围的视频图像并上传；
步骤102：当终端为静止状态时，获取终端周围的图片图像并上传。
在具体实施时，需要确定佩戴终端的用户状态，该终端可为智能导盲装置。因此还包括以下步骤：
监测用户行进状态，接收用户状态参数；
根据该状态参数，判断用户行进状态，当该用户为运动状态时，获取用户周围的视频图像；当该用户为静止状态时，获取用户周围的图片图像；
调整该视频图像和/或图片图像；
根据经过调整的该视频图像和/或图片图像，识别该用户的导航状态和环境信息，并基于该导航状态和环境信息提供避障信息。
请参考图4，为了确保避障信息的准确，每个识别的导航状态和环境信息均进行置信度判断，该图像处理方法还包括：
步骤202：确定该导航状态和环境信息的置信度；
步骤203：判断置信度是否低于阈值；
步骤204：该导航状态和环境信息的置信度高于阈值时，继续基于该导航状态和环境信息提供避障信息；
步骤205：该导航状态和环境信息的置信度低于阈值时，连接至人工服务端，获取人工服务端发送的避障信息。
实施例2
本实施例中，该图像处理方法和系统应用在智能导盲装置上。以下从智能导盲装置10一侧阐述本申请技术方案。
从完成降低对网络资源的消耗的技术方案角度进行阐述，该智能导盲装置包括：视频获取模块32b、图片获取模块32a以及发送模块35。
当智能导盲装置为运动状态时，该视频获取模块32b获取终端周围的视频图像并由发送模块35上传；当智能导盲装置为静止状态时，该图片获取模块32a获取终端周围的图片图像并由发送模块35上传。
该智能导盲装置还包括调整模块33和检测模块31。该调整模块33包括压缩模块34。该压缩模块34将获取的到的视频图像调整至第一图像质量标准，该第一图像质量标准小于该图片图像的质量标准。在第一图像质量标准下的视频图像用于障碍识别，该检测模块31根据识别的障碍确定导航状态。
该调整模块33将获取的到的图片图像调整至第二图像质量标准。在第二图像质量标准下的图像能够用于环境信息的识别。
该调整模块33调整图像的方式很多，除了压缩图像降低图像数据处理量。该调整模块33还可包括单通道灰度图像转换模块或者图像二值化模块或者提取边缘图像模块或者以上模块的组合，以完成图像的调整和处理。
请参考图5，所示为该智能导盲装置的一实施例，本实施例中，生成模块设 置在该智能导盲装置上。
该智能导盲装置包括检测模块31、图像获取模块32、调整模块33、发送模块35、接收模块36、语音交互模块37以及生成模块39。该图像获取模块32包括该视频获取模块32b以及图片获取模块32a。
该检测模块31监测用户行进状态，接收用户状态参数。
该图像获取模块32用于根据该状态参数，判断用户行进状态，当该用户为运动状态时，该视频获取模块32b获取用户周围的视频图像；当该用户为静止状态时，图片获取模块32a获取用户周围的图片图像。
该生成模块39用于根据经过调整的该视频图像和/或图片图像，识别该用户的导航状态和环境信息，并基于该导航状态和环境信息提供避障信息。
该生成模块39还用于确定该导航状态和环境信息的置信度。
该避障信息包括避障行进指令，诸如停止/前进/左转/右转等避障行进指令。该语音交互模块37将避障信息以语音方式播放给用户，并语音识别用户的语音反馈，以完成语音交互控制。
该接收模块36接收来自云端服务器的数据。
该检测模块31，用于监测用户行进状态，接收用户状态参数。该检测模块31的实施方式很多，以下逐一进行阐述。
该检测模块31的第一实施例中，该检测模块31在智能导盲装置的系统启动后一直处于侦测工作状态，以及时判断和切换图像获取方式。该检测模块31监测惯性传感器、GPS模块等传感器信号，该惯性传感器为陀螺仪。当用户处于运动状态并持续数秒，本实施例中，持续时间值为5秒，则启动摄像头摄像获取视频图像；若监测惯性传感器、GPS模块等传感器信号，判断用户由运动模式进入停止模式并持续数秒，本实施例中，持续时间值为5秒，则启动摄像头拍照获取图片图像。
在该检测模块31的第二实施例中，通过图像传感器来监测用户状态。比如通过图像传感器采集环境画面，基于算法分析计算帧间图像的差异，从而估算用户的运动状态:
作为运动图像分析的一实施方式，该检测模块31包括用于监测用户行进状态的光流运动分析模块。通过计算光流，亦即时变图像中模式运动速度，估计用户的运动速度，当运动速度超过设定阈值，如1.5公里/小时，并持续数秒， 本实施例中，持续时间值为5秒，则认为用户处于运动状态，导盲应用进入寻路导盲模式，系统启动视频采集获取用户周围的视频图像；如果运动速度小于设定的阈值，并持续数秒，则认为用户处于静止状态，需要智能导盲装置进入环境认知模式以识别物体，则启动摄像头拍照获取图片图像。
作为运动图像分析的另一实施方式，该检测模块31包括用于监测用户行进状态视觉导航模块。通过视觉实时定位建模(Visual Simultaneous Localization and Mapping，vSLAM)实现检测，本实施例中称为视觉导航模块。通过计算用户的帧间位移和姿态变化，进而得到用户的运动速度和角速度，若运动速度和角速度超过设定阈值，并持续数秒，比如持续时间值为5秒，则认为用户处于运动状态，导盲应用进入寻路导盲模式，系统启动视频采集获取用户周围的视频图像；如果运动速度和角速度小于设定阈值，并持续数秒，比如持续时间值为5秒，则认为用户处于静止状态，需要智能导盲装置进入环境认知模式以识别物体，则启动摄像头拍照获取图片图像。
在该检测模块31的第三实施例中，该检测模块包括用于监测用户行进状态的，通过传感器组合监测用户状态。
该传感器组合为惯性传感器、电子罗盘传感器、GPS模块之一或者其组合。
以下举例通过惯性传感器、电子罗盘传感器、GPS模块以及他们的组合估计用户的运动状态。比如通过惯性传感器实时测量用户的线性加速度和角速度，估计的到用户的运动速度和角速度。同时通过电子罗盘传感器测量用户的绝对方位，结合时间的变化，可间接估计用户转动速度。若用户运动速度和角速度超过设定阈值，并持续数秒，比如持续时间值为5秒，则认为用户处于运动状态；如果运动速度和角速度小于设定阈值，并持续数秒，则认为用户处于静止状态。
又如通过GPS模块测量用户的绝对位置，结合时间的变化，可间接估计用户的运动速度和转向速度。若用户运动速度和角速度超过设定阈值，并持续数秒，比如持续时间值为5秒，则认为用户处于运动状态；如果运动速度和角速度小于设定阈值，并持续数秒，则认为用户处于静止状态。
再如将上述三种传感器结合，可以得到用户更为准确的运动速度和角速度。若用户运动速度和角速度超过设定阈值，并持续数秒，比如持续时间值为5秒，则认为用户处于运动状态；如果运动速度和角速度小于设定阈值，并持续数秒， 则认为用户处于静止状态。
在该检测模块31的第四实施例中，为了更准确检测用户状态组合了视觉实时定位建模与传感器组合。该检测模块31包括用于精确监测用户行进状态的视觉导航模块和传感器组合。为了得到更为准确的运动速度和角速度，可通过卡尔曼滤波器将基于图像的估计结果和惯性传感器、电子罗盘传感器、GPS模块的估计结果进行融合，最终输出更准确的用户状态。
请参考图6，在智能导盲装置10的另一本实施例中，为了减少智能导盲装置本地数据处理量和方便集中升级统一管理，其生成模块设置在云端服务器50上。
在此实施例中，该智能导盲装置包括检测模块31、图像获取模块32、调整模块33、发送模块35、接收模块36以及语音交互模块37。该图像获取模块32包括该视频获取模块32b以及图片获取模块32a。
本实施例中，由云端服务器50生成该避障信息，其中，该避障信息为根据该视频图像和/或图片图像识别该用户的导航状态和环境信息，并基于该导航状态和环境信息而生成。
该调整模块33调整该视频图像和/或图片图像的文件尺寸。
该调整模块33还包括压缩模块34。该压缩模块34压缩该调整后的视频图像和/或图片图像，再发送至云端服务器50。该调整模块33在调整该视频图像和/或图片图像的文件尺寸时，可以采用单通道灰度图像转换模块或者图像二值化模块或者提取边缘图像模块或者以上模块的组合。
请参考图7，其中，本实施例还提供该智能导盲装置一侧的实现方法。该智能导盲装置一侧的方法包括以下步骤：
步骤301：监测用户行进状态，接收用户状态参数；
步骤302：根据该状态参数，判断用户行进状态，当该用户为运动状态时，获取用户周围的视频图像；当该用户为静止状态时，获取用户周围的图片图像；
步骤303：调整并发送该视频图像和/或图片图像；
步骤304：接收避障信息，其中，该避障信息为根据该视频图像和/或图片图像识别该用户的导航状态和环境信息，并基于该导航状态和环境信息而生成。
其中，接收避障信息，可以是接收本地生成模块39计算判断的避障信息，也可以是从该云端服务器50接收避障信息。
其中，调整该视频图像和/或图片图像的文件尺寸的方法包括单通道灰度图像转换或者图像二值化或者提取边缘图像或者以上方法的组合。
该智能导盲装置在接收避障信息后，通过语音交互模块37以语音方式与该用户完成交互控制。
实施例3
请参考图8，本实施例从云端服务器50一侧阐述本申请技术方案。
以下实施例中，以云端服务器62完成避障信息的生成为例加以阐述。该云端服务器50包括接收模块41、生成模块42、发送模块44以及语音交互模块45。该生成模块42包括图像识别模块43。
在由智能导盲装置10完成避障信息生成的实施例中，云端服务器62不包括生成模块42和图像识别模块43。
该接收模块41接收来自智能导盲装置10的视频图像和/或图片图像。该视频图像为导盲装置监测用户行进状态接收用户状态参数，根据该状态参数判断用户行进状态，当该用户为运动状态时获取的用户周围的视频图像；该图片图像为导盲装置在该用户为静止状态时获取的用户周围的图片图像。
为了减少智能导盲装置和云端服务器之间的数据传输，提高系统效率和相应速度，该视频图像和/或图片图像的文件尺寸先被调整至合理大小。在具体实现上，调整图像大小时可以采用单通道灰度图像转换模块或者图像二值化模块或者提取边缘图像模块或者以上模块的组合。
该调整后的视频图像和/或图片图像还需经过压缩进一步减少传输的数据量，再发送至云端服务器50。
该云端服务器50在确定该避障信息后，由发送模块44将该避障信息发送回来源智能导盲装置。
本实施例中，该云端服务器50也设置语音交互模块45。该语音交互模块45在云端服务器50选中成为人工服务端60时，用于播放用户传送的音频消息以及采集人工服务端60的语音回复。再由发送模块44发送给来源智能导盲装置10进行播放，以实现实时语音沟通。
请参考图8，所示为从云端服务器一侧实现智能导盲的方法流程图。该智能导盲方法，包括以下步骤：
步骤401：接收视频图像，该视频图像为当终端为运动状态时，获取的用户周围的视频图像，该视频图像为导盲装置监测用户行进状态接收用户状态参数，根据该状态参数判断用户行进状态，当该用户为运动状态时获取的用户周围的视频图像；
步骤402：接收图片图像，该图片图像为当终端为静止状态时，获取的用户周围的图片图像；
步骤403：生成实时避障信息，该避障信息为根据该视频图像和/或图片图像，识别用户的导航状态和环境信息，并基于该导航状态和环境信息而生成；以及
步骤404：发送该避障信息。
在识别该用户的导航状态和环境信息的过程中，还包括确定该导航状态和环境信息的置信度。
该导航状态和环境信息的置信度低于阈值时，连接至人工服务端60，获取人工服务端60发送的避障信息。
本申请实施例提供的图像处理方法和系统以及智能导盲装置，根据用户所处的状态和环境，自动选择相适应的图像获取装置来拍摄图像，当该用户为运动状态时，获取用户周围的视频图像；当该用户为静止状态时，获取用户周围的图片图像，用以降低对网络资源的消耗。并且，本申请实施例提供的图像处理方法和系统以及智能导盲装置根据获取的图像，生成实时避障信息，该实时避障信息包括识别出的无障碍路线和避障行进指令，为盲人提供更实时、更方便、更准确的避障信息以及实现安全的导盲导航。进一步地，本申请的检测模块可准确快速的检测用户当前行进状态，为实时导盲交互提供准确导航状态和环境信息。另外，本技术方案在识别图像信息之前，对图像进行合理的大小调整，减轻系统数据处理负担。在图像数据传输前压缩图像，以减少数据传输负荷。同时，本申请技术方案对该导航状态和环境信息设置置信度，以提高导盲避障决策的准确性。
实施例4
图10是本申请实施例提供的图像处理方法的电子设备600的硬件结构示意图，如图10所示，该电子设备600包括：
一个或多个处理器610、存储器620、一个或者多个图形处理器(GPU)630以及通信组件650，图10中以一个处理器610和一个图形处理器630为例。该存储器620存储有可被该至少一个处理器610执行的指令，该指令被该至少一个处理器执行时，通过通信组件650建立数据通道，以使该至少一个处理器能够执行该图像处理方法。
处理器610、存储器620、图形处理器630以及通信组件650可以通过总线或者其他方式连接，图10中以通过总线连接为例。
存储器620作为一种非易失性计算机可读存储介质，可用于存储非易失性软件程序、非易失性计算机可执行程序以及模块，如本申请实施例中的图像处理方法对应的程序指令/模块(例如，附图5所示的检测模块31、图像识别模块32、调整模块33、生成模块39，附图6所示的生成模块45、图像识别模块27以及语音交互模块45)。处理器610通过运行存储在存储器620中的非易失性软件程序、指令以及模块，从而执行服务器的各种功能应用以及数据处理，即实现上述方法实施例中的图像处理方法。
存储器620可以包括存储程序区和存储数据区，其中，存储程序区可存储操作系统、至少一个功能所需要的应用程序；存储数据区可存储根据终端，比如智能导盲装置的使用所创建的数据等。此外，存储器620可以包括高速随机存取存储器，还可以包括非易失性存储器，例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实施例中，存储器620可选包括相对于处理器610远程设置的存储器，这些远程存储器可以通过网络连接至机器人交互电子设备。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。
所述一个或者多个模块存储在所述存储器620中，当被所述一个或者多个处理器610执行时，执行上述任意方法实施例中的图像处理方法，例如，执行以上描述的图3中的方法步骤101至步骤102，执行以上描述的图4中的方法步骤202至步骤205以及执行以上描述的图7中的方法步骤301至步骤304，实现附图5所示的检测模块31、图像识别模块32、调整模块33、生成模块39，附图6所示的生成模块45、图像识别模块27以及语音交互模块45等的功能。
上述产品可执行本申请实施例所提供的方法，具备执行方法相应的功能模块和有益效果。未在本实施例中详尽描述的技术细节，可参见本申请实施例所 提供的方法。
本申请实施例提供了一种非易失性计算机可读存储介质，所述计算机可读存储介质存储有计算机可执行指令，该计算机可执行指令被一个或多个处理器执行，例如，执行以上描述的图3中的方法步骤101至步骤102，执行以上描述的图4中的方法步骤202至步骤205以及执行以上描述的图7中的方法步骤301至步骤304，实现附图5所示的检测模块31、图像识别模块32、调整模块33、生成模块39，附图6所示的生成模块45、图像识别模块27以及语音交互模块45等的功能。
以上所描述的装置实施例仅仅是示意性的，其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。
通过以上的实施方式的描述，本领域普通技术人员可以清楚地了解到各实施方式可借助软件加通用硬件平台的方式来实现，当然也可以通过硬件。本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程是可以通过计算机程序来指令相关的硬件来完成，所述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)或随机存储记忆体(Random Access Memory,RAM)等。
最后应说明的是：以上实施例仅用以说明本申请的技术方案，而非对其限制；在本申请的思路下，以上实施例或者不同实施例中的技术特征之间也可以进行组合，步骤可以以任意顺序实现，并存在如上所述的本申请的不同方面的许多其它变化，为了简明，它们没有在细节中提供；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。

Claims (21)

  1. 一种图像处理方法，其特征在于，包括以下步骤：在终端处，
    当终端为运动状态时，获取终端周围的视频图像并上传；当终端为静止状态时，获取终端周围的图片图像并上传。
  2. 根据权利要求1所述的方法，其特征在于，所述获取终端周围的视频图像并上传，包括：
    将获取的到的视频图像调整至第一图像质量标准，所述第一图像质量标准小于所述图片图像的质量标准。
  3. 根据权利要求2所述的方法，其特征在于，在第一图像质量标准下的视频图像能够用于障碍识别，以及根据识别的障碍确定导航状态。
  4. 根据权利要求1所述的方法，其特征在于，所述获取终端周围的图片图像并上传，包括：
    将获取的到的图片图像调整至第二图像质量标准；在第二图像质量标准下的图像能够用于环境信息的识别。
  5. 根据权利要求2所述的方法，其特征在于，所述将获取的到的视频图像调整至第一图像质量标准，包括：
    在确定当前环境符合设定的环境条件时，将获取的到的视频图像调整至第一图像质量标准，其中，所述设定的环境条件包括：处于非危险环境，在当前环境为路口时，确定当前环境为危险环境。
  6. 根据权利要求3或4所述的方法，其特征在于，还包括确定所述导航状态和环境信息的置信度。
  7. 根据权利要求6所述的方法，其特征在于，所述导航状态和环境信息的置信度低于阈值时，连接至人工服务端，获取人工服务端发送的避障信息。
  8. 一种图像处理系统，其特征在于，包括终端以及与终端无线通信的云端服务器，该终端包括视频获取模块、图片获取模块以及发送模块，
    当终端为运动状态时，所述视频获取模块获取终端周围的视频图像并由发送模块上传至云端服务器；当终端为静止状态时，所述图片获取模块获取终端周围的图片图像并由发送模块上传至云端服务器。
  9. 根据权利要求8所述的系统，其特征在于，所述终端包括调整模块，所述调整模块包括压缩模块，所述压缩模块用于将获取的到的视频图像调整至第 一图像质量标准，所述第一图像质量标准小于所述图片图像的质量标准。
  10. 根据权利要求9所述的系统，其特征在于，所述终端还包括检测模块，在第一图像质量标准下的视频图像用于障碍识别，所述检测模块根据识别的障碍确定导航状态。
  11. 根据权利要求8所述的系统，其特征在于，所述终端包括调整模块，所述调整模块用于将获取的到的图片图像调整至第二图像质量标准；在第二图像质量标准下的图像能够用于环境信息的识别。
  12. 根据权利要求9所述的系统，其特征在于，所述压缩模块还用于在确定当前环境符合设定的环境条件时，将获取的到的视频图像调整至第一图像质量标准，所述设定的环境条件包括：处于非危险环境，在当前环境为路口时则属于危险环境。
  13. 根据权利要求9或11所述的系统，其特征在于，还包括生成模块，所述生成模块用于确定所述导航状态和环境信息的置信度。
  14. 根据权利要求13所述的系统，其特征在于，还包括连接至云端服务器的人工服务端，所述导航状态和环境信息的置信度低于阈值时，通过所述人工服务端获取避障信息。
  15. 一种智能导盲装置，其特征在于，包括：视频获取模块、图片获取模块以及发送模块，
    当智能导盲装置为运动状态时，所述视频获取模块获取终端周围的视频图像并由发送模块上传；当智能导盲装置为静止状态时，所述图片获取模块获取终端周围的图片图像并由发送模块上传。
  16. 根据权利要求15所述的智能导盲装置，其特征在于，还包括调整模块和检测模块，所述调整模块包括压缩模块，所述压缩模块用于将获取的到的视频图像调整至第一图像质量标准，所述第一图像质量标准小于所述图片图像的质量标准，在第一图像质量标准下的视频图像用于障碍识别，所述检测模块根据识别的障碍确定导航状态；所述调整模块还用于将获取的到的图片图像调整至第二图像质量标准；在第二图像质量标准下的图像能够用于环境信息的识别，所述调整模块还包括单通道灰度图像转换模块或者图像二值化模块或者提取边缘图像模块或者以上模块的组合。
  17. 根据权利要求15所述的智能导盲装置，其特征在于，
    还包括语音交互模块，用于在接收避障信息后，通过语音与所述用户交互。
  18. 一种图像处理方法，其特征在于，包括以下步骤：
    接收视频图像，所述视频图像为当终端为运动状态时，获取的用户周围的视频图像；
    接收图片图像，所述图片图像为当终端为静止状态时，获取的用户周围的图片图像；
    生成实时避障信息，所述避障信息为根据所述视频图像和/或图片图像，识别用户的导航状态和环境信息，并基于所述导航状态和环境信息而生成；以及
    发送所述避障信息。
  19. 根据权利要求18所述的方法，其特征在于，所述识别所述用户的导航状态和环境信息，还包括确定所述导航状态和环境信息的置信度，所述避障信息包括避障行进指令，所述导航状态和环境信息的置信度低于阈值时，连接至人工服务端，获取人工服务端发送的避障信息。
  20. 一种电子设备，其中，包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器以及通信组件；其中，
    所述存储器存储有可被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行时，通过通信组件建立数据通道，以使所述至少一个处理器能够执行权利要求1-7任一项所述的方法。
  21. 一种计算机程序产品，其中，所述计算机程序产品包括存储在非易失性计算机可读存储介质上的计算机程序，所述计算机程序包括程序指令，当所述程序指令被计算机执行时，使所述计算机执行权利要求1-7任一项所述的方法。
PCT/CN2017/095676 2017-08-02 2017-08-02 一种图像处理方法、系统以及智能导盲装置 Ceased WO2019024010A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201780003265.9A CN108366899A (zh) 2017-08-02 2017-08-02 一种图像处理方法、系统以及智能导盲装置
PCT/CN2017/095676 WO2019024010A1 (zh) 2017-08-02 2017-08-02 一种图像处理方法、系统以及智能导盲装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/095676 WO2019024010A1 (zh) 2017-08-02 2017-08-02 一种图像处理方法、系统以及智能导盲装置

Publications (1)

Publication Number Publication Date
WO2019024010A1 true WO2019024010A1 (zh) 2019-02-07

Family

ID=63011240

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/095676 Ceased WO2019024010A1 (zh) 2017-08-02 2017-08-02 一种图像处理方法、系统以及智能导盲装置

Country Status (2)

Country Link
CN (1) CN108366899A (zh)
WO (1) WO2019024010A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110812142A (zh) * 2019-10-18 2020-02-21 湖南红太阳新能源科技有限公司 一种智能导盲系统及方法
CN111427343A (zh) * 2020-01-16 2020-07-17 黑龙江科技大学 一种智能导盲方法及智能轮椅

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201899668U (zh) * 2010-09-03 2011-07-20 浙江大学 一种智能手机导盲器
CN103312899A (zh) * 2013-06-20 2013-09-18 张家港保税区润桐电子技术研发有限公司 一种带有导盲功能的智能手机
CN103637900A (zh) * 2013-12-20 2014-03-19 北京航空航天大学 基于图像识别的智能导盲杖
CN106618981A (zh) * 2016-12-06 2017-05-10 天津瑞世通科技有限公司 基于物联网的盲人生活辅助系统及带有该系统的盲人拐杖
CN106859929A (zh) * 2017-01-25 2017-06-20 上海集成电路研发中心有限公司 一种基于双目视觉的多功能导盲仪

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5973618A (en) * 1996-09-25 1999-10-26 Ellis; Christ G. Intelligent walking stick
US7626612B2 (en) * 2006-06-30 2009-12-01 Motorola, Inc. Methods and devices for video correction of still camera motion
CN101986673A (zh) * 2010-09-03 2011-03-16 浙江大学 一种智能手机导盲器及导盲方法
CN102123194B (zh) * 2010-10-15 2013-12-18 张哲颖 利用增强实景技术优化移动导航和人机交互功能的方法
CN102293709B (zh) * 2011-06-10 2013-02-27 深圳典邦科技有限公司 一种智能导盲装置
CN202409427U (zh) * 2011-12-01 2012-09-05 大连海事大学 便携式智能电子导盲仪
CN105227810B (zh) * 2015-06-01 2018-10-23 西北大学 一种基于bibavr算法的自动聚焦头盔摄像机
KR20170040974A (ko) * 2015-10-06 2017-04-14 주식회사 신성테크 상체 보호용 재킷을 이용한 시각장애인 보호시스템
CN105591882B (zh) * 2015-12-10 2018-03-06 北京中科汇联科技股份有限公司 一种智能机器人与人混合客服的方法及系统
CN205622764U (zh) * 2016-01-06 2016-10-05 国网重庆市电力公司电力科学研究院 一种视频监控设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201899668U (zh) * 2010-09-03 2011-07-20 浙江大学 一种智能手机导盲器
CN103312899A (zh) * 2013-06-20 2013-09-18 张家港保税区润桐电子技术研发有限公司 一种带有导盲功能的智能手机
CN103637900A (zh) * 2013-12-20 2014-03-19 北京航空航天大学 基于图像识别的智能导盲杖
CN106618981A (zh) * 2016-12-06 2017-05-10 天津瑞世通科技有限公司 基于物联网的盲人生活辅助系统及带有该系统的盲人拐杖
CN106859929A (zh) * 2017-01-25 2017-06-20 上海集成电路研发中心有限公司 一种基于双目视觉的多功能导盲仪

Also Published As

Publication number Publication date
CN108366899A (zh) 2018-08-03

Similar Documents

Publication Publication Date Title
WO2016031105A1 (ja) 情報処理装置、情報処理方法、及びプログラム
CN204814723U (zh) 一种导盲系统
CN104205828B (zh) 用于自动3d图像创建的方法和系统
CN112020411B (zh) 用于向用户提供服务的移动机器人装置和方法
WO2018046015A1 (zh) 车辆报警方法、装置及终端
JP2017529521A (ja) 社会的且つ環境的認識を提供する装用可能イヤピース
JP2004299025A (ja) 移動ロボット制御装置、移動ロボット制御方法及び移動ロボット制御プログラム
WO2019179468A1 (zh) 智能设备的控制方法、装置、设备和存储介质
WO2018103023A1 (zh) 人机混合决策方法和装置
US10881937B2 (en) Image processing apparatus, analysis system, and method for processing images
WO2016013409A1 (ja) 制御装置、制御方法、プログラム、および制御システム
US20140193036A1 (en) Display device and method for adjusting observation distances thereof
CN110378246A (zh) 地面检测方法、装置、计算机可读存储介质及电子设备
WO2015154359A1 (zh) 拍摄的实现方法及装置
CN109199808B (zh) 基于计算机视觉的盲人智能手杖
WO2017166725A1 (zh) 拍摄控制方法、设备及系统
JP2021051716A (ja) 情報を出力するための方法及び装置
CN112188059A (zh) 可穿戴设备、智能引导方法及装置、引导系统
KR20200136398A (ko) 노광 제어 장치, 노광 제어 방법, 프로그램, 촬영 장치, 및 이동체
CN115526897A (zh) 特高压换流站消防机器人火焰定位方法及系统
WO2019024010A1 (zh) 一种图像处理方法、系统以及智能导盲装置
CN108140124A (zh) 提示信息确定方法、装置、电子设备和计算机程序产品
CN108700885B (zh) 一种飞行控制方法、遥控装置、遥控系统
JP2023133343A (ja) ロボット、方向制御方法及びプログラム
KR20210061157A (ko) 인공지능 기반의 실시간 반응형 휴대용 소형 시각 보조장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17919828

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 29/05/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17919828

Country of ref document: EP

Kind code of ref document: A1