ENGINEERING CAPSTONE PROJECT

This is the University of Melbourne Road to Endeavour video of the capstone project - 5G USE CASE DEMONSTRATION. --------------------------------------------------------------------------------------------------------------------------- Script: Wei Dai: 5G, is the fifth generation of mobile networks. Compared with 4G, it provides higher bandwidth, lower latency, and some new technologies such as network slicing, which definitely pushes more fields into the digital world. In collaboration with Ericsson, our group build a demo of a typical master-slave system in 5G use cases, a Remote-controlled robotic arm with haptic feedback and real-time video streaming. The related scenarios could be the Remote Surgery, completing specific tasks for industrial use, or entertainments. Imagine that when the patients need to have surgery, the doctors can perform it in another city because there is 5G. Yang Cao: Our demo system has two terminals. The control terminal has a glove and a video display pad. The remote terminal is composed of a robotic arm and a camera. People can wear the glove to control the robotic arm remotely while monitoring the movement on the display. When the clamp grips items, the sensor will be triggered and you can feel the force feedback. Weihan Su: 2020 is a challenging year, with the epidemic breaking in, Melbourne undergoing a stage 4 lockdown rapidly. Hardware-related projects can't be conducted in the lab anymore. I was also part of a team that was in a ling hotel quarantine on the other side of the world. To keep the project moving forward, our team had switched to a fully online collabration mode months earlier, we actively utilized the advantage of the Cloud to realize real-time documents and desktop sharing and editing and online meetings. We also took full advantage of workload breaking. Each team member tackling their own part at home, and we will combine each part and finalize the project in the final phase. Zongqi Wu: We intend to make this system more perfect, complete and become an interesting demo that everyone can easily operate. And We will show how the quality of the internet boosts the system by simulating the advantages of 5G and network slicing. And we will also establish the system on the real 5G network in Australia to execute some experiments.

This is the University of Melbourne Endeavour Presentation film of the capstone project - 5G USE CASE DEMONSTRATION. Script: Yang Cao (Ivy): To realize such a system, one of the key points is to capture arm movements. The motion tracking sensors are chosen to do the job, which can measure the gravitational acceleration and angular velocity. By placing two sensors on user’s hand and forearm, the measurement data can be collaboratively utilized to compute pitch, roll, and yaw of user’s each joint. That information could then be used to control the five motors on our robotic arm to simulate the corresponding motions. Roll and pitch of the wrist are taken care of by the two motors here. Other two motors are placed here for simulating the pitch and yaw of user’s elbow. We also have another motor in the middle which enhances its ability to follow user’s motion. But to fully simulate the actions of user’s hand, we also need the control of fingers. As you can see, our robotic arm has a clamp controlled by the motor here, which could just be used to simulate the thumb and the index finger. To obtain the information of fingers’ movements, a flex sensor is tied to the user’s fingers, whose resistance will change as it is bent. Zongqi Wu (Jeremy): In order to complete the system, we also need to deliver the haptic feedback to the user. A simple way to realize this is to use force sensor resistors and shaftless vibrators. Once the clamp touches the object, the resistance of the force sensor resistors attached on it will change with the force applied on it. The pressure signal therefore could be used by the vibrators attached on users’ fingers for generating different levels of vibrations which represents the haptic feedback. Weihan Su (Peter): Ever since the advent of modern society, the speed of technological development has always depended on the speed of information transmission. The 5th generation of mobile network is the latest area of exploration at this stage. With the development of 5G, network slicing allows Internet Service Providers to better target their limited resources to provide more appropriate network services to clients with different requirements, such as providing higher speeds and lower latency. Because of these advantages over the 4G network, tasks such as remote surgery and remote bomb disposal, which previously required the presence of experts on site, can now be performed using 5G to build high precision gesture-based remote-control system with haptic feedback and real-time High-Definition video streaming. Wei Dai (Davy): Good Afternoon, welcome to the endeavor exhibition. I’m the team leader David. Our topic is 5G use case demonstration: remote-controlled robotic arm with haptic feedback and real-time video steaming. We’re building such a system demo for the applications inspired by the advantage of 5G in collaboration with the industry partner Ericsson. Firstly, please let me invite one of our team members Peter who is unfortunately not able to be here to give a brief introduction of 5G. Thanks Peter. Let’s now focus on the design strategy of our demo. Thanks Ivy and Jeremy. Our system uses two raspberry pi’s as the processors of each terminal. Two pi’s as clients will firstly collect and compute all the sensor data locally and send them to a network server. The server will then pass the data to the destination terminals to control the actuators. Also, the real-time video will be captured by the camera and encoded by the raspberry pi connected to it so that the pi on the other terminal can decode and play the video on its screen. Next, I will be giving a brief demonstration. Now, we’ve already connected our two terminals to the 5G network and they're communicating with each other through a network server we rent in Sydney. I’m trying to place the toy block into the red box while looking at the screen. As the incomplete Australia 5G infrastructure is still not much different from 4G. What we do is to manually add extra delay to illustrate how the network quality influences the system. This could also be treated as a simulation of the 5G network slicing technology where better networks could be sliced for the use cases in need. With the bad network, the system is very difficult to be controlled and the video quality is not guaranteed. Our system is just a demo, it has its limitations. For example, the movement is not smooth due to the motor quality, and the feedback approach is far from realistic. We used it to successfully complete some tasks such as stacking toy blocks and drawing simple shapes. In real-world applications, the system could be designed to be expensive or cheap depending on the requirements and specifically modified to fit the use cases. If you’re interested in our project, please come to our table for any further discussion. Thank you for watching.