In a previous blog post, we showed how “bringing the code to the data” can highly improve computation performance through the active storage (also known as computational storage) concept. In our journey in investigating how to best make computation and storage ecosystems interact, in this blog post we analyze a somehow opposite approach of “bringing the data close to the code“. What the two approaches have in common is the possibility to exploit data locality moving away in both cases from the complete disaggregation of computation and storage.
The approach in focus for this blog post, is at the basis of the Alluxio project, which in short is a memory speed distributed storage system. Alluxio enables data analytics workloads to access various storage systems and accelerate data-intensive applications. It manages data in-memory and optionally on secondary storage tiers, such as cheaper SSDs and HDDs, for additional capacity. It achieves high read and write throughput unifying data access to multiple underlying storage systems reducing data duplication among computation workloads. Alluxio lies between computation frameworks or jobs, such as Apache Spark, Apache MapReduce, or Apache Flink, and various kinds of storage systems, such as Amazon S3, OpenStack Swift, GlusterFS, HDFS or Ceph. Data is available locally for repeated accesses to all users of the compute cluster regardless of the compute engine used avoiding redundant copies of data to be present in memory and driving down capacity requirements and thereby costs.
For more details on the components, the architecture and other features please visit the Alluxio homepage. In the rest of the blog post we will present our experience in integrating Alluxio on our Ceph cluster and use a Spark application to demonstrate the obtained performance improvement (the reference analysis and testing we aimed to reproduce can be found here).
In most of the distributed storage systems, the data nodes are decoupled from compute nodes. Disaggregation of storage from the compute servers is motivated by an improved efficiency of storage utilization and a better and mutually independent scalability of computation and storage.
While the above consideration is indisputable, several situations exist where moving computation close to the data brings important benefits. In particular, whenever the stored data is to be processed for analytics purposes, all the data needs to be moved from the storage to the compute cluster (consuming network bandwidth). After some analytics on the data, in most cases the results need to go back to the storage. Another important observation is that large amounts of resources (CPU and memory) are available in the storage infrastructure which usually remain underutilized. Active storage is a research area that studies the effects of moving computation close to data and analyzes the fields of application where data locality actually introduces benefits. In short, active storage allows to run computation tasks where the data is, leveraging storage nodes’ underutilized resources, reducing data movement between storage and compute clusters.
There are many active storage frameworks in the research community. One example of active storage is is theOpenStack Storlets framework, developed by IBM and integrated within OpenStack Swift deployments. IOStack is European funded project, that builds around this concept for object storage. Another example is ZeroVM, which allows developers to push their application to their data instead of having to pull their data to their application.
In June we could participe to the 28th edition of EuCNC, an international conference sponsored by the IEEE Communications Society, the European Association for Signal Processing, and supported by the European Commission. EuCNC is one of the most prominent communications and networking conferences in Europe, which efficiently brings together cutting-edge research and world-renown industries and businesses.
The third robotics and ROS in Zürich meetup was organized and hosted by ICCLab on June 19th 2019. There was a good turnout from representatives in both academia and industry, totaling about 25 people in attendance. For our third meetup we had two presentations: “Perception and action planning in complex environments with ROS” by Rastislav Marko and Martin Möller from F&P Personal Robotics and “Self-calibrating camera position and grasping with Niryo arm” by Dimitrios Dimopoulos from ICCLab, ZHAW.
Summary of presentation #1: Perception and action planning in complex environments with ROS by Marko Rastislav and Martin Möller from F&P Personal Robotics
Martin Möller opened the first talk with a brief introduction of the company and one of its collaborations with ZHAW in the past. Next an overview of the hardware components of P-Rob 2R was given. This robot lies at the heart of the company’s solutions, including Lio, a mobile service robot, which was the focus of the talk. Following that, myP, the core robot library with its accompanying graphical control interface, was presented in action and a quick look at its architecture and configuration was showcased. Finally, Martin mentioned the key sensor components used by the mobile service robot.
The second robotics and ROS meetup in Zürich was organized by ICCLab and hosted by Dr. Romana Rust and Gonzalo Casas from Gramazio Kohler Research, ETH Zürich, on May 14th 2019. There was a good turnout from representatives in both academia and industry, totaling about 45 people in attendance. For this second meetup we had three presentations: “ROS for Digital Fabrication in Architecture”, “ROS Integration into Magic Leap” and “Next Generation Security” from Wecorp.
Summary of presentation #1: ROS for Digital Fabrication in Architecture by Dr. Romana Rust and Gonzalo Casas from ETH Gramazio Kohler Research group
Dr. Romana Rust opened the first talk by showcasing ongoing and past projects of the Gramazio Kohler Research group. Specifically she presented the usage of industrial grade robots and ROS in additive digital fabrication and the ways they allow for a novel approach in building non-standardized architectural components.
In most development processes hiccups are unavoidable. Our grasping application using the Niryo One arm was no exception. During testing, we had two of our arms break down and with this post, we would like to share our experiences with debugging and resolving these issues.
As far as we can understand, the axis 6 motor (Dynamixel XL-320 model) in the first arm, which is responsible for turning the gripper around, was damaged due to the gripper hitting the table. Since the gripper does not have an applied force feedback shutdown procedure, one of the motors probably broke down from overloading. Note that there is no gripper URDF model provided and octomap integration into the project was not yet complete at the time, so the kinematics planner was not aware of the table’s existence. As for our second arm, the culprit was the power adapter. The Dynamixel XL-430 motors are rated for 11.1 Volts, but the adapter supplied is a 12V one, which can cause permanent damage due to overheating if the arm is operating for prolonged periods of time. This design oversight was amended in Niryo One models shipped after November 2018, but in any case, you should check the rating of the power adapter provided and request a replacement if needed.
ICCLab organized the first robotics and ROS meetup in Zürich on April 9th 2019. There was a good turnout from representatives in both academia and industry, totaling almost 60 people in attendance. This meetup is the first of hopefully many that we intend to organize, as part of our effort to build a local network of ROS users across many robotic disciplines. Besides networking, our goal for these meetups is to also provide a platform for individuals to share and teach specific robotics/ROS knowledge. For this initial meetup we had two presentations: vision for navigation in autonomous robotics, and ROS applications at ICCLab.
As we are making progress on the development of robotic applications in our lab, we experience benefits from providing an easy-to-deploy common ROS Kinetic environment for our developers so that there is no initial setup time needed before starting working on the real code. At the same time, any interested users that would like to test and navigate our code implementations could do this with a few commands. One git clone command is now enough to download our up-to-date repository to your local computer and run our ROS kinetic environment including a workspace with the current ROS projects.
To reach this goal we created a container that includes the ROS Kinetic distribution, all needed dependencies and software packages needed for our projects. No additional installation or configuration steps are needed before testing our applications. The git repository of reference can be found at this link: https://github.com/icclab/rosdocked-irlab