Mountpoint for Amazon S3 connects AWS S3 buckets to Linux filesystems. While intended to bridge the gap between object storage and filesystems (and Kubernetes using a CSI driver), it has several downsides that cause problems for developers: it has limited POSIX compatibility, limited directory handling and file editing, and inconsistent file writing behavior.
This article summarizes the alternatives to Mountpoint, including open-source solutions with better POSIX compatibility, and cunoFS, which provides the best possible POSIX compatibility and outperforms Mountpoint by up to 7x in Linux and serverless environments.
Mountpoint for Amazon S3 compatibility: it’s not very good
When choosing the storage component that you will use as the foundation for your HPC, ML, media, and other high-throughput workflows, you will want to address three main concerns: compatibility, performance, and reliability. Unfortunately, Mountpoint for Amazon S3 fails 2/3 of these requirements:
- Poor compatibility: Mountpoint does not support all POSIX operations (it’s actually incredibly limited), as shown in our FFmpeg benchmarks where it was unable to even complete common FFmpeg tasks.
- Adequate performance: Mountpoint performs similarly to other FUSE-based file system adapters and is usable for some production scenarios, but it’s far from the best.
- Poor reliability: Lack of support for common POSIX operations, as well as no guaranteed support for other object-storage solutions other than AWS S3, means that file operations and existing workflows that are expected to succeed may fail.
Mountpoint is more ”S3” than “mount”: its limitations mean you still have to make significant alterations to your existing code, or heavily tailor new solutions to its way of doing things — often to the point where you may as well use the S3 API itself rather than attempting to work with an inconsistent POSIX implementation.
While Mountpoint’s existence is justified for some specific AWS-centric use cases, when it comes to general HPC applications developers tend to seek alternatives that are more flexible and are compatible with other cloud and on-premises object-storage systems.
Mountpoint-S3 alternatives: s3fs, Goofys, s3ql, and cunoFS
s3fs-fuse, Goofys, s3ql, and other commercial offerings provide varying levels of POSIX support, as well as compatibility with other cloud and on-premises object storage providers. However, the performance of these FUSE-based solutions is poor, and, as shown in our FFmpeg benchmarks, some are still unable to complete a standard FFmpeg workload due to POSIX issues.
A diagram showing how s3fs, Goofys, s3ql, and other commercial products all solve at least one of the problems developers face when implementing object storage, but not all of them.
cunoFS is a Mountpoint for Amazon S3 alternative that outperforms s3fs, goofys, and other FUSE-based object storage filesystem layers, as well as offering better performance than S3’s own tools, and better cost and performance than AWS EFS, EBS, and FSx for Lustre.
A graph showing the performance advantages of cunoFS over Mountpoint for Amazon S3 using cp command to copy 5x 32GiB files to/from ramdisk (smaller time means faster).
cunoFS supports Kubernetes CSI and serverless
cunoFS has the best POSIX compatibility for object storage file system layers, with near-native compatibility. It addresses all of the compatibility and performance issues discussed in this article in on-premises, hosted, containerized, and serverless environments.
While Mountpoint and goofys provide CSI (Kubernetes container storage interface) drivers, the inherent downsides discussed for each still remain. cunoFS provides our own CSI driver for Kubernetes, so that you can use your AWS S3, Google Cloud, and Azure Cloud storage within your Kubernetes clusters.
Advantages of using cunoFS over Mountpoint for Amazon S3
cunoFS directly addresses the biggest problems developers, sysadmins, and DevOps engineers face when working with object storage using AWS Mountpoint.
Here are some of the compatibility problems we’ve seen when developers try to run their code using Mountpoint, all of which were solved by switching to cunoFS:
- Mountpoint never allows overwriting a file, so you need to delete the file first if it exists: While not a complete showstopper, this could result in your having to significantly rework and re-test any existing applications. Even the –allow-overwrite Mountpoint flag does not let you do something simple like appending to a file using the >> operator.
- Directories don’t exist in Mountpoint, so clearing the cache throws away any empty directories: Files in Mountpoint can have a path, but there are no standalone directories, so empty directories are discarded when data is written to S3. This can have implications depending on how your data is structured (or not structured).
- Mountpoint doesn’t support random access writes: This is a dealbreaker for many workloads, as many applications write data non-linearly. This includes FFmpeg, which requires random access writes to support its full range of file formats including common ones like mp4.
- Mountpoint doesn’t support rename: This means you cannot use the mv command to rename files/directories in a bucket. This also affects commonly used tools like rsync, which rename file uploads to their final name once the data transfer to a temporary file has completed. This creates a bad development experience, where Mountpoint’s limitations must be tiptoed around.
Additionally, while Mountpoint for AWS S3 is faster than other FUSE-based alternatives, it’s still up to 7x slower than cunoFS and still failed our recursive copy benchmark. cunoFS also keeps your files intact: unlike some alternatives, it does not scramble or split them, but keeps them fully accessible from your object storage.
cunoFS is a Mountpoint alternative for Kubernetes and Docker that also works for Windows and Mac
cunoFS was initially developed for high-throughput, POSIX-compatible file system access to object storage for high-performance workloads in the genomics industry. We have since expanded on this to make it a full Mountpoint alternative that can mount AWS S3, Google Cloud, and Azure Blob storage as a filesystem in almost any Linux environment.
cunoFS can also be run locally or in the cloud in MacOS and Windows operating systems by using Docker and WSL.
How to connect to S3 as a filesystem in minutes
Setting up cunoFS is easy: all you need is an AWS S3 bucket and a few minutes. Here’s how to get things set up:
- First, create an AWS S3 bucket (or use an existing bucket — cunoFS works with all of your data already stored in S3!) and an IAM user with read and write permissions for the bucket.
- Set up cunoFS according to the instructions for your environment or operating system.
Once cunoFS is installed and connected to an S3 bucket, enable cunoFS Direct Interception mode using the cuno command. This command loads cunoFS in your shell, letting you access files in your S3 buckets from your POSIX environment using S3 URI-style paths and perform commands that would otherwise fail on Mountpoint:
| mkdir s3://my-bucket/test_folder touch s3://my-bucket/test_folder/test_file.txt echo “foo” >> s3://my-bucket/test_folder/test_file.txt cd s3://my-bucket/test_folder/ mv test_file.txt test_file_2.txt |
With cunoFS enabled, you can run all of your existing applications and scripts as you usually would, supplying S3 URIs instead of filesystem paths. Unlike Mountpoint for Amazon S3, everything will ”just work,” allowing you to work directly with files in S3 as if they were stored locally.
If you wish to use cunoFS in scripts or automated environments, you can add it to your shell profile or enable automatic interception in containerized environments.
If you are deploying to Kubernetes and using CSI, you can integrate cunoFS by following our CSI documentation.
Download cunoFS for free evaluation and personal use, or contact us at sales@cuno.io if you have any enquiries or questions.
