cunoFS joins the distributed cloud
Our article Is POSIX Outdated in the Cloud Era? spurred a bit of discussion in the developer community (especially in our native field of high-performance cloud computing), and these discussions have solidified our position: while POSIX within Linux is here to stay, we’re not the only ones who want to innovate beyond what the Linux …
Virtualizing Syscalls with Dynasaur: Extending Linux into the Future Read More »
FSx for Lustre is a distributed parallel filesystem that’s used as part of HPC (high-performance computing) workloads across a variety of complex fields (including life sciences, machine learning, meteorology, finance, and media). It is AWS’s managed version of the Lustre filesystem, and while it’s a popular high-performance storage option, it can quickly become unaffordable as …
cunoFS is a Faster and Cheaper Alternative to FSx for Lustre Read More »
Migrating on-premises scientific workloads to the cloud has many benefits in terms of cost and performance that are attractive to both academic and business organizations. Microsoft Azure is a popular cloud platform within the scientific research community as it offers a variety of features and tools to build and scale the infrastructure for your workflows, …
How to migrate your on-premises scientific workloads to Azure Read More »
High-performance computing, machine learning (ML) and AI, and scientific workloads must be built to be scalable, a requirement that is often difficult for traditional filesystem storage solutions to meet. Object storage provides a fast, networked, and highly scalable storage option for these use cases, but there are some inherent performance and compatibility problems that need …
Mountpoint for Amazon S3 connects AWS S3 buckets to Linux filesystems. While intended to bridge the gap between object storage and filesystems (and Kubernetes using a CSI driver), it has several downsides that cause problems for developers: it has limited POSIX compatibility, limited directory handling and file editing, and inconsistent file writing behavior. This article …
Most companies that use FFmpeg and store their data in object storage (such as S3, Azure, Google Cloud, or on-premises solutions like MinIO) stage their data to a filesystem, then run FFmpeg on it there, and finally upload the result back into object storage. This is because FFmpeg requires POSIX file access, and existing POSIX …
Affordable and Fast: S3 is Now the Best Place to Process your FFmpeg Data Read More »
When choosing the EFS (Elastic File System) configuration options for your enterprise cloud storage, the choices can be overwhelming. Practical information can be hard to find, so we’ve assembled all of the relevant facts in one place, along with additional information that will help you pick the right mix of price and performance for your …
Performance and cost are important considerations when designing and implementing machine learning (ML) systems at scale. The speed and efficiency with which you can train your ML models and extract the maximum value from your data will determine the overall success of your project. Data loading is one of the biggest performance factors in the …
Optimizing PyTorch Machine Learning Cost and Performance Using cunoFS Read More »
Copyright © 2023. All rights reserved.
