cunoFS is the culmination of almost 100 developer-years of work that started in 2014, by a team of software engineers who have deep expertise in parallelization and threading, networking, and user-mode virtualization.

cunoFS is a product of PetaGene, a software company that was founded in 2016 and whose award-winning transparent compression and encryption products have been widely adopted by Pharma and other Heathcare and Life Sciences customers. PetaGene extended its transparent user-mode virtualisation read-back technology in 2018 to give high-throughput virtual file access to data in S3 and other flavours of object storage. The raw transfer speeds that PetaGene achieved at the time were 3x those of AWS CLI S3 commands, and when combined with PetaGene’s 10x lossless genomic compression, gave a throughput speed-up of up to 30x. This access method proved to be very popular, with customers and prospective customers using it to access all of their data stored in object storage, not just their compressed or genomic data.

So PetaGene decided to spin out cunoFS as a separate product, unrelated to genomic compression, but first made it faster and capable of high throughput for all kinds of workloads. PetaGene also made cunoFS fully POSIX-compatible, but without the need for any modifications to the objects in the object storage. cunoFS’s raw transfer speed is now 30x that of AWS CLI S3 for a single node, and has been benchmarked at 11 Tbps for 256 nodes. Many novel optimisations mean that cunoFS achieves high throughput and scalability for both large-file and small-file workloads, even when workflows involve random shuffling of data. cunoFS is thus a highly-convenient and intuitive tool for POSIX access to object storage, with many optimizations that provide high throughput in a wide variety of workflows and applications.