Following June 5th 2017 presentation, I thought updating my article and sharing it was great timing. File systems are key to systems operation, and few people have a clue about key details so here we go!
After running MacOS Sierra (High Sierra standard to come) for the past weeks and now year, I need to take some time and write about APFS, the new Apple File System.
APFS is not yet a full feature on Sierra but it the “new” filesystem format that Apple is planning to use on next operating systems; plans for next year [this year – 2017] for a potential user production environment in iOS and macOS…
But at the moment, APFS misses out on some key features in the world of storage, including that it’s block based and, it somewhat lacks full end-to-end integrity. But time will show us what is what, if HFS lasted 30+ years, I think we’re good for now with APFS!
Apple details the new APFS file system as such:
“Apple File System is a new, modern file system for iOS, macOS, tvOS, and watchOS. It is optimized for Flash/SSD storage and features strong encryption, copy-on-write metadata, space sharing, cloning for files and directories, snapshots, fast directory sizing, atomic safe-save primitives, and improved file system fundamentals.”
So currently, MacOS Sierra and previous versions of OSX use HFS+ and older system HFS file systems are more than 30 years ago… If by any chance you have no clue about what I’m talking about, take a little and read the following before going forward.
Here the main differences between APFS and HFS+ as detailed on Apple’s website:
|Features||Mac OS Extended (HFS+)||Apple File System (APFS)|
|Number of allocation blocks||232 (4 billion)||263 (9 quintillion)|
|Maximum file size||263 bytes||263 bytes|
|Time stamp granularity||1 second||1 nanosecond|
|File and directory clones||✔|
|Full disk encryption||✔||✔|
|Fast directory sizing||✔|
With all this information this in mind, Sierra does blocks many powerful features of APFS; hey it’s still in beta… So here are the main limitations currently on Sierra beta release of APFS:
- No APFS on a startup disk (slowly will come of age in the next release of MacOS High Sierra)
- No APFS on Fusion Drives
- No network sharing
- No encryption with FileVault
- All APFS volumes are case-sensitive
- No APFS volume with Time Machine
Once APFS is in production, most if not all of these limitations will be removed and available. Of course, Apple may change things here and there, but the idea is to make APFS the standard in the next evolution of MacOS High Sierra.
This change is somewhat simple to understand, as APFS will greatly improve writing efficiency… new macs now use, SSD’s.
Some added value comes from the fact that SSD’s need to erase data before writing anything new on the disk.
It sounds a bit weird, but that’s how it works. So if Apple can find a way to streamline the writing and removing of data on SSD’s using advance techniques and caching, which would make a great difference… but wait…? Apple is doing that with the help of APFS!
To support these new fast drives, APFS will increase indexing file numbers (Inodes) by using and supporting 64 bit inodes. This is compared to HFS+ that uses 32 bit inodes. This means the old file system (32 bit) supports around 4 billion nodes, but with the ever-demanding data needs, this is getting a bit tight for bigger disk.
For example, APFS support 9 quintillion files on a single volume; ish, now there’s a big gap!
Also, other advance features such as real-time cloning, snapshots, space sharing, crash protection and single and multiple key encryption are all natively part of APFS. As this file system will find itself on tvOS, IOS, macOS and more devices (Apple Car anyone?) these features are necessary.
But remember, most mac users may not need these sizes and key features, but when you start going into the development, research or graphic environments, you need these; keeping in consideration that Petabyte environments are not that far away for users…
The change to APFS also enables the addition of file attributes within the same environment. You won’t need to separate the files and the file attributes like in HFS. When you’re in a disk with a lot of data, you can go from milliseconds to nanoseconds, and that, makes a big difference in the grand scheme of things. Suddenly, key features such as massive searches and so on are more responsive and thus, make the whole system faster and more responsive.
APFS also supports advance encryption methods outside of filevault (AES-XTS and AES-OBC) based on hardware and not only software. So again, performance gains right there.
So all in all, many other features are already or will be available for new MacOS releases, but as Apple put it on the developer site:
“HFS+ and its predecessor HFS are more than 30 years old. These file systems were developed in an era of floppy disks and spinning hard drives, when file sizes were calculated in kilobytes or megabytes.”
Q&A details publicly given by Apple
Why did Apple develop APFS?
Apple File System provides strong encryption, ultra-low latency and limited memory overhead. It is optimized for Flash/SSD storage and can be used on everything from an Apple Watch to a Mac Pro.
Can RAID be used with Apple File System?
Yes. Apple File System does not directly implement software RAID; however APFS-formatted volumes can be combined with an Apple RAID volume to support Striping (RAID 0), Mirroring (RAID 1), and Concatenation (JBOD). APFS-formatted volumes can also be used with direct-attached hardware RAID solutions.
Does Apple File System support directory hard links?
Directory hard links are not supported by Apple File System. All directory hard links are converted to symbolic links or aliases when you convert from HFS+ to APFS volume formats on macOS.
Does Apple File System support ditto blocks?
Ditto blocks are primarily used to protect against corrupted sectors in large storage arrays with unreliable hard disk drives. Apple File System takes advantage of modern hardware with strong checksums and error correction in firmware, and does not depend on ditto blocks.
Does Apple File System support redundant metadata?
With modern Flash/SSD storage, writing two blocks of data to different locations does not guarantee that the blocks will be written to separate locations. The Flash translation layer typically groups writes together into the same NAND block. Therefore it affords no extra protection to write a second copy at the same time the first copy is written.
What has Apple done to ensure the reliability of my data?
To protect data from hardware errors, all Flash/SSD and hard disk drives used in Apple products use Error Correcting Code (ECC). ECC checks for transmission errors, and when necessary, corrects on the fly. Apple File System uses a unique copy-on-write scheme to protect against data loss that can occur during a crash or loss of power. And to further ensure data integrity, Apple File System uses the Fletcher’s checksum algorithm for metadata operations.
Does Apple File System use journaling?
Apple File System uses copy-on-write to avoid in-place changes to file data, which ensures that file system updates are crash protected without the write-twice overhead of journaling.
Does Apple File System support data deduplication?
No. With Apple File System individual extents can be encrypted, making it impossible to examine and deduplicate files and their content. Apple File System uses clone files to minimize data storage and data duplication.
Does Apple File System support TRIM operations?
Yes. TRIM operations are issued asynchronously from when files are deleted or free space is reclaimed, which ensures that these operations are performed only after metadata changes are persisted to stable storage.
Is APFS open source?
An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format specification when Apple File System is released for macOS in 2017.