MacOS – TimeMachine est lent!

Time Machine est le service de sauvegarde automatisé de MacOS depuis des années. Avec cette maturité, comment est-il possible que celui-ci soit si lent?

Et bien j’ai une solution pour vous. Celle-ci est simple et n’affecte pas négativement votre Mac!

Pour vous donner une idée, je viens de compléter une pleine sauvegarde de mon Mac (environ 30 GB) en 5 minutes. En temps normal, ceci aurait pris 4 heures…

Cette commande permet donc de faire une sauvegarde de plusieurs gigaoctets (GB) en quelques minutes; plutôt qu’en quelques heures!

Sans trop entrer dans les détails, Time Machine est configuré pour rouler en arrière-plan sur votre Mac pour ne pas impacter votre travail. Sur ce, quand vous voulez accélérer les choses, une simple commande dans votre terminal sera en mesure d’accélérer les choses pour faire une sauvegarde rapide; très rapide.

Il s’agit de donner une priorité supérieur à Time Machine. Pour ramener les choses telles qu’elles étaient, il s’agit de simplement redémarrer votre Mac et le tour est joué.

Pour entrer la commande, démarrer votre terminal trouvé dans /Applications/Utilities/ ou dans le répertoire Utilités dans vos applications si vous avez un Mac en Français ou dans tout autre langues.

sudo sysctl debug.lowpri_throttle_enabled=0

Cette commande indique à votre Mac de prendre les processus d’arrière plan et leurs donner une priorité comme les autres. Vous ne devriez pas remarquer de différence dans vos autres apps, mais croyez-moi, Time Machine va rouler comme un champion.

Pour remettre le tout à la normale faire la commande suivante:

sudo sysctl debug.lowpri_throttle_enabled=1

Et voilà tout est revenu à la normale!

Sinon, il est toujours possible de redémarrer votre Mac et le tour est joué!

When all fails: MacOS Time Machine

Don’t like macOS migration assistant? Want to do things manually and have full control? Well this section is technical, but may yield key results for people having significant problems with Time Machine backups and the dreaded ACLs!

But the most importantly…

  • Don’t change Time Machine ACLs!
  • Just strip the ACLs of files you copied from it!
  • So find, copy locally, and then change permissions.

UNIX Background

Added to UNIX file permissions such as user/group/everyone each having their own read/write/execute permissions, MacOS also uses ACLs (Access Control Lists). These ACLs allow much more granular file and folder permissions settings.

UNIX File ownership

File ownership provides a secure method for storing files. Every file in UNIX has these basic attributes.

  • Owner: controls actions the owner of the file can perform on the file.
  • Group: secures actions a user, who is a member of the group that a file belongs to, can perform on the file.
  • Other (world): what action all other users can perform on the file.

UNIX File Access Modes

The permissions of a file are the first line of defense in the security of a UNIX system.

  • Read: Grants the capability to read or view the contents of the file.
  • Write: Grants the capability to modify, or remove the content of the file.
  • Execute: User with execute permissions can run a file as a program.

UNIX Directory Access Modes

Directory access modes are listed and organized similarly as any other file.

  • Read: Access to a directory means that the user can read the contents. The user can look at the filenames inside the directory.
  • Write: Access means that the user can add or delete files to the contents of the directory.
  • Execute: Executing a directory doesn’t really make a lot of sense so think of this as traverse permission.

Note: more details available on www.tutorialspoint.com/unix/

Back to Time Machine

Once we understand the basic of UNIX file security, we can now appreciate the fact that by default, Time Machine adds the following ACL to all files:

group:everyone denyadd_file,delete,add_subdirectory,delete_child,writeattr,writeextattr,chown

This ACL simply means that all files and folders inside a Time Machine backup are locked for everyone (even the root user).

So if you restore your files manually from a Time Machine backup, all files and folders will keep those annoying Time Machine ACLs attached to them (good for security, but bad for portability).

Solving the problem

Once we understand the challenge, it is quite easy to remove the Time Machine ACLs. For all three options you need the Terminal which you will find in /Applications/Utilities.

Either way, the key is to access the Time Machine files directly from the Finder window, then copy the file or folder directly to the destination folder.

Option 1 – Swing the axe

If we know the files we want are in folder called “Documents”, let’s copy that folder on our local desktop. Once the copy is done, we can type the following into our Terminal window (make sure you follow the path of where you’re trying to run the command).

Note: If you don’t know the Terminal-way of specifying a path file or folder, simply drag and drop the file/folder you want onto the Terminal window and the Terminal will type the correct file/folder name for you.

chmod -R -N ~/Desktop/Documents/ Files

Option 2: Remove the first ACL entry

Same example as above. You have a folder called “Documents” on your Desktop. But in this case you have a few files with custom ACLs that you want to preserve. Type the following into the Terminal window:

chmod -R -a# 0 ~/Desktop/Documents/ Files

What makes the above solution “dangerous” is that it is not idempotent.

An idempotent operation is an operation that can be applied over and over without changing the result after it has been applied once. Kind of like multiplying a number by 1. You can keep doing it but the result is always the same.

Why does that matter? Well, let’s say that you have a file that already had an ACL before Time Machine prepended its own ACL entry.

If you run the above command twice then you will have removed both the Time Machine ACL as well as the ACL that you probably didn’t want to lose.

Plus the above solution is also not ideal for Time Machine files that are mixed in with other files. If any of these other (non-Time Machine) files have ACLs then the above command will remove those ACLs.

Option 3: Remove specific restrictions from an ACL

Aside from being able to specify which number entry of an ACL you want to remove you can also specify the specific restrictions you want to remove. So you could do this:

chmod -R -a "group:everyone deny add_file,delete,add_subdirectory,delete_child,writeattr,writeextattr,chown" ~

Note: “~” means “my home directory”, i.e. if your username is bob then “~” = “/Users/bob”)

The above command is also idempotent!

Finally

If we want to view the UNIX permissions as well as the ACLs of a particular file/folder you can pop open the Terminal and type

ls -led /path/to/file_or_folder

(again, just drag and drop the file/folder you want onto the Terminal window if you don’t know how to specify it the Terminal-way).

If you want to learn more about certain commands, you can start by typing them into the Terminal window and you’ll be on your way to becoming a deepgeek.

man ls
man chmod

(space bar to page forward, q to exit the man[ual] page)

 

Key sources:

http://galvanist.com/post/94558522941/strip-time-machine-acls

https://www.tutorialspoint.com/unix/unix-file-permission.htm

https://discussions.apple.com/

https://www.apple.com/

http://www.opengroup.org/unix

 

Macos – How to access a Time Machine drive from another mac?

1 – Attach the disk Time Machine or attach the Time Capsule to your main Wi-Fi/internet router.

2 – Once available to your current Mac, hold down Alt (Option on some keyboards) on your Mac and click the Time Machine menu bar icon.

3 – Select Browse “Other Backup Disks”.

4 – The Time Machine disk should appear in a list, and once selected it you’ll see the standard Finder-based Time Machine view.

5 – Once in the finder window, move back in time to retrieve older files and simply choose what you want

Note: If the backup is encrypted you’ll be prompted to enter the password before you can gain access.

Apple APFS (Apple File System)

Following June 5th 2017 presentation, I thought updating my article and sharing it was great timing. File systems are key to systems operation, and few people have a clue about key details so here we go!

After running MacOS Sierra (High Sierra standard to come) for the past weeks and now year, I need to take some time and write about APFS, the new Apple File System.

APFS is not yet a full feature on Sierra but it the “new” filesystem format that Apple is planning to use on next operating systems; plans for next year [this year – 2017] for a potential user production environment in iOS and macOS…

But at the moment, APFS misses out on some key features in the world of storage, including that it’s block based and, it somewhat lacks full end-to-end integrity. But time will show us what is what, if HFS lasted 30+ years, I think we’re good for now with APFS!

Apple details the new APFS file system as such:

“Apple File System is a new, modern file system for iOS, macOS, tvOS, and watchOS. It is optimized for Flash/SSD storage and features strong encryption, copy-on-write metadata, space sharing, cloning for files and directories, snapshots, fast directory sizing, atomic safe-save primitives, and improved file system fundamentals.”

So currently, MacOS Sierra and previous versions of OSX use HFS+ and older system HFS file systems are more than 30 years ago… If by any chance you have no clue about what I’m talking about, take a little and read the following before going forward.

Here the main differences between APFS and HFS+ as detailed on Apple’s website:

Features Mac OS Extended (HFS+) Apple File System (APFS)
Number of allocation blocks 232 (4 billion) 263 (9 quintillion)
File IDs 32-bit 64-bit
Maximum file size 263 bytes 263 bytes
Time stamp granularity 1 second 1 nanosecond
Copy-on-write
Crash protected Journaled
File and directory clones
Snapshots
Space sharing
Full disk encryption
Sparse files
Fast directory sizing

With all this information this in mind, Sierra does blocks many powerful features of APFS; hey it’s still in beta… So here are the main limitations currently on Sierra beta release of APFS:

  • No APFS on a startup disk (slowly will come of age in the next release of MacOS High Sierra)
  • No APFS on Fusion Drives
  • No network sharing
  • No encryption with FileVault
  • All APFS volumes are case-sensitive
  • No APFS volume with Time Machine

Once APFS is in production, most if not all of these limitations will be removed and available. Of course, Apple may change things here and there, but the idea is to make APFS the standard in the next evolution of MacOS High Sierra.

This change is somewhat simple to understand, as APFS will greatly improve writing efficiency… new macs now use, SSD’s.

Some added value comes from the fact that SSD’s need to erase data before writing anything new on the disk.

It sounds a bit weird, but that’s how it works. So if Apple can find a way to streamline the writing and removing of data on SSD’s using advance techniques and caching, which would make a great difference… but wait…? Apple is doing that with the help of APFS!

To support these new fast drives, APFS will increase indexing file numbers (Inodes) by using and supporting 64 bit inodes. This is compared to HFS+ that uses 32 bit inodes. This means the old file system (32 bit) supports around 4 billion nodes, but with the ever-demanding data needs, this is getting a bit tight for bigger disk.

For example, APFS support 9 quintillion files on a single volume; ish, now there’s a big gap!

Also, other advance features such as real-time cloning, snapshots, space sharing, crash protection and single and multiple key encryption are all natively part of APFS. As this file system will find itself on tvOS, IOS, macOS and more devices (Apple Car anyone?) these features are necessary.

But remember, most mac users may not need these sizes and key features, but when you start going into the development, research or graphic environments, you need these; keeping in consideration that Petabyte environments are not that far away for users…

The change to APFS also enables the addition of file attributes within the same environment. You won’t need to separate the files and the file attributes like in HFS. When you’re in a disk with a lot of data, you can go from milliseconds to nanoseconds, and that, makes a big difference in the grand scheme of things. Suddenly, key features such as massive searches and so on are more responsive and thus, make the whole system faster and more responsive.

APFS also supports advance encryption methods outside of filevault (AES-XTS and AES-OBC) based on hardware and not only software. So again, performance gains right there.

So all in all, many other features are already or will be available for new MacOS releases, but as Apple put it on the developer site:

“HFS+ and its predecessor HFS are more than 30 years old. These file systems were developed in an era of floppy disks and spinning hard drives, when file sizes were calculated in kilobytes or megabytes.”

 

Article Notes

Q&A details publicly given by Apple

Why did Apple develop APFS?

Apple File System provides strong encryption, ultra-low latency and limited memory overhead. It is optimized for Flash/SSD storage and can be used on everything from an Apple Watch to a Mac Pro.

Can RAID be used with Apple File System?

Yes. Apple File System does not directly implement software RAID; however APFS-formatted volumes can be combined with an Apple RAID volume to support Striping (RAID 0), Mirroring (RAID 1), and Concatenation (JBOD). APFS-formatted volumes can also be used with direct-attached hardware RAID solutions.

Does Apple File System support directory hard links?

Directory hard links are not supported by Apple File System. All directory hard links are converted to symbolic links or aliases when you convert from HFS+ to APFS volume formats on macOS.

Does Apple File System support ditto blocks?

Ditto blocks are primarily used to protect against corrupted sectors in large storage arrays with unreliable hard disk drives. Apple File System takes advantage of modern hardware with strong checksums and error correction in firmware, and does not depend on ditto blocks.

Does Apple File System support redundant metadata?

With modern Flash/SSD storage, writing two blocks of data to different locations does not guarantee that the blocks will be written to separate locations. The Flash translation layer typically groups writes together into the same NAND block. Therefore it affords no extra protection to write a second copy at the same time the first copy is written.

What has Apple done to ensure the reliability of my data?

To protect data from hardware errors, all Flash/SSD and hard disk drives used in Apple products use Error Correcting Code (ECC). ECC checks for transmission errors, and when necessary, corrects on the fly. Apple File System uses a unique copy-on-write scheme to protect against data loss that can occur during a crash or loss of power. And to further ensure data integrity, Apple File System uses the Fletcher’s checksum algorithm for metadata operations.

Does Apple File System use journaling?

Apple File System uses copy-on-write to avoid in-place changes to file data, which ensures that file system updates are crash protected without the write-twice overhead of journaling.

Does Apple File System support data deduplication?

No. With Apple File System individual extents can be encrypted, making it impossible to examine and deduplicate files and their content. Apple File System uses clone files to minimize data storage and data duplication.

Does Apple File System support TRIM operations?

Yes. TRIM operations are issued asynchronously from when files are deleted or free space is reclaimed, which ensures that these operations are performed only after metadata changes are persisted to stable storage.

Is APFS open source?

An open source implementation is not available at this time. Apple plans to document and publish the APFS volume format specification when Apple File System is released for macOS in 2017.