Portable Ubuntu USB Stick with Persistent Storage
UNetBootin allows creation of a such a stick, but it only takes care of creating stuff that's needed for legacy BIOS boot[1]. If you are using UEFI (Macs don't do legacy boot), things won't work as expected. To get it working, add persistent to the "Try Ubuntu without…