All this because of a bug with the ubiquity installer that mismaps (and it is next to impossible to correct) the location of the bootloader.<br />Reinstalling GRUB doesn't change anything.<br />Removal of the FAT32 EFI boot partition and installing grub to the main EXT4 partition where the system is installed to, also doesn't work.<br />Installing to a solo ext4 partition and manually installing grub doesn't work either (despite marking it as "boot" in gparted).<br /><br />However, debian and other distributions that don't use ubiquity don't have this problem. You are free to use any/all distributions for a NON-UEFI USB installation on older machines.