Depending on the total size of what you are backing up, you can also use a Flash Drive. But, they are still relatively expensive when compared to an external hard drive. For example, you can get a "decent", 1 Tb (Terabyte) external drive for about $50. But, the least expensive 128 gig Flash Drive I have recently seen is about $35 to $40 (sometimes you can find them on sale for less). Of course, a Flash Drive is extremely convenient. If you do decide to purchase an external drive, it would be best to go with Seagate drives. They "play nice" with Macs. Myself, I have used numerous Seagate drives over the years with my Macs, both internal and external and in fact I currently have two external drives with Seagate mechanisms.
Regarding backups, Cory states it correctly. For Time Machine, it does not back up OS/System-related files. So, if you had a real bad software-related catastrophe with your internal drive, you would first need to boot your machine via an "external" process, use Disk Utility to Erase, Format, and Partition the drive, install the OS you are using (plus any updates), and then use Migration Assistant to recover everything else from your most Time Machine backup.
Backing up with either SuperDuper! or Carbon Copy Cloner (CCC) makes an exact
bootable copy of your entire system. In the event of a real bad software-related catastrophe with your internal drive, you can boot your machine from such a backup/clone, use Disk Utility to Erase, Format, and Partition the drive, and then do a restore of the backup/clone made by either product to your internal hard drive. You are then back in business.
Myself, given that I got "hit" with some kind of virus somewhat recently (had to do with Bit Coin, which I have never used), along with something using quite a bit of memory, I actually did a combination of those two routes. That is, I had a very recent SuperDuper! backup, and I first saved some updated files from my "somewhat infected" system (Quicken data files, newer versions of various software, Outlook's Main Identity database. I confirmed none of those files were infected). Next, I booted from that SuperDuper! backup, used Disk Utility there to Erase, Format, and Partition the drive, installed Yosemite (OS 10.10) fresh, applied the OS 10.10.4 Combo Updater, and then used Migration Assistant to migrate all the non-Apple System files, folders, Applications, etc. from that backup to my internal hard drive (SSD in my case). I then booted the machine from the "new" system, and copied the files I had saved. It worked like a charm!
I am actually contemplating following that same route once OS X 10.11 (El Capitan) comes out soon. We'll see.
Finally, this article does a good job in explaining backup strategies with Macs. A good amount of it actually contains what Cory and I said above:
http://download.cnet.com/blog/download-blog/mac-backup-basics?tag=rb_content;main