You can back up do CD's or DVD's if you have a burner. They're cheap, but only good for maybe five years, so not good for archival for the ages.
The traditional approach is tape backup. This is still good, but kind of expensive. You have to buy a tape drive, and the tapes aren't cheap. They also eventually lose data, maybe in five to fifteen years.
You can save to a GMail account, or some other online storage space provider, as long as it's nothing too sensitive.
You can get another external hard drive, and copy to it from time to time.
The solution I mostly use is to copy important files between multiple computers. Just make sure you have plenty of disk space.