One of my favorite little features of having Android 2.2 (Froyo) on my Dell Streak is that Gmail recognizes .apk files and lets you install them directly from the message.
I usually download non-Market apps on the Streak and then install them with a file manager, but if grab them on my computer, I email the files to myself so I don't have to mess with a cable to get them on the device.
Before Froyo, the Gmail app wasn't a good choice for this because trying to do anything with the attachment would often result in either a parsing error or a useless preview option. The stock Email app fared much better, as you can see from the screenshots (Email on Eclair) below, but it still required the use of a file manager.
Nothing is wrong with having to use a file manager to install non-Market apps, of course, but if an option exists that reduces the amount of taps and screens, why wouldn't you do it? Attached .apk files are still handled the same way in the Froyo Email app as the Eclair app, so Gmail (I'm using version 2.3 from the Market) is absolutely the way to go for installing apps quickly and easily.