I come from WHT. What I don't like folks there, is they usually just say go for dedicated!! dedicated FTW!! I don't understand why such a professional forum act like that when a topic like VPS vs. Dedicated comes up. Well, we all know that dedicated is much better "ultimately" but definitely not on a cheap box.
The only advantage of low-end dedicated is you are the only one on it using it and a bit more secure, bandwidth & space. Other than that, it's all VPS. To the end, if you are running somethings serious, VPS and low-end dedicated may not be your choice either.
Clearly VPS advantages:
1. Go up to level 2/3 support for free as many as you want while cheap dedicated is unmanaged or only level1 support.
2. You get the top notch hardware from brand name vendors like Dell/HP while cheap dedicated might/should be on an used server which is a lot older than my desktop, and from an unknown vendor. Aged hardware tends to get more unstable things and HD failures.
3. RAID protection for an extra benefit(anyway, offsite backup is still required).
4. Premium bandwidth(reliable, fast, good uptime) vs. value/slow/outage network
5. When somethings go wrong, money back guarantee(or least say, extra credits) is more likely to real happen on a good VPS provider than a cheap dedicated box provider
6. Resources upgrade/downgrade on VPS is much more easier
7. There might be some hidden fees on low-end dedicated providers
8. OS re-image, backup/restore, hard reboot is very easy via VPS panel but normally you won't get these stuffs for free on cheap dedicated.
9. There are many decent VPS providers but only a couple of good cheap dedicated providers there I know of. You get more choice.
The only time I will look for a cheap dedicated is I need plenty of space & bandwidth.