If you think about it, $1 million dollars is more than most people who are retired now ever earned in their entire career. My dad said he never earned a total of $600,000 and he was an engineer who always worked for the same firm and never had any period of unemployment in his life. And he retired early. You just need some sort of career where it happens that you get a pension after 20 to 30 years. Actually his pension came because his company was bought out and they paid people off to retire early. But there are careers like cop, military, etc. where it is actually practical to plan to retire early and you don't have to rely on a 1980s wave of buy-outs.
Of course, if you really think you need $50,000 a year to live on, then I would not expect you to ever be able to retire. This is way more than most people's entire after-tax income when they are working, isn't it?
You can travel in many countries on an income of $10,000 a year. Have you ever visited Rita Gelman's website or read her book? I will try to get a link. I think she was downsized as a housewife at 47, and because her own income (writer) was only around $10,000 to $15,000 a year, she took to the road, traveling first in Mexico, then Nicarauga, then many other countries. It beats hoping and praying that somehow you can find a passive investment that can generate enough income to live on. With these low interest rates, people who relied on passive investments like CDs, bonds, Treasury notes, etc. have been totally screwed. My passive investments generate all of around $100 or so a month, and some of that is dividends from risky investments like stocks.
My income is lower than Gelman's and I am retired pretty much involuntarily (I am also a writer) and I have been able to stay debt-free but I live in a cheap area. And yes, you are very right about getting your house paid off. If you have to pay rent or a house note, I don't know how you could possibly retire. You need to get rid of as many recurring expenses as you can. Hey. It's easier than getting a million dollars in one place, at least if you're not a criminal or a developer (tree killer) in my humble opinion!
On Edit: Here's Gelman's website to get you started but her book tells a lot more so be sure to ask at your library -- ours had it on the shelf:
http://www.ritagoldengelman.com/