There's no hard-n-fast definition of long distance hiking, so here's my personal one:
  covering over 200 miles and taking over 2 weeks
 
The National Park Service definition of a National Scenic Trail is: 100 miles or longer, continuous, primarily non-motorized routes of outstanding recreation opportunity.

Some hikers would consider 100 miles just a weekend jaunt
    - the Pacific Crest Trail is over 2650 miles.
Some hikers would consider 15 days just a warm-up outing
    - the Appalachian Trail takes 100 to 200 days.
A 200+ mile, 2+ week trek means you won't carry all your food from the start and you will wear your body down, not just surviving on stored-up energy.

So, when you decide to go the distance, how does that change from day hikes? Where do you go? What gear changes? What else is needed? Good questions!
 

Where Can I Hike Long Distances?

Congress passed the National Trails System Act in 1968, establishing a framework for a nationwide system of scenic, recreational, and historic trails.
You may be surprised at just how many thousands of miles of trails there are across the country, and not too far from your home.
Here are the (as of 2011) 30 National Scenic Trails and National Historic Trails:
 

Trail Name	Year Est'd	Authorized Length (miles)	Adm. Agency
Appalachian NST	1968	2,158	NPS
Pacific Crest NST	1968	2,638	USDA-FS
Continental Divide NST	1978	3,100	USDA-FS
Oregon NHT	1978	2,170	NPS
Mormon Pioneer NHT	1978	1,300	NPS
Lewis and Clark NHT	1978	3,700	NPS
Iditarod NHT	1978	2,350	BLM
North Country NST	1980	3,200	NPS
Overmountain Victory NHT	1980	275	NPS
Ice Age NST	1980	1,000	NPS
Florida NST	1983	1,300	USDA-FS
Potomac Heritage NST	1983	700	NPS
Natchez Trace NST	1983	95	NPS
Nez Perce (Nee-me-poo) NHT	1986	1,170	USDA-FS
Santa Fe NHT	1987	1,203	NPS
Trail of Tears NHT	1987	5,045 (1)	NPS
Juan Bautista de Anza NHT	1990	1,200	NPS
California NHT	1992	5,665	NPS
Pony Express NHT	1992	1,966	NPS
Selma to Montgomery NHT	1996	54	NPS
El Camino Real de Tierra Adentro NHT	2000	404	NPS & BLM
Ala Kahakai NHT	2000	175	NPS
Old Spanish NHT	2002	2,700	NPS & BLM
El Camino Real de los Tejas NHT	2004	2,580	NPS
Captain John Smith Chesapeake NHT	2006	3,000	NPS
Star-Spangled Banner NHT	2008		NPS
Arizona NST	2009	761	USDA-FS
New England NST	2009	190	NPS
Washington-Rochambeau Revolutionary Route NHT	2009	2,020	NPS
Pacific Northwest NST	2009	1,200	USDA-FS
(1) Includes both overland and water routes between Tennessee and Oklahoma.

 
You don't have to do long distance hiking on these designated trails, though.
Make your own trek route. Get maps of a wild area you'd like to explore and map out your trip on the trails that exist.
Or, figure out how to hike from Albuquerque to Zuni in New Mexico or from Ada to Zumbrota in Minnesota. Google maps is a good tool to help with working on walking routes but it mostly follows roadways.

What Do I Need?

The equipment needed for a long distance hike is not much different than a week-long trek. You'll just go through more consumable items - food, fuel, maps, sunscreen, bug spray, ..., and all that. Since you won't want to carry lots of it, that means you'll need to arrange resupply points along your hike. See more about supplying food.

Long distance hiking also requires minimizing weight of equipment as much as possible since you'll carry it every step of the way. A 4pound tent for the weekend might be ok to carry for 10 miles, but a 1.5pound tarptent being carried every day for 6 months is a lot more comfortable.
Your equipment will wear out too. A pair of trail shoes might last 1000 or 2000 miles before the sole wears completely away or the seams fray out, but at some point every item will break. Duct tape, needle and thread, and improvization can get you by for awhile, but having a back-up plan for replacement items from the trail is important.

How Much Will It Cost?

Long distance hiking is expensive - at least in my world where I'm a real spendthrift.

• If a $100 pair of trail shoes lasts 1000 miles, that's 10 cents/mile.
• If you spend $500 on other gear that wears out over a 2000 mile trek, that's 25 cents/mile.
• You might be able to get 1000 calories of food for $2.50, but it will probably be higher than that. Hiking 10 miles/day and consuming 3500 calories will cost at least 87 cents/mile.
• Then, there's transportation to get TO and FROM the trailhead, meals in trail towns, maybe an occasional hotel stay or shower, and all the little items that add up.

So, long distance hiking costs at least $1.50/mile - a more realistic figure would be $2.50 per mile.
I spent $2350 for my 2012 800-mile Arizona Trail trip. This was almost $3.00/mile because the plane fare and celebrating in towns. A longer trail spreads the plane expense over more miles so the $$/mile drops compared to shorter trails. More experienced long-distance hikers stay out of towns as much as possible since they suck up all your $$$.
I spent under $2000 for the 1,100-mile Ice Age Trail in 2013 - I had no plane ride and already had most gear I needed. I stayed in no hotels.

Why Will I Fail?

Like all legends, long distance success rates are not precise so we don't know exactly how many people attempt and fail to thru-hike a long trail each year.
There are some general estimated annual success rates on the more popular trails:

• Appalachian Trail - about 750 of 3000 finish (~25%) - see stats
• Pacific Crest Trail - about 180 of 300 finish (~60%) - see stats
• Continental Divide Trail - about 15 of 30 finish (~50%) - see stats

 
Most folks tackle the AT hike as their first long distance attempt. Many have no idea what they're getting into and drop out within a week.
The PCT and CDT thru-hikers know the game and still half of them don't make it.

What causes most people to leave a thru-hike incomplete?

• Unrealistic Expectations - a thru-hike is a romantic notion, being at one with the wild for the summer. In reality, it is hard, dirty work day after day.
  People get homesick and realize that humans really are social creatures.
  Some hikers get bored with the monotony of sleep, hike, eat, hike over and over.
• Injury - one mistep and your trek is over. It is crucial to take good care of feet and legs with sturdy shoes, socks, and support as needed. Simple cuts can get infected. Eating bad food or not treating water can make you sick enough to leave the trail.
• Weather - many long trails have a short start and end window based on the spring thaw and winter snow. After a winter of extreme snow, a trail might not be passable. At any time along the hike, severe storms can ruin equipment, cause hypothermia, or make you so miserable that it's not worth trudging through yet another day of mud.
• No More $$$ - many hikers start a thru-hike because they are at a point in life with flexibility and few commitments. This often also means not enough funding to cover the expenses. If you can't buy food, you can't hike.