In 1987, a few months after I finished graduate school, my wife and I bought our first house. Although the small, vinyl-sided starter-home was the nicest place either my wife or I had ever lived, she wanted a swimming pool. I wanted to pay off the mortgage. After reading a Dear Abby letter from a widow whose newly-retired, workaholic husband died just before they took their long-awaited, trip-of-a-lifetime to Italy, I agreed to the pool.

So we built a $20,000 pool in the backyard of our $76,915 house. In percentage terms, it was the worst financial investment of my life. At best, the pool added zero resale value to the house. It was also the centerpiece of millions of dollars worth of memories.

Every summer, we and three other couples would host a giant pool party in our tiny backyard. People would bring seven-layer dip and crock pots full of cocktail weenies. Kegs of beer would be consumed. The game Speedy Toss was invented, as my large friends competed to see who could throw my small friend furthest across the pool. I don’t remember who won, but Speedy always lost.

The nexus of that large group of huge people in my tiny backyard were college friends. Last week, the fourth member of our 1980s pool party group died. Tim Daniels, a gentle giant who inspired and loved more than he scared and screamed, passed away at the age of 58. He, Doug Furnas, David Douglas and David James, integral characters at those pool parties, all passed away in their 50s. Their souls join those of too many of my college friends, including four of the two-deep offensive linemen from the 1985 Tennessee Sugar Bowl team.

If my wife and I had invested the money instead of building that pool, I figure that $20,000 would be $200,000 today, enough to send each of my kids to some ridiculously overpriced college.

Most people are woefully underinvested for retirement – not because they spend too much money on experiences, but because they spend too much money on cars and too much time on SnapFace and other mind-numbing activities. Preparing for retirement doesn’t require the deprivation of all pre-retirement pleasure. The purpose of the middle third of life is not simply to lay the groundwork for the last third of it.

Warren Buffett once lamented the people who spend a lifetime doing things they don’t enjoy simply because it will look good on their resume and might someday help land their dream job. “Isn’t that a lot like saving up sex for your old age?” Theologians can debate the purpose of human life, but its quality is a function of our human connections.

David Moon is president of Moon Capital Management. A version of this piece originally appeared in the USA TODAY NETWORK.