Here's an article based on the tax returns of F. Scott Fitzgerald from 1919 to 1940. Fitzgerald averaged about the contemporary equivalent of a half million bucks per year for two decades, mostly from short stories (and movie sales of short stories, such as Benjamin Button.) Short stories were a remarkably lucrative line of work between the wars, since the big magazines like the Saturday Evening Post were the chief venues for advertising nationally distributed products like cars. After 1950, advertising dollars moved to television, and the really big magazines like Life, Look, and the Post folded up within a couple of decades.
People used to like to read short stories because each one was a story and it was short. Now, though, nobody reads short stories except other short story writers. And the stories always end with an "epiphany" in which the main character realizes his life is hopeless, as utterly doomed as, say, the contemporary short story author's life.
I've never seen cause and effect concerning the decline of short stories untangled. Did short stories become unpopular when they became depressing? Or did they become unpopular first, which then made the short story writers all depressed?
Novels don't come with advertising, so they weren't as lucrative for Fitzgerald. (By the way, why don't novels include advertising? I'm slowly making notes for a novel and I would be happy to have the book include advertising.) Fitzgerald only made $8,397 in royalties off The Great Gatsby during his lifetime. It didn't become a classic until WWII, when the Pentagon gave away paperback novels to soldiers. For whatever reason, it struck a chord with servicemen at war, and has since become a high school staple. Fitzgerald's grandchildren make a half million per year off the 84-year-old book.
That's pretty wild that Fitzgerald's descendant are making over a half mil per year off his books 69 years after he died. Do copyrights last forever these days? I recall that 98-year-old Irving Berlin was sore when his 75-year copyright on the song Alexander's Rag Time Band ran out in 1987, so I guess Congress decided 75 years just wasn't long enough.
I think the reason why everybody loves Gatsby is because everybody wishes they'd been invited to Gatsby's parties, but the book also lets you feel morally superior to the soulless people who, unlike you, were invited. So, what's not to like?