Stories you read when you're the right age never quite leave you. You may forget who wrote them or what the story was called. Sometimes you'll forget precisely what happened, but if a story touches you it will stay with you, haunting the places in your mind that you rarely ever visit.
I'm a failed poet. Maybe every novelist wants to write poetry first, finds he can't, and then tries the short story, which is the most demanding form after poetry. And, failing at that, only then does he take up novel writing.