As for me, I grew up in a little town (Mer Rouge, in Northeastern Louisiana) that was so sleepy that it only sprung to life when the train came through (which luckily was several times a day).  As a small child, I played so much on a old steam engine at the Bernstein Park Zoo in Monroe that my parents thought I had imprinted on it like Konrad Lorenz's ducks.

My dad never really had any interest in trains but I think everyone else in the family did. My paternal grandfather was a railfan and in his early twenties was playing around with a Class B Shay on a local logging operation until he got a "real job" as an electrician but still had a fascination with what was then Missouri Pacific Iron Mountain. He got me interested in his stories of the Hammer-heads (local slang for the 6600 class 4-6-2's with the Elesco feedwater heaters) and the parade of "Mikes" and "Ten Wheelers" that used to ply the Louisiana division south of Little Rock.  Sadly, neither he or his railfanning buddy Mr. Hume took photos although they were well known around Star City, Reader, and other places where steam was still operating. When I was born, he encouraged my interest in trains and aircraft (he also took up flying as a hobby). Family travels around Louisiana to visit relatives introduced me to Bunkie and Plaquemine on the
TP, and to the Mopac Natchez branch. (My maternal grandfather's family had a store in Wisner, LA, across the street from the MoPac depot on the branchline between Collinston and Natchez and they were only too happy to sell steel toed boots to railroad employees).

My godfather had an octogenarian uncle who was an engineer on the Gulf Coast Lines and ran the Orleanian/Houstonian in spite of his cataracts-- the joke was if Uncle Eddie was driving the train that day, you took the friendly Espee-- I think he started out on the TP but moved over to the GCL after MoPac took over. 

In 1966 I got to ride to Monroe, LA on a MoPac passenger train become hopelessly hooked. In 1970 I got a Mantua-Tyco F7 which I soon managed to crudely repaint and decal with buzzsaws.  When I got a little Tyco 4-6-0 the next Christmas, I was in mortal fear my grandfather would take away from me and give it the full St. Louis Iron Mountain and Southern treatment. Sadly, my grandfather died a few years after that but had he taken better car of himself, he probably would have stolen my treasured Joe Collias books like Last of Steam or Mopac Power.

High school led to Scouting (my biggest regret was missing the Jamboree where a fellow named Downing Jenks gave the keynote address), football, girls, and cars, so trains got pushed to the side. After my freshman year in college, I injured my right hand in an accident and got the use back by building 1/35 military miniatures and my model trains. I dabbled with model trains all through grad school (doing this kind of stuff is a great way to wind down after a long day in lab). I still model today and am planning on building either Hope or Gurdon in a spare room in my next home.