There is much conflicting information about the construction of the attractive brick and stone Oakdale Railroad Station. It was built to replace a shabby wooden structure "not in keeping" with other Idle Hour buildings, which was constructed in 1869 when the rail line was opened between Islip and Patchogue. It is possible that this building was located closer to the East Gate House.

According to the Patchogue Advance of November 29, 1889, the Oakdale Railroad Station was designed by Sayville architect Isaac Greene, working at the direction of Alva Vanderbilt, and constructed at a cost of $30,000. One or two years later the decision was made to raise the roof and install dormers in it, so that the stationmaster could live above his office. It is said that Commodore Bourne paid for this refitting. In 1890 the Suffolk County News reported that Julia Roberts, the wife of Christopher Roberts, the owner of Pepperidge Hall, adjacent to the Vanderbilt estate, conveyed property to the Long Island Railroad Station on the north side of the old Montauk Highway.

One story has the station built so that the wedding guests arriving for the Vanderbilts' daughter Consuelo's nuptials would have a suitable debarkation point. However, Consuelo was wed in New York City; only she and her groom, the Duke of Marlboro, and their servants, journeyed to Idle Hour for their honeymoon. It is also mentioned that the train arrived ahead of schedule, so that there was no carriage awaiting them. Consuelo and her new husband walked part of the way to the estate before they met the carriage on its way to the station.

Another story has the station constructed so that it would be ready for the wedding, held at Idle Hour, of Mary Virginia Smith, the sister of Alva, to Fernando Yznaga.

In 1899, the Suffolk County News had another Vanderbilt story related to the railroad station. Reportedly, a large crowd had gathered to await the special train bringing William K. Vanderbilt and his bride, Virginia Fair, to Idle Hour for their honeymoon. As the train pulled into the station, there was a rush of reporters and their cameras. Everyone got a very good view of some baggage and a bicycle intended for the bride's use, but that was all. The train had stopped at the crossing adjacent to the West Gate Lodge. The bridal pair alighted there and walked quickly through the massive iron gates and on to the mansion. It is said that the groom gave the five man train crew two hundred dollars as a tip.

In 1963, in response to an inquiry, the Long Island Rail Road Public Relations Department stated that "the Vanderbilts had no part in the design, location, or construction of the station."

Finally, a look at some smaller buildings here & there in Idle Hour.