The Walker House is a Queen Anne Revival private residence located in Garrison, New York that is listed on the National Register of Historical Places. It was built in 1885 by Samuel Sloan, a prominent 19th century railroad magnate, on the grounds of his own Garrison estate, Oulagisket. In 1894, Sloan gave the Walker House to his daughter, Elizabeth, and her husband, Joseph Walker, Jr., although it had been initially used by one of his sons, William S. Sloan. Sloan built it next to Wyndune, the house that he had built and given in 1883 to Elizabeth's older sister, Margaret, and her husband, Rev. Joseph Rankin Duryee. The Walker House, Oulagisket, and Wyndune all remain today as privately owned homes, monuments to a long-ago era in the history of the Hudson Valley.

Samuel Sloan, a regular business partner and ally of J.P. Morgan, was a well-known railroad executive, developer, and builder during the second half of the nineteenth century, best known for his 32-year tenure as president and a director of the Delaware, Lackawanna and Western Railroad. As a prominent New York businessman, he also served as a director of a variety of other corporations, including Western Union Telegraph Company, United States Trust Company, Consolidated Gas Company, Farmers Loan and Trust Company, and the predecessor of what is today Citibank. His name is engraved in stone on the wall in the former Citibank headquarters at 55 Wall Street, and a large statue of him is located the Hoboken Terminal in Hoboken, New Jersey. His obituary in The New York Times said of him, "Samuel Sloan, Builder of Railroads: No name in the financial and railroad world is more closely identified with building of the great railroad systems which serve New York than that of Samuel Sloan" (obituary of Samuel Sloan, The New York Times, September 23, 1907).

Garrison is located in the Hudson Valley approximately 40 miles north of New York City and sits in the middle of the Hudson Highlands, on the eastern bank of the Hudson River, directly across from West Point. As rail service increasingly developed north of New York City, some of its wealthiest and most prominent families began to build large, luxurious estates in and around Garrison and throughout the Hudson Valley, with the most affluent utilizing their own private rail cars to travel back and forth. Family names such as Morgan, Vanderbilt, Gould, Roosevelt, Osborn, Fish, Livingston, Cornish, Mills, and others increasingly proliferated throughout the Hudson Valley and surrounding Hudson Highlands.

Sloan was noted for having been deeply devout, and one son-in-law, Joseph Rankin Duryee, was a prominent minister in New York City during the last quarter of the 19th century and first quarter of the 20th. From 1879 through 1920 he served as pastor of what was initially called the Seventh Avenue Chapel of the Dutch Collegiate Reformed Church, located at the corner of Seventh Avenue and 54th Street. It later became independent and was renamed the Grace Reformed Church, with Duryee continuing as its pastor (The Century Association Biographical Archives Foundation, entry for Joseph R. Duryee;see also, "A Daytonian in Manhattan," May 7, 2018, "The Lost Grace Reformed Church").

In 1927 Duryee wrote and published a 175-page-long biography of Sloan and his wife, Margaret Elmendorf Sloan, covering their lives and family, time in the Hudson Valley, and Sloan's business career (Duryee, Joseph Rankin: The Story of Samuel and Margaret Sloan; Published Privately for Family Circulation Only, 1927, 100 copies; reprinted, 2013, ISHI Press International). The biography states that, because of the deaths of two of their children in 1861, the Sloans began looking for a location where they could build a summer estate outside of New York City. It was common at the time to believe that summering outside of crowded metropolitan areas improved health generally and particularly for children. The Sloans began buying land in Garrison during the Civil War and, according to the Duryee biography, were encouraged to do so by Governor Hamilton Fish and also by William H. Osborn, both of whom had recently built extensive summer estates in Garrison. The Sloans built their own Garrison summer estate, Oulagisket (later renamed Lisburne Grange by his son) in 1864, immediately adjacent to the Osborn estate.

Sloan and his wife built the Walker House near Oulagisket and adjacent to Wyndune in 1885. It was originally occupied by their son, William, but was given to their daughter, Elizabeth La Grange Sloan (1862 - 1960), and her husband, Joseph Walker, Jr.(1858-1927). The Walkers had been married on April 27, 1887, at the South Reformed Church on Fifth Avenue. In its report of the wedding, the next day's New York Times described it in the headline as "FIFTH AVENUE THE SCENE OF AN UNUSUALLY BRILLIANT WEDDING." Guests included the recently elected reform mayor of New York City, Abram Hewitt, and his wife, Sarah Cooper Hewitt, and the paper reported that the Walkers left on an afternoon train up the Hudson, to be followed by a three-month tour abroad. Walker became a senior partner and president of the investment banking firm of Joseph Walker & Sons and member of the New York Stock Exchange. In 1982 both the Walker House and Oulagisket were added to the National Register of Historic Places as part of the Hudson Highlands Multiple Resource Area.

The Queen Anne Revival architectural style flourished in the United States during the last quarter of the 19th century. It followed the Second Empire style, which became popular beginning in the second quarter of the 19th century and reached its peak during the third quarter. The Walker House contains many elements of the Queen Anne Revival style of architecture, including an asymmetrical facade; dominant front-facing and side-facing gables; a wrap-around covered porch; spindle work; a polygonal bay; and a slate roof. It is situated on a four-acre (1.6 ha) lot. It is surrounded by tall trees; the deep ravine created by Arden Brook as well as the steep slope of Fort Hill elevate the house above the road and, together, partially obstruct the view of the house from it. A driveway curves up the slope from the east to the front facade.

The building itself is a three-story frame home on a brick foundation with several nested roofs shingled in slate, together with a copper porch roof. The wraparound porch extends across the front of the house and around most of the west side and part of the east side. The steeply-pitched main roof is done in a chateau style with eyebrow vents on three sides of the house; the existing badly-deteriorated vents were replaced with custom-fitted copper vents in 2015, while the original crockets on the gable ridges were removed when the original slate roof was replaced with new slates in 2001–2002.

Three large brick chimneys pierce the extensions to the north and south, and a polygonal bay extends further from the north. A veranda with turned posts and corner brackets extends around three sides of the structure.

In 1891, the New York City firm of Watson & Co. published its monumental atlas of the Hudson River in 35 separate sheets (Beers, Frederick W.: Atlas of the Hudson River Valley from New York City to Troy,; Watson & Co., 1891). This great work of cartography was issued in 35 separate sheets, each approximately 30" x 20". Each sheet shows a different section of the Hudson River measuring from four to five miles north to south, together with a view of approximately four miles east to west on either side of the Hudson. Each sheet identifies the owners of the various estates and properties and outlines their approximate dimensions.

Garrison is shown on Sheet 12, and it identifies Oulagiskat, Wyndune, and the Walker House, except that the Walker House is listed as that of W. S. Sloan, which is consistent with the Duryee biography's use of 1894 as the date of transfer of the house by Sloan to the Walkers. Section 12 also shows the Osborn estate immediately south of the Sloan estate, with the Fish estate below that and Cragston, the great 700 acre J.P. Morgan estate across the river and to the south. When built, the Walker House was located on the large Sloan family summer estate, together with Oulagisket, Wyndune, and numerous supporting buildings. The original Sloan estate has been divided over the years; its various out-buildings have all been converted to residential use; and all are now under separate ownership. According to the Duryee biography, Sloan had built the first summer house on his estate for his daughter, Maria, in 1872, following her marriage to Edgar Stirling Auchincloss. Its location on the Sloan estate is not otherwise described by Duryee, and he refers to it only as the "Auchincloss Cottage."

The house remained in the Walker family until 1956, when it was sold to Walker O. and Abby H. Cain. Walker O. Cain (no known relation of Joseph Walker), a prominent twentieth-century architect, practiced at McKim, Meade & White, which firm was ultimately succeeded by his own firm, Walker O. Cain Associates. Since the house had been a summer residence when owned by the Walkers, modern mechanical services were added to the house after the 1956 sale to make it usable year-round. The house was later sold again in 2000 and then in 2005. Following both the sale in 2000 and that in 2005, the house was extensively renovated and updated. In 1978, the railing from the widow's walk on top of the roof was removed because of its general deterioration, but kept in storage for possible future restoration. The widow's walk was completely rebuilt and reinstalled in 2012. Because only two families (the Walkers and then the Cains) successively owned the house for over 100 years, many original period details remained intact and were preserved in later restorations.