They chose the younger brother, Frederick (Rick) Law Olmsted Jr., whose accomplishments were already legion, to manage the project. For the mansion, they hired the young architecture firm of John Mead Howells and Isaac Newton Phelps Stokes from New York City.
Initial work on the property encompassed entrance and service roads, boundary walls and gates, a location for the garage and stables, and site infrastructure. In a hollow to the northeast, near the proposed tennis court, an “Informal Garden” was designated — the future location for Mrs. James’ Blue Garden.
From 1911–1913, plans for the Blue Garden were developed and refined. A cruciform shape, terminated at each side by a semi-circular apse, was delineated by low walls. Within the walls, planting beds and paths echoed the garden’s form, with a square pool at the center of the cross axis. At the southwest end of the garden an elevated pergola created a vantage point from which an uninterrupted vista along the garden’s axis, across the long pool and open lawn — the "plaisance" — was terminated by another elevated pergola at the northern end.
This garden room was shaped to classical proportions and enlivened by an unusual horticultural palette requested by Mrs. James, “a monochromatic concentration of purples and blues.” Hidden behind a dense border of trees and shrubs, it was a secret garden, a hortus conclusus. From its opening celebratory spectacle, “The Masque of the Blue Garden” in August 1913, events in this garden were covered in detail by newspapers and periodicals across the country.
After the deaths of Arthur Curtiss and Harriet James in 1941, attention paid to maintain the Blue Garden, a labor-intensive property, plummeted. The mansion was devastated by fire in August 1967 and eventually demolished. The property was sub-divided into commodious house lots in the 1970s. Slowly, the plantings and pergolas of the once-celebrated Blue Garden were subsumed under a thick covering of invasive trees and vines.