The Sisters were shown in an 1850 survey map of the San Francisco Bay area made by Cadwalader Ringgold. They, along with The Brothers on the opposite side of the San Pablo Strait, were reserved for military purposes by order of President Andrew Johnson in 1867. After many a court battle the plans were scrapped.