Carroll City-Mount Olivet Cemetery is a historic site located in Carroll, Iowa, United States. It was listed on the National Register of Historic Places in 2022.

Carroll City, as Carroll was originally known, established a cemetery southeast of the central business district and south of the railroad tracks in the early 1870s. It was maintained by the local chapter of the Independent Order of Odd Fellows and a part-time city employee. Little else is known about the cemetery's early years. An orchard between the city and Catholic cemeteries was acquired by the city in 1933 for use by the city cemetery. The following year Ray Wyrick, a Des Moines landscape engineer, presented a plan to redevelop the cemetery. In addition to design work and plantings, the plan included updating the cemetery's rules and regulations to make it self-supporting. He was hired by the city to carry-out the work over five years. Workers were provided through the Works Progress Administration (WPA).

In the mid-1870s St. Joseph's was established as the first Catholic parish in Carroll. They established a parish cemetery south of the city cemetery in 1876. In 1885, Saints Peter and Paul Parish was created for German speaking Catholics. They established their own cemetery south of St. Joseph Cemetery the following year. It was expanded to the west in 1927. St. Joseph and Saints Peter and Paul Cemeteries became known as the Catholic Cemetery of Carroll. In 1954, they merged and were renamed Mount Olivet Cemetery. In 1970, the cemetery's deed was transferred to the City of Carroll.

The City Cemetery features a stone wall capped with an wrought iron fence that was built by the WPA in 1936. The gates were created by Gate City Iron Works of Des Moines. In the Mount Olivet section of the cemetery is a "Crucifixion group" of statues. The original group of statues of the Virgin Mary, John the Apostle, and the crucified Christ was erected in the early 1900s. In the early 1960s a new marble cross and altar were added to the original statues. The stone Anneberg family mausoleum, the only one in the cemetery, was built in 1927. Three above-ground crypts were added between 1955 and 2014. There are also war memorials from the American Civil War onward.

A variety of grave marker styles are employed throughout the cemetery. Obelisks are prominent in Section 1 of the original City Cemetery. Large die, base, and cap granite and marble markers are found in the older sections of the cemetery. Pedestal markers are located in the older sections of both cemeteries. In the Catholic cemetery, the pedestal markers are often capped with a cross. In other sections they are capped with vaulted tops, urns, books, or figures. Some markers have inscriptions in German. There are markers that are stone carved to look like logs or tree trunks. There are also stone benches and metal crosses. Two grave markers in the Catholic section incorporate concrete kneelers. Three family plots are marked with large boulders with metal plaques. Other markers include "chest tombs, cruciforms, round columns or pillars, bolsters (stone cylinders) on footings; four polished granite ledger slabs, and one ledger slab covered with tiles." The markers in the Potter's field sections are simple, horizontal stone markers.