Rappahannock Academy is an unincorporated community in Caroline County, Virginia, United States. Rappahannock Academy is located on U.S. Route 17 13 miles (21 km) southeast of Fredericksburg. Rappahannock Academy has a post office with ZIP code 22538. The community was named after the Rappahannock Academy & Military Institute, a now-defunct military academy in the community.

Moss Neck Manor was listed on the National Register of Historic Places in 1999.