Officially named F. W. Gross High School on January 2, 1936, the Victoria Colored School, in Victoria County, Texas, United States, was built in 1901 under authorization of the newly elected school board. The Victoria Independent School District, formed in 1898, chose a design by architect Jules Leffland for the district's second high school, a segregated school for the "coloreds". The Colored School was built at 702 E. Convent Street, in the Diamond Hill area of Victoria, which had been settled by freedmen after the Civil War. The building was next door to the Freedmen's School (built in 1868 by the Freedmen's Bureau of Goliad County), which was turned into a dormitory for teachers of The Colored School.

Slavery had been illegal in Texas under Mexican rule but, in 1836, with the establishment of the Republic of Texas, slavery was legalized. For the first time, cotton farming in the Victoria area was profitable. The flat, fertile, well-watered, "empty" ranch lands throughout Texas interested cotton growers in the southeast. They began immigrating to Victoria, through the ports of Indianola and Linnville, buying land and bringing slaves with them to work the fields.

Victoria had played a role in the Confederacy. Confederate ports were blockaded by Union troops. The Trans-Mississippi Department of the Confederacy developed a supply line, the Cotton Road, that funneled cotton overland, through Texas, to Matamoros in Mexico. There, the cotton was exchanged for supplies.

Twenty-five years after the legalization of slavery, at the outbreak of the Civil War, slaves equaled more than half of Victoria County's population. The influx of southerners had surpassed the influx of the strict-abolitionist German immigrants, and Victoria County voted overwhelmingly for the succession of Texas.

The war eventually destroyed the market for cotton. Former slaves either became sharecroppers or drifted into the city, establishing a community and social infrastructure. After the war, area leaders wanted to establish schools for African Americans that would channel them into vocational training for the kind of workforce area businesses demanded. Children in grades one to seven attended free, but citizen resistance to free education resulted in students in grades eight through 10 paying a fee until 1906.

The first principal of the Victoria Colored School was Frederick William Gross (1861–1915), a native of Marshall, Texas, with degrees from several well-known universities. He encouraged literary pursuits and stressed academic performance among the students and, under his leadership the first six years, the school became recognized among black colleges.

In 1907, C.H. McGruder succeeded Gross as principal. By 1920, the enrollment in McGruder's pet program, the Industrial Arts Department, strained the capacity of the school.

Despite these successes, when the school applied for accreditation as a preparatory school for the state college in 1932, it was denied. It was eventually revealed that a lack of qualified teachers, space, and science equipment had been cited as reasons for the school's failure to gain accreditation. Seven years and federal grants later brought accreditation to the school. The school continued to act as an exclusively black school until 1966.

The building at 702 E. Convent Street no longer serves as a school. On December 9, 1986, the location was certified by the National Register of Historic Places and, in August, 1986, became a Recorded Texas Historic Landmark.

• List of High Schools in Victoria County, Texas
• American Civil War
• Bagdad, Mexico
• National Register of Historic Places listings in Victoria County, Texas