Avery Brundage (/ˈeɪvri ˈbrʌndɪdʒ/; September 28, 1887 – May 8, 1975) was the fifth president of the International Olympic Committee, serving from 1952 to 1972, the only American and only non-European to attain that position. Brundage is remembered as a zealous advocate of amateurism and for his involvement with the 1936 and 1972 Summer Olympics, both held in Germany.

Brundage was born in Detroit in 1887 to a working-class family. When he was five years old, his father moved his family to Chicago and subsequently abandoned his wife and children. Raised mostly by relatives, Brundage attended the University of Illinois to study engineering and became a track star. He competed in the 1912 Summer Olympics, where he participated in the pentathlon and decathlon, but did not win any medals; both events were won by teammate Jim Thorpe. He won national championships in track three times between 1914 and 1918 and founded his own construction business. He earned his wealth from this company and from investments, and never accepted pay for his involvement in sports.

Following his retirement from athletics, Brundage became a sports administrator and rose rapidly through the ranks in United States sports groups. As leader of America's Olympic organizations, he fought zealously against a boycott of the 1936 Summer Olympics, which had been awarded to Germany before the rise of the Nazi regime and its escalating persecution of Jews. Brundage successfully prevented a US boycott of the Games, and he was elected to the IOC that year. He quickly became a major figure in the Olympic movement and was elected IOC president in 1952.

As President of the American Olympic Committee, Brundage fought strongly for amateurism and against the commercialization of the Olympic Games, even as these stands increasingly came to be seen as incongruous with the realities of modern sports. The advent of the state-sponsored athlete of the Eastern Bloc countries further eroded the ideology of the pure amateur, as it put the self-financed amateurs of the Western countries at a disadvantage. The 1972 Summer Olympics at Munich, West Germany, were his final Games as president of the IOC. The event was marred by tragedy and controversy when eleven Israeli team members were murdered by Palestinian terrorists. At the memorial service, Brundage decried the politicization of sports and refused to cancel the remainder of the Olympics, declaring "the Games must go on." Although those in attendance applauded Brundage's statement, his decision to continue the Games has since been harshly criticized, and his actions in 1936 and 1972 seen as evidence of antisemitism. In retirement, Brundage married his second wife, a German princess. He died in 1975 at age 87.

Avery Brundage was born in Detroit, Michigan, on September 28, 1887, the son of Charles and Minnie (Lloyd) Brundage. Charles Brundage was a stonecutter. The Brundages moved to Chicago when Avery was five, and Charles soon thereafter abandoned his family. Avery and his younger brother, Chester, were raised mostly by aunts and uncles. At age 13 in 1901, Brundage finished first in an essay competition, winning a trip to President William McKinley's second inauguration. Avery attended Sherwood Public School and then R. T. Crane Manual Training School, both in Chicago. Crane Tech was a journey of 7 miles (11 km) by public transportation, which he undertook only after completing a newspaper delivery route. Even though the school had no athletic facilities, Brundage made his own equipment (including a shot and a hammer to throw) in the school's workshop and by his final year was written of in the newspapers as a schoolboy track star. According to sportswriter William Oscar Johnson in a 1980 article in Sports Illustrated, Brundage was "the kind of man whom Horatio Alger had canonized—the American urchin, tattered and deprived, who rose to thrive in the company of kings and millionaires".

After he graduated from Crane Tech in 1905, Brundage enrolled at the University of Illinois, where he undertook an arduous schedule of civil engineering courses. He received an honors degree in 1909. He wrote for various campus publications and continued his involvement in sports. Brundage played basketball and ran track for Illinois, and also participated in several intramural sports. In his senior year, he was a major contributor to Illinois' Western Conference championship track team, which defeated the University of Chicago (coached by Amos Alonzo Stagg).

After graduation, Brundage began work as a construction superintendent for the leading architectural firm of Holabird & Roche. In the three years he worked for the firm, he supervised the construction of $7.5 million in buildings—3 percent of the total built in Chicago in that time frame. He disliked the corruption of the Chicago building trades. Brundage's biographer, Allen Guttmann, points out that the young engineer was in a position to benefit from influence if he had wanted to, as his uncle, Edward J. Brundage, was by then-Republican leader of Chicago's North Side and would become Attorney General of Illinois.

Brundage had been successful in several track and field events while at Illinois. In 1910, as a member of the Chicago Athletic Association (CAA), he finished third in the national all-around championships (an American predecessor of the decathlon), sponsored by the Amateur Athletic Union (AAU), and continued training, aiming at the 1912 Olympics in Stockholm. At Stockholm, Brundage finished sixth in the pentathlon and 16th in the decathlon. Far behind on points, after eight events he dropped out of the decathlon, which he always regretted. He later moved up one spot in the standings in each event when his fellow American, Jim Thorpe, who had won both events, was disqualified after it was shown that he had played semi-professional baseball: this meant Thorpe was considered a professional athlete, not an amateur as was required for Olympic participation. Throughout his tenure as president, Brundage refused to ask the IOC to restore Thorpe's medals despite advocacy by Thorpe supporters. The committee eventually did so in 1982, after the deaths of both men. Brundage's refusal led to charges that he held a grudge for being beaten in Stockholm.

Upon his return to Chicago, Brundage accepted a position as construction superintendent for John Griffith and Sons Contractors. Among the structures he worked on for Griffith were the Cook County Hospital, the Morrison Hotel, the Monroe Building, and the National Biscuit Company warehouse. In 1915, he struck out on his own in construction, founding the Avery Brundage Company, of which his uncle Edward was a director. Brundage continued his athletic career as well. He was US all-around champion in 1914, 1916, and 1918. Once he had ceased to be a track star, he took up handball. As a young man, he was ranked in the top ten in the country and even in 1934, at the age of 46, he won one game out of two against Angelo Trulio, who had recently been the US national champion.

As Brundage approached the end of his track career, he began to involve himself in sports administration, at first through the CAA, then through the Central Association of the Amateur Athletic Union (of which the CAA was a member) and then, beginning in 1919, in the AAU. That group was involved in an ongoing battle for dominance over US amateur sports with the National Collegiate Athletic Association (NCAA). Athletes were often used as pawns in the battle, with one organization threatening to suspend those who participated in events sponsored by the other. Another venue for conflict was in the United States' National Olympic Committee (NOC), which was then called the American Olympic Committee (AOC), and which was AAU-dominated. In 1920, there was public outcry when the AOC chartered a disused troopship to carry home America's representatives in the 1920 Olympic Games in Antwerp; much of the team instead booked passage by ocean liner. In response, the AAU founded an American Olympic Association as a separate group, although it was still initially dominated by AAU representatives—it then selected the AOC. In 1928, on the resignation of then-AOA president General Douglas MacArthur, Brundage was elected president of the AOA; he was also elected president of the AOC, a post he held for over 20 years.

In 1925 Brundage became vice-president of the AAU, and chairman of its Handball Committee. After a year as first vice-president, he became president in 1928 and kept the post (except for a one-year break in 1933) through 1935. In that capacity, he was able to secure peace between the NCAA and AAU, with the former gaining the right to certify college students as amateurs, and greater representation on the AOA's executive board.

Brundage quickly displayed what writer Roger Butterfield termed "a dictatorial temperament" in a 1948 article for Life magazine. In 1929, American track star Charlie Paddock stated that Brundage and other sports officials were making money for the AOC by using him as a gate attraction, while treating him badly; Brundage shot back accusing Paddock of "untruths" and "sensationalism of the rankest sort". The runner turned professional, escaping Brundage's jurisdiction. In 1932, soon after winning three medals at the 1932 Summer Olympics in Los Angeles, track star Mildred "Babe" Didrikson appeared in an automobile advertisement, and the Brundage-led AAU quickly suspended her amateur status. Didrikson objected that she had not been paid and that regardless, the rules for maintaining amateur status were overcomplex. In the first of several well-publicized run-ins he had with female athletes, Brundage replied that he had not had any problem with the rules when an Olympic athlete himself, and stated, "You know, the ancient Greeks kept women out of their athletic games. They wouldn't even let them on the sidelines. I'm not so sure but they were right." According to Butterfield, Brundage was suspicious of female athletes, suspecting that some were actually men in disguise.

In 1931, the IOC awarded the 1936 Olympics to Germany, with the winter games in Bavaria and the summer games in the capital city, Berlin. After Germany was selected, several IOC members indicated that they were showing support for its democratic government, which was under attack from extremists in the hard economic times of the Great Depression. The Berlin Games were thrown in doubt, however, by the German July 1932 elections, in which the Nazi Party, led by Adolf Hitler, unexpectedly won the most seats in the Reichstag, the national legislature. The Nazis had expressed little interest in international sport, instead preferring the idea of "German games," in which German athletes would compete without what they deemed subhuman "Untermenschen" such as people of Jewish, Gypsy or African descent, thereby promoting their ideas of Aryan racial superiority and Germans as a "master race." When the Nazis attained power in January 1933, the Olympics were thought likely to be moved elsewhere.

Although the Nazis were suspicious of the chairman of the local Olympic organizing committee, Theodor Lewald, because he had a Jewish grandmother, they quickly saw the propaganda potential in hosting the Olympic Games. Lewald had intended to stage the Games on a shoestring budget; instead, the Reich threw its resources behind the effort. As the Nazi hatred of the Jews manifested itself in persecution, there were calls to move the Olympics from Germany, or alternatively, to boycott the Games. As head of the US Olympic movement, Brundage received many letters and telegrams urging action. American Olympic champion Lillian Copeland accused Brundage of "deliberately concealing the truth" about Hitler and Nazi Germany, was one of 24 former U.S. Olympic champions who petitioned the IOC in 1933 to move the Games from Germany, to no avail, and herself ultimately boycotted the Games.

IOC President Comte Henri de Baillet-Latour wrote to Brundage in 1933, "I am not personally fond of jews [sic] and of the jewish [sic] influence, but I will not have them molested in no way [sic] whatsoever." According to historical writer Christopher Hilton in his account of the 1936 Games, "Baillet-Latour, and the great and good around him, had no idea what was coming, and if the [IOC's] German delegates kept offering assurances, what else could they do but accept them?" Baillet-Latour opposed boycotting the Games, as did Brundage (who had learned in 1933 that he was being considered for IOC membership).

In her 1982 journal article on his role in the US participation in the 1936 Summer Games, Carolyn Marvin explained Brundage's political outlook:

The foundation of Brundage's political world view was the proposition that communism was an evil before which all other evils were insignificant. A collection of lesser themes basked in the reflected glory of the major one. These included Brundage's admiration for Hitler's apparent restoration of prosperity and order to Germany, his conception that those who did not work for a living in the United States were an anarchic human tide, and a suspicious anti-Semitism which feared the dissolution of Anglo-Protestant culture in a sea of ethnic aspirations.

Nazi pledges of non-discrimination in sports proved inconsistent with their actions, such as the expulsion of Jews from sports clubs, and in September 1934, Brundage sailed for Germany to see for himself. He met with government officials and others, although he was not allowed to meet with Jewish sports leaders alone. When he returned, he reported, "I was given positive assurance in writing ... that there will be no discrimination against Jews. You can't ask more than that and I think the guarantee will be fulfilled." Brundage's trip only increased the controversy over the question of US participation, with New York Congressman Emanuel Celler stating that Brundage "had prejudged the situation before he sailed from America." The AOC heard a report from Brundage on conditions in Germany and announced its decision. On September 26, 1934, the Committee voted to send the United States team to Berlin.

Brundage took the position that as the Germans had reported non-discrimination to the IOC, and the IOC had accepted that report, US Olympic authorities were bound by that determination. Nevertheless, it became increasingly apparent that Nazi actions would prohibit any Jew from securing a place on the German team. On this issue, Brundage stated that only 12 Jews had ever represented Germany in the Olympics, and it would hardly be surprising if none did in 1936.

Those who had advocated a boycott were foiled by the AOC, and they turned to the Amateur Athletic Union, hoping that the organization, though also led by Brundage, would refuse to certify American athletes for the 1936 Olympics. Although no vote took place on a boycott at the AAU's December 1934 meeting, Brundage did not seek re-election, and delegates elected Judge Jeremiah T. Mahoney as the new president, to take office in 1935. Although pro-boycott activities briefly fell into a lull, renewed Nazi brutality against the Jews in June 1935 sparked a resurgence, and converted Mahoney to the pro-boycott cause. In October, Baillet-Latour wrote to the three American IOC members—William May Garland, Charles Sherrill, and Ernest Lee Jahncke—asking them to do all they could to ensure a US team was sent to Germany. Garland and Sherrill agreed; Jahncke, however, refused, stating that he would be supporting the boycott. Brundage, at Baillet-Latour's request, took the lead in the anti-boycott campaign. Matters came to a head at the AAU convention in December 1935. Brundage's forces won the key votes, and the AAU approved sending a team to Berlin, specifying that this did not mean it supported the Nazis. Brundage was not magnanimous in victory, demanding the resignation of opponents. Although not all quit, Mahoney did.

Brundage believed that the boycott controversy could be used effectively for fundraising, writing, "the fact that the Jews are against us will arouse interest among thousands of people who have never subscribed before, if they are properly approached." In March 1936, he wrote to advertising mogul Albert Lasker, a Jew, complaining that "a large number of misguided Jews still persist in attempting to hamper the activities of the American Olympic Committee. The result, of course, is increased support from the one hundred and twenty million non-Jews in the United States, for this is a patriotic enterprise." In a letter which David Large, in his book on the 1936 Games, terms "heavy-handed," Brundage suggested that by helping to finance American participation in the Olympic Games, Jews could decrease anti-Semitism in the US. However, "Lasker, to his credit, refused to be blackmailed," writing to Brundage that "You gratuitously insult not only Jews but the millions of patriotic Christians in America, for whom you venture to speak without warrant, and whom you so tragically misrepresent in your letter."

Brundage led the contingent of American athletes and officials who embarked for Hamburg on the S.S. Manhattan at New York Harbor on July 15, 1936. Immediately upon arrival in Germany, Brundage became headline news when he and the AOC dismissed American swimmer Eleanor Holm, who was a gold medalist in 1932 and expected to repeat, for allegedly getting drunk at late-night parties and missing her curfew. There were various rumors and accounts of the married swimmer's pursuits while on board the ship; the gossip included statements that she was at an "all-night party" with playwright Charles MacArthur, who was traveling without his wife, actress Helen Hayes. Brundage discussed the matter with fellow AOC members, then met with Holm. Although the AOC attempted to send her home, Holm pleaded in vain for reinstatement; "to the AOC's horror," she remained in Berlin as a journalist, watching from the stands as the gold medal went to Dutch swimmer Nida Senff. Decades later, Holm told Olympic sprinter Dave Sime that Brundage had held a grudge against her, having propositioned her, and she turned him down. According to Guttmann, "Brundage has appeared, ever since [1936], in the guise of a killjoy." Butterfield noted that through the efforts of sportswriters who supported Holm, "Brundage became celebrated as a tyrant, snob, hypocrite, dictator and stuffed shirt, as well as just about the meanest man in the whole world of sports."

On July 30, 1936, six days after the American arrival in Germany, the IOC met in Berlin and unanimously expelled Jahncke. Two places for the United States were vacant, as Sherrill had died in June, but the minutes specifically note that Brundage was elected to the IOC in Jahncke's place.

One of the sensations of the Games was black American track star Jesse Owens, who won four gold medals. According to some American press stories, Hitler left the stadium rather than shake hands with him. This was not the case; IOC president Baillet-Latour had told Hitler not to shake hands with the winners unless he was prepared to shake hands with all gold medalists, which he was not. This, however, was not publicized. According to Butterfield, in later years, retellings of what Brundage termed "a fairy tale" roused the American to "acute fury." Hitler was, however, asked by his youth leader, Baldur von Schirach, to meet Owens, and he refused, saying, "Do you really think that I'd allow myself to be photographed shaking hands with a Negro?"

The question of the US 4 × 100 meters relay squad was another controversy that may have involved Brundage. The scheduled team included sprinters Sam Stoller and Marty Glickman, who were both Jewish. After Owens won his third gold medal, both men were removed from the relay squad in favor of Owens and fellow black athlete Ralph Metcalfe. The US track coach, Lawson Robertson, told Stoller and Glickman that the Germans had upgraded their squad and it was important to have the fastest team possible. In the event, the US team turned in back-to-back world record times in the heats and final to take the gold medal; the Italians were a distant second, edging out the Germans for the silver medal. Stoller and Glickman, who were the only Jews on the US track team and the only American athletes who went to Berlin and did not compete, did not believe the stated reason for their replacement. Stoller recorded in his diary that he and Glickman had been left out of the relay because the two other participants, Foy Draper and Frank Wykoff, had been coached by one of Robertson's assistants at the University of Southern California. Glickman conceded college favoritism as a possible reason, but thought anti-Semitism more likely, and his position—that he and Stoller had been replaced so as not to embarrass Hitler by having him see Jews, as well as blacks, win gold medals for the US track team—hardened in the following years. He believed Brundage was behind the replacement. Brundage denied any involvement in the decision, which remains controversial. Glickman went on to a lengthy career as a sports broadcaster, and was given the inaugural Douglas MacArthur Award (for lifetime achievement in the field of sports) in 1998, after Stoller's death, by the United States Olympic Committee (successor to the AOC). USOC chairman William Hybl stated that while he had seen no written proof that Brundage was responsible, "I was a prosecutor. I'm used to looking at evidence. The evidence was there"—though, as Large notes, "exactly what evidence, he didn't say." In the report that he submitted after the Games, Brundage called the controversy "absurd"; he noted that Glickman and Stoller had finished fifth and sixth at the Olympic trials at New York's Randall's Island Stadium and that the US victory had validated the decision.

Brundage's first IOC session as an incumbent member was at Warsaw in June 1937. The vice president of the IOC, Baron Godefroy de Blonay of Switzerland, had died, and Sweden's Sigfrid Edström was elected to replace him. Brundage was selected to fill Edström's place on the executive board. Edström had been a Brundage ally in the boycott fight, writing to the American that while he did not desire the persecution of the Jews, as an "intelligent and unscrupulous" people, "they had to be kept within certain limits". Brundage wrote to a German correspondent regretting that Leni Riefenstahl's film about the Berlin Olympics, Olympia, could not be commercially shown in the United States, as "unfortunately the theaters and moving picture companies are almost all owned by Jews".

The Berlin Games had increased Brundage's admiration for Germany, and he spoke out at a speech before the pro-Nazi German-American Bund at Madison Square Garden in October 1936, stating that "five years ago they [Germans] were discouraged and demoralized—today they are united—sixty million people believing in themselves and in their country ... We can learn much from Germany." In 1938, his construction company received the contract to build a new German embassy in Washington (this was not fulfilled as World War II intervened). Brundage joined the Keep America Out of War Committee and became a member of America First (he resigned from both the day after Pearl Harbor). Brundage was the subject of an FBI investigation in 1942, following allegations of pro-Nazi sympathies.

Although the 1940 Olympic Games were canceled due to World War II, Brundage sought to organize Western Hemisphere games that might be able to proceed despite the troubled international climate. Brundage was one of the leaders in the founding of the Pan-American Games, participating in the initial discussions in August 1940 in Buenos Aires. On his return, he arranged for the American Olympic Association to be renamed the United States of America Sports Federation (USASF), which would organize the United States Olympic Committee (as the AOC would now be called) and another committee to see to American participation in the Pan-American Games. Brundage became an early member of the international Pan-American Games Commission, although the inaugural event in Buenos Aires was postponed because of the war and was eventually held in 1951, with Brundage present. Despite his role in founding them, Brundage viewed the Pan-American Games as imitative, with no true link with antiquity.

War postponed any future Olympics, and fractured the IOC geographically and politically. With Baillet-Latour in German-occupied Belgium, Brundage and IOC vice president Edström did their best to keep channels of communication open between IOC members; according to Guttmann, "He and Edström perceived themselves as keepers of the sacred flame, guardians of an ideal in whose name they were ready once again to act as soon as the madness ended." Baillet-Latour died in 1942; Edström took on the duties of president, although he continued to style himself vice-president. Edström and Brundage did not await the end of war to rebuild the Olympic movement; Brundage even sent parcels to Europe in aid of IOC members and others in places where food was scarce. With Edström turning 74 in 1944, the Swede expressed concern as to who would lead the IOC if he should die, and suggested that Brundage become second vice-president, a newly created position. A mail ballot of IOC members who could be reached confirmed the choice the following year. When Edström was made president by the first postwar IOC session at Lausanne in September 1946, Brundage was elected first vice-president.

As vice president, Brundage served on a commission appointed at the IOC's London session in 1948 to recommend whether the 1906 Intercalated Games, held in Athens, should be considered a full Olympic Games. All three members of what came to be known as the Brundage Commission were from the Western Hemisphere and met in New Orleans in January 1949. The commission found that there was nothing to be gained by recognizing the 1906 games as Olympic, and it might set an embarrassing precedent. The full IOC endorsed the report when it met later that year in Rome.

Edström intended to retire following the 1952 Summer Olympics in Helsinki, when a successor would be elected. Brundage's rival for the presidency was Great Britain's Lord Burghley, an Olympic gold medalist in track in 1928 and president of the International Amateur Athletic Federation (IAAF). The balloting took place at the IOC session in the Finnish capital before the Games. Although Brundage was the executive board's candidate, he was disliked by some IOC members; others felt that the president should be a European. Private notes kept during the balloting reveal it to have been very close, but on the 25th and final ballot, Brundage received 30 votes to 17 for Burghley and was elected.

Throughout his career as a sports official, according to Guttmann, Brundage "was unquestionably an idealist." He often concluded speeches by quoting from John Galsworthy:

Sport, which still keeps the flag of idealism flying, is perhaps the most saving grace in the world at the moment, with its spirit for rules kept, and regard for the adversary, whether the fight is going for or against. When, if ever, the spirit of sport, which is the spirit of fair play, reigns over international affairs, the cat force, which rules there now, will slink away, and human life emerge for the first time from the jungle.

This ideal was best realized, Brundage believed, in amateur sports. The athlete, he stated, should compete "for the love of the game itself without thought of reward or payment of any kind," with professionals being part of the entertainment business. Amateurism, to Brundage, expressed the concept of the Renaissance man, with abilities in many fields, yet a specialist in none.

As the definition of "amateur" varied by sport, many of the battles Brundage engaged in concerned the question of what money or valuables an athlete could accept while retaining their amateur status, with some sports more liberal than others. In 1948, tennis allowed expense payments of up to $600 per tournament, while boxing permitted valuable prizes as awards. Enforcement of these rules often fell to National Olympic Committees, and Brundage found them less than enthusiastic about rules which hampered their own athletes in the pursuit of medals.

Both before and after becoming IOC president, Brundage was involved in a number of controversies involving threats to, or sometimes actual disqualification of, athletes for breach of amateur rules. In 1932, he was part of a special committee of the IAAF which disqualified Finnish runner Paavo Nurmi from the Los Angeles Games for allegedly accepting monetary compensation. At the 1948 Winter Olympics in St. Moritz, rival US ice hockey teams, sponsored by different accrediting organizations (one from the AAU and the other from AHAUS), came to the Games. The dispute proved difficult, and the IOC initially voted to cancel the tournament and eliminate ice hockey as an Olympic sport, but relented as organizers had sold thousands of tickets. A compromise was then reached: the AAU team, backed by Brundage and the AOC, would march in the opening ceremony, while the AHAUS team, not favored by Brundage but supported by the LIHG (the forerunner to today's IIHF), featuring former semi-professional players, were allowed to compete but could not earn an Olympic medal. However, since at the time the Olympic hockey tournament also doubled as that year's Ice Hockey World Championships, their results would be recorded for that competition, in which they finished fourth. In 1972, Brundage banned Austrian skier Karl Schranz from the Sapporo Winter Olympics for commercial activities, calling him "a walking billboard."

Eastern bloc countries were known for skirting the edge of the rules by having state-sponsored "full-time amateurs." Their Olympic athletes were given everything they needed to live and train, but were not technically paid to do it, and all the money came from the government. The Soviet Union entered teams of athletes who were all nominally students, soldiers, or working in a profession, but many of whom were in reality paid by the state to train on a full-time basis. This put the self-financed amateurs of the Western countries at a disadvantage. Near the end of the 1960s, the Canadian Amateur Hockey Association (CAHA) felt their amateur players could no longer be competitive against the Soviet team's full-time athletes and the other constantly improving European teams. They pushed for the ability to use professional players, but met opposition from the IIHF and IOC; Brundage was opposed to the idea of amateurs and professionals competing together. At the IIHF Congress in 1969, the organization decided to allow Canada to use nine non-NHL professional hockey players at the 1970 World Championships in Montreal and Winnipeg, Manitoba, Canada. The decision was reversed in January 1970 after Brundage said that ice hockey's status as an Olympic sport would be in jeopardy if the change was made. In response, Canada withdrew from international ice hockey competition and officials stated that they would not return until "open competition" was instituted. Günther Sabetzki became president of the IIHF in 1975, after Brundage had left the post of IOC president, and helped to resolve the dispute with the CAHA. In 1976, the IIHF agreed to allow "open competition" between all players in the World Championships. However, NHL players were still not allowed to play in the Olympics because of the IOC's amateur-only policy.

As IOC president, Brundage's views on amateurism came increasingly to be seen as outdated in the modern world, as the rules were tested by athletes who saw everyone making money but themselves. In 1962, against Brundage's opposition, the IOC amended the rules to allow sports federations to offer athletes "broken time" payments, compensating them for time missed from work, but only if they had dependents in need. In 1972, Brundage called for the elimination of the Winter Olympics after 1976, finding them hopelessly polluted by rampant commercialism, especially in alpine skiing. In his final speech to the IOC in Munich in 1972, Brundage maintained his position on amateurism: "There are only two kinds of competitors. Those free and independent individuals who are interested in sports for sport's sake, and those in sports for financial reasons. Olympic glory is for amateurs."

No German team was allowed at the 1948 Summer Olympics in London or the Winter Games in St. Moritz. Brundage was anxious to reintegrate Germany into the Olympic movement once the Federal Republic of Germany (West Germany, through Brundage's lifetime) was formed in 1949. Soon after the state's formation, its National Olympic Committee approached the IOC, seeking recognition, but there was still much animus towards Germany. Just prior to the IOC session in Vienna in 1951 (Brundage was still vice president), the German Democratic Republic (East Germany) also formed an NOC and requested recognition. This created controversy, as the Federal Republic and its NOC claimed to represent both West and East Germany, but did not control the latter. Despite lengthy discussions, no resolution was reached in 1951, and the matter was put over until February 1952, when a negotiating session was scheduled for Copenhagen. Although the East Germans came to Copenhagen, they refused to attend the session, which was eventually cancelled by Edström after the IOC officials and West Germans waited for hours in vain. The German team which competed in Helsinki that summer was entirely West German (with Saarland, then a French protectorate, competing as an independent Saar team).

In 1954, the East Germans resumed their attempts at recognition. The following year, after Brundage received assurances that the East German NOC was not government-run, the IOC voted to recognize it, but required that both East and West Germany (as well as the Saar) compete as part of a single German team in 1956. East Germany sent only 37 athletes to the 1956 Summer Olympics in Melbourne, and they lived and trained separately from their West German counterparts. For the Summer Olympics at Rome in 1960, under continuing IOC insistence that the two states send a single team, East Germany contributed 141 of the 321 athletes; competitors from both states lived in the same area of the Olympic Village. At the Opening Ceremony at Rome, Italian President Giovanni Gronchi marveled, much to Brundage's delight, that the IOC had obtained the German reunification which politicians had been unable to secure; Brundage responded, "But in sport, we do such things." Brundage saw the German participation as symbolic of the potential for the Olympic Games to overcome divisions to unite.

Despite the construction of the Berlin Wall beginning in 1961, which increased tensions between East and West, Brundage was successful in securing a joint German team for the 1964 Summer Olympics in Tokyo. Nevertheless, the East Germans, supported by IOC members from Warsaw Pact nations, aspired to have their own team. They made a major breakthrough when the IAAF (led by the Marquess of Exeter, the former Lord Burghley) recognized a separate East German team beginning with the 1966 European Athletics Championships. The East Germans did their best to get Brundage's support, and, at the IOC session at Mexico City in 1968, they were granted full membership, with their own team under their own flag, which they displayed on West German soil four years later at the Opening Ceremony at Munich. Brundage, while finally supporting full membership for East Germany, considered the matter a defeat for Olympic ideals.

Although Tsarist Russia had sent athletes to the Olympic Games, after the Soviet Union was formed, it declined to participate, considering the Olympics bourgeois. As early as 1923, the IOC attempted to lure the Soviets back into the fold; Brundage visited the USSR in 1934. He was impressed by the progress which had been made there since a visit he had made in 1912 after competing in Stockholm. Despite his anti-communism, Brundage wanted the Soviets to join the Olympic movement. According to Guttmann, "When Brundage had to choose between his hostility to communism and his commitment to the ideal of Olympic universality, he chose the latter. He wanted the Russians [sic] in the Olympics, communists or not."

During World War II, Brundage wrote to other IOC members that he had no objection to Soviet involvement in international sports, with representation on the IOC, if the USSR joined the international sports federations (ISFs). The IOC required that an NOC be independent of the government of the territory which it represents; there were concerns a Soviet NOC would not be. This was a problem not unique to communist states; a number of Latin American countries were starting to bring the local NOCs into the political structure, with an official naming the NOC chair—who might even be the country's political leader. This mixture of sports and politics worried Brundage.

Beginning in 1946, the Soviets began to join international federations; in 1951 their NOC was recognized by the IOC, and they began Olympic competition the following year. As few Soviet sports officials were internationally known, the IOC had little alternative than to accept the nominees of the USSR's government if they wished to have Soviet IOC members. The Soviet members were believers in sport, and completely loyal to their nation and to communist ideals. They quickly became the leaders of the IOC members from behind the Iron Curtain, who voted in accord with the Soviet members. Brundage visited the USSR at Soviet invitation (though at his own expense) in 1954. He deemed the nation's physical education program as "creating the greatest army of athletes the world has ever seen," warning (as he would often through the 1950s) that Americans were by comparison soft and unfit. Brundage found his view, often expressed in the press, that physical education and competitive sports made for better citizens, especially in the event of war, more enthusiastically embraced in the Soviet Union than in the United States. According to David Maraniss in his account of the 1960 Rome Games, Brundage's admiration for the Soviet Union's sports programs "in some ways mirrored his response two decades earlier to his encounters with Nazi Germany".

On his return, he related in an article for The Saturday Evening Post that he had confronted Soviet officials with information from defectors stating that the USSR was running year-round training camps and giving athletes material inducements for success. He also repeated the Soviet response, which questioned the defectors' integrity: "These men are deserters, traitors. Would you attach any truth to their statements had they been Americans and had turned against your country?" Since Brundage did not comment on the response, there was a storm of controversy in the press, which accused Brundage of being a Soviet dupe.

Despite the evident conflicts between amateurism and the Soviet system in which athletes received salaries and property at state expense, allowing them to train full-time, Brundage took no action against the USSR or Warsaw Pact nations with similar systems; when challenged on this point, he argued that Western nations did similar things, citing athletic scholarships as an example. The Soviet system remained in place.

The Republic of China, which then governed the mainland, had joined the Olympic movement in 1924, when the China National Amateur Athletic Federation was recognized by the IOC as the nation's NOC. China participated in the 1932 Olympics in Los Angeles, as well as in Berlin four years later and the first post-war Olympics at London in 1948. When the communists were successful in the Chinese Civil War and established the People's Republic of China (PRC) in 1949, most NOC members fled the mainland for the island of Taiwan. This left China with two rival NOCs, one on the mainland and one on Taiwan, each claiming to represent the whole of China.

Matters came to a head in 1952, when the mainland NOC (the All-China Athletic Federation), considering itself a continuation of the pre-1949 committee, wrote to the IOC stating that it desired to participate in the Helsinki Olympics to be held that year. As the Taiwanese also proposed to send a team, this conflicted with IOC rules stating that only one committee could represent a country, and both Chinese groups were unwilling to negotiate with the other, or to send a joint team. After considerable deliberation, the IOC decided that if either committee was recognized by the ISF for a sport, the committee could send athletes to participate in events in that discipline. In protest, Taiwan withdrew from the Games; the PRC sent a team to Helsinki, though it arrived ten days after the start of the Games. Brundage, president-elect when the decision was made to allow PRC athletes to compete, argued against the decision to allow mainland participation before its NOC was recognized, but he was overruled by his colleagues.

In 1954, the Brundage-headed IOC, in a narrow vote, recognized both committees, thus allowing both states to participate at Melbourne. Only the PRC's committee initially accepted, but when the Taiwanese NOC changed its mind and decided to send a team to the Games, the mainlanders withdrew in protest. Brundage took the position that despite similar concerns about state sponsorship as with the USSR, once the PRC's committee was recognized and reported to the IOC that all eligibility rules were observed, the international committee had to accept that unless it had evidence to the contrary. He was frustrated by the continuing controversy, considering the squabble a distraction from the goal of advancing the Olympic movement.

When continuing efforts to exclude the Taiwanese failed, in 1958 the mainlanders withdrew from the IOC. The following year, the IOC ruled the Taiwanese could not compete under the name Republic of China Olympic Committee, but would have to compete under some other name which did not imply they governed sports in China. Brundage and Exeter both advocated for the ruling, which they compared to having an Italian NOC represent only Sicily. The press interpreted the ruling to mean that Nationalist China had been expelled from the Olympic movement, and for the next year, the anti-communist Brundage found himself under attack in the press as a communist sympathizer. Although United States State Department officials attempted to persuade them to stand on principle, Taiwanese officials decided to participate in the Rome Games, hoping to secure China's first medal, and believing their NOC's continued presence helped keep mainland China out of the Games. Taiwanese athletes competed under the designation Formosa (an alternate name for Taiwan), and caused a sensation by briefly displaying a sign reading "Under Protest" at the Opening Ceremony; when Yang Chuan-Kwang took the silver medal in the decathlon, he was not allowed to display the Nationalist Chinese flag at the medals ceremony.

Brundage, through his tenure, slowly came around to the position advocated by Iron Curtain IOC members, that the important thing was to recognize the mainland, with Taiwan of less importance. Although the mainland Chinese were invited by the Munich Olympic organizers to send an observer delegation to Munich (they declined due to the Taiwanese presence), it was not until 1975, after Brundage's departure as president, that the PRC applied to rejoin the Olympic movement. The PRC again participated at the 1980 Winter Games at Lake Placid and then the 1984 Summer Olympics in Los Angeles; the island NOC competed as the Republic of China in 1968 and 1972; when refused permission to compete under that name in 1976, after Brundage's death, it boycotted the 1976 and 1980 games, returning in 1984 as Chinese Taipei.

In the late 1950s, protest against South Africa's apartheid regime reached the stage of seeking to exclude the nation from international sport. In 1956, government rules requiring separate events for whites and non-whites in South Africa were issued; non-whites received poorer facilities. Brundage initially opposed taking any action. The run-up to the 1960 Rome Olympics had seen tumult in South Africa, including the Sharpeville massacre and a crackdown on the African National Congress. Activists attempted to persuade Brundage that South Africa should be excluded from the Games. Brundage initially took the word of South African sport leaders that all citizens were able to compete for a place on the Olympic team, and that non-white South Africans simply were not good enough.

The drive towards a boycott was fueled by the large number of African nations which became independent in the late 1950s and early 1960s. To prevent the new nations from overwhelming the ISFs, Brundage proposed that the federations adopt weighted voting systems to allow earlier members to wield disproportionate influence, which some did. By 1962, with the suspension of South Africa from FIFA (the association football governing body), Brundage had come around to the position that South Africa's racist policies were inconsistent with the ideals of the Olympic movement. At the 1963 IOC session in Baden-Baden (moved there from Nairobi when Kenyan officials refused to issue visas to South African representatives), the IOC voted to suspend South Africa from the Olympics unless its NOC and government adopted non-discrimination policies regarding Olympic selection. This did not come to pass, and South Africa did not participate in 1964. In 1968, Brundage and the IOC invited a South African team (supposedly to be multiracial) to the Mexico City Games, but under a threatened boycott and with evidence of minimal South African compliance, withdrew it.

In 1971, the IOC, at its Amsterdam session, voted to strip the South African NOC of recognition. Although Brundage had hoped to keep South Africa within the Olympic movement, he believed that those who sought its expulsion had made the stronger case. South Africa did not return to the Olympics until the 1992 Summer Olympics in Barcelona, after the end of its apartheid government.

A parallel problem was that of Rhodesia, the British colony which had unilaterally declared its independence from the United Kingdom in 1965. Rhodesia had a white minority government. In May 1968, the United Nations Security Council condemned its government and asked nations not to honor its passports, and the Mexican government, set to host the Olympics later that year, complied with the ban. The IOC initially believed that sports facilities in the breakaway colony were not segregated, despite its government's policies. The proposed 16-member Olympic team included two black athletes. Because of this, Brundage supported Rhodesian participation at Mexico City, but he was overruled by the IOC; according to the head of the Rhodesian Olympic Committee, Douglas Downing, "His voice cries in a wilderness of spite." For Munich in 1972, the IOC decided to allow the Rhodesians to compete as British subjects, which by international law they were. African nations again threatened to boycott if the Rhodesians were allowed to participate, and, at its Munich session in 1972 just before the Games, the IOC narrowly voted to exclude the Rhodesians. Brundage was livid at the decision, believing that the IOC had yielded to blackmail. In 1974, after Brundage left office, the IOC found evidence of segregated facilities in Rhodesia, and it subsequently withdrew recognition from its NOC. Rhodesia returned to the Olympics in 1980 as recognized independent Zimbabwe.

Unpaid as IOC president, even for his expenses, Brundage sometimes spent $50,000 per year to finance his role. In 1960, the IOC had almost no funds. Brundage and the IOC had considered the potential of television revenue as early as the Melbourne Games of 1956, but had been slow to address the issue, with the result that television rights for the 1960 Games were in the hands of the Rome organizing committee; the IOC received only 5% of the $60,000 rights fee. Accounts submitted by the Rome organizers showed they lost money on the Olympics; the IOC would have received a portion of the profits, and had no money to offer the sports federations who wanted a percentage of the proceeds. In future years, the sale of television rights became a major source of revenue for the IOC, rising to $10 million by the 1968 Summer Olympics in Mexico City, and $1.2 billion, long after Brundage's death, at Athens in 2004. Brundage was concerned about the increasing revenue, warning IOC members in 1967, "The moment we handle money, even if we only distribute it, there will be trouble ..."

NOC representatives had met with Brundage and the IOC executive board from time to time, but many NOC representatives felt that Brundage was taking no action in response to concerns expressed by the NOC attendees. In the early 1960s, many NOCs, led by Italian IOC member Giulio Onesti, sought to bypass Brundage and the IOC by forming a Permanent General Assembly of National Olympic Committees (PGA-NOC), which Brundage strongly opposed and the IOC refused to recognize. The PGA-NOC from 1965 demanded a share of television revenue; it also desired that the ISFs, not the IOC, set policy on amateurism.

Brundage had been initially elected in 1952 for an eight-year term; he was re-elected unanimously in 1960 for an additional four years. Despite talk that he would be opposed by Exeter, Brundage's 1952 rival nominated him for the new term. Brundage was re-elected in 1964 by an announced unanimous vote, though Guttmann records that Brundage actually only narrowly turned back a challenge by Exeter. As Brundage's term as president neared its end in 1968, some IOC members, who saw him as hidebound, or just too old at 81 to effectively lead the organization, sought his ouster. Nevertheless, he was easily re-elected at the IOC session in Mexico City that year, though he pledged not to seek another four-year term, but to retire in 1972. Ireland's Lord Killanin was elected first vice-president. Killanin, seen (correctly) as Brundage's likely successor, was more sympathetic to the concerns of the NOCs, and attended PGA-NOC meetings. Brundage did not recognize the PGA-NOC, but did establish joint IOC-NOC committees to address NOC concerns. Although the PGA-NOC did not gain Olympic recognition, it remained a significant outside organization through Brundage's presidency, and according to Guttmann, "Brundage won a less than total victory and Onesti suffered a far from complete defeat. The I.O.C. had become far more attractive to the national Olympic committees and to their interests, and that is what Onesti called for in the first place."

With Brundage in Chicago or at his California home, day to day IOC operations were overseen at "Mon Repos", the IOC headquarters in Lausanne, by Otto Meyer, the IOC's chancellor. Brundage came to consider Meyer too impetuous, and dismissed him in 1964, abolishing the office. Eventually, Brundage promoted Monique Berlioux to be IOC director in the last years of his tenure, and apparently found her services satisfactory. Mon Repos, the former home of the founder of the Modern Olympics, Baron Pierre de Coubertin, proved too cramped for the IOC, which had to share space with de Coubertin's widow, who lived to be 101. In 1968, the IOC moved to new quarters at Lausanne's Château de Vidy.

The year 1968 had seen turmoil in the United States, including hundreds of riots, both before and after the assassination of Martin Luther King Jr. and continuing after the assassination of Robert F. Kennedy. Prior to the Olympics in Mexico City in October 1968, some African Americans, led by activist Harry Edwards, had urged a boycott of the Games, but found little enthusiasm among athletes, reluctant to waste years of effort. The atmosphere was made more tense by unrest in Mexico City before the Games, which left dozens dead.

There were racial tensions between black US athletes and their white counterparts; in one incident, African Americans blocked whites from the track. One black runner, Tommie Smith, told writers on October 15, "I don't want Brundage presenting me any medals". The following day, Smith won the 200 meters, and fellow African-American John Carlos took the bronze medal. The two men, after receiving their medals from IAAF president Lord Exeter, and as "The Star-Spangled Banner" played, raised black-gloved fists, heads down, in salute of black power. Brundage deemed it to be a domestic political statement unfit for the apolitical, international forum the Olympic Games were intended to be. In response to their actions, he ordered Smith and Carlos suspended from the US team and banned from the Olympic Village. When the US Olympic Committee refused, Brundage threatened to ban the entire US track team. This threat led to the expulsion of the two athletes from the Games. Other demonstrations by African-Americans also took place: the three African Americans who took the medals in the 400 meters race, led by gold medalist Lee Evans, wore black berets on the podium but took them off before the anthem while African-American boxer George Foreman, triumphant in the heavyweight division, waved a small American flag around the boxing ring and bowed to the crowd with fellow American boxers. Brundage's comment about the Smith-Carlos incident was "Warped mentalities and cracked personalities seem to be everywhere and impossible to eliminate." The USOC's official report omits the iconic photograph of Smith and Carlos with their fists raised; the local organizing committee's official film showed footage of the ceremony. Brundage, who termed the incident "the nasty demonstration against the American flag by negroes", objected in vain to its inclusion.

At the same IOC session in August 1972 in Munich at which the Rhodesians were excluded, the IOC elected Killanin as Brundage's successor, to take office after the Games. Brundage cast a blank ballot in the vote which selected the Irishman, considering him an intellectual lightweight without the force of character needed to hold the Olympic movement together.

Brundage hoped that the Munich Games would take the sting out of his defeat over the Rhodesian issue. Munich was one of his favorite cities (in 1975, the Brundageplatz there would be named after him), and the heitere Spiele ('cheerful Games') were designed to efface memories of 1936 and Berlin in the eyes of the world. They initially seemed to be doing so, as athletic feats, like those of gymnast Olga Korbut and swimmer Mark Spitz captivated viewers. In the early morning of September 5, 1972, Palestinian terrorists from the organization Black September entered the Olympic Village and took 11 Israelis hostage, demanding freedom for hundreds of Palestinians held in Israeli custody. Brundage, once informed, rushed to the Olympic Village, where he conferred with German and Bavarian state officials through the day, playing what Guttmann describes as a modest role in the discussions. German officials moved the hostages and their captors to Fürstenfeldbruck Air Base, where German police and troops tried a rescue late that evening. The attempt was bungled; the nine remaining hostages (two had been murdered earlier) and three of their captors were killed.

Even before the ill-fated rescue attempt, IOC officials began conferring. Killanin and other officials were in Kiel for the yacht racing; they hurried back to Munich. Just before 4 pm, Brundage called off the remainder of the day's events, and announced a memorial service honoring those who had already died for the following morning. Many Olympic leaders were critical of Brundage for his participation in the discussions with the government, feeling that this should have been left for the authorities and the local organizing committee, but all supported the memorial service, which was held the following day in the Olympic Stadium. There, before the audience in the stadium and the millions watching on television, Brundage offered what Guttmann called "the credo of his life":

Every civilized person recoils in horror at the barbarous criminal intrusion of terrorists into the peaceful Olympic precincts. We mourn our Israeli friends, victims of this brutal assault. The Olympic flag and the flags of all the world fly at half mast. Sadly, in this imperfect world, the greater and more important the Olympic Games become, the more they are open to commercial, political and now criminal pressure. The Games of the 20th Olympiad have been subjected to two savage attacks. We lost the Rhodesian battle against naked political blackmail. We have only the strength of a great ideal. I am sure the public will agree that we cannot allow a handful of terrorists to destroy this nucleus of international cooperation and goodwill we have in the Olympic movement. The Games must go on and we must continue our efforts to keep them clear, pure and honest and try to extend sportsmanship of the athletic field to other areas. We declare today a day of mourning and will continue all the events one day later than scheduled.

The crowd in the stadium responded to Brundage's statement with loud applause; according to Stars & Stripes, "Brundage's statement that 'the games must go on' took much of the heavy gloom away which has permeated Munich since early Tuesday [September 5, the day of the attack]."

Killanin, after his own retirement as IOC president, stated that "I believe Brundage was right to continue and that his stubborn determination saved the Olympic Movement one more time" but that Brundage's mention of the Rhodesian question was, while not inappropriate, at least better left for another time. According to future IOC vice president Dick Pound, the insertion of the Rhodesian issue into the speech "was universally condemned, and Brundage left office under a cloud of criticism that effectively undermined a lifetime of well-intentioned work in the Olympic movement". Brundage said after the Games, "I was severely criticized for that ... but the fact is that I did it on purpose. I had to. There was a principle involved and altho [sic] it was a terrible thing that some lives were lost, principles are just as important as human lives." Brundage subsequently issued a statement that he did not mean to imply the decision to exclude the Rhodesians, which he stated was "purely a matter of sport", was comparable to the murder of the Israelis. According to Alfred Senn in his history of the Olympics, the decision to continue the games "sat poorly with many observers"; sportswriter Red Smith of The New York Times was among the critics:

This time surely, some thought, they would cover the sandbox and put the blocks aside. But, no. "The Games must go on," said Avery Brundage, and 80,000 listeners burst into applause. The occasion was yesterday's memorial service for eleven members of Israel's Olympic delegation murdered by Palestinian terrorists. It was more like a pep rally.

Brundage retired as IOC president after the 1972 Summer Games. There were differing accounts of Brundage's state of mind during his retirement. IOC director Berlioux stated that Brundage would come to the Château de Vidy and take telephone calls or look at correspondence while he waited for Lord Killanin to turn to him for help. According to Berlioux, Brundage sometimes called her from Geneva and asked her to go there. The two would spend hours wandering the streets, saying little. Brundage's longtime factotum, Frederick Ruegsegger (1920-2000), described a different, tranquil, Brundage, whom he compared to an abdicated Japanese emperor.

His wife of nearly half a century, Elizabeth, died in 1971. Brundage had once jested that his ambition was to wed a German princess. In June 1973, this came to pass when he married Princess Mariann Charlotte Katharina Stefanie von Reuss (1936–2003), daughter of Heinrich XXXVII, Prince of Reuss-Köstritz. Von Reuss had worked as an interpreter during the Munich Games; she stated that she had met Brundage in 1955, when she was 19. When Brundage was asked by reporters about the 48-year difference in their ages, Brundage responded that he was young for his age and she mature for hers, and instead of 85 years to 37, it should be thought of as more like 55 to 46. Ruegsegger refused to be best man and stated after Brundage's death that the couple had dissipated much of Brundage's fortune through free spending, though Guttmann notes that some of those purchases were of real estate, which could be deemed investments.

In January 1974, Brundage underwent surgery for cataracts and glaucoma. The necessary arrangements had initially been made by Brundage's protégé, Spanish IOC member Juan Antonio Samaranch, who would become IOC president in 1980. At the last moment, Brundage cancelled the plans, choosing to have the surgery in Munich, near the home he had purchased in Garmisch-Partenkirchen, site of the 1936 Winter Olympics. After a month and a half, Brundage was discharged from the hospital, though whether the surgery had improved his vision was disputed, with Mariann Brundage stating that it did and Ruegsegger stating the contrary. Now frail, at age 87 he went with his wife on a final tour of the Far East. Despite the efforts of Olympic officials on his behalf, he was not given an invitation to mainland China, source of much of the art he loved. In April 1975, Brundage entered the hospital at Garmisch-Partenkirchen with flu and a severe cough. He died there on May 8, 1975, of heart failure, and was buried at Rosehill Cemetery in Chicago.

In his will, Brundage provided for his wife and for Ruegsegger, as well as making several charitable bequests. He left his papers and memorabilia to the University of Illinois; he had already given it $350,000 to fund scholarships for students interested in competing in sports who do not receive an athletic scholarship.

In 1927, at the age of 40, Brundage married Elizabeth Dunlap, who was the daughter of a Chicago banker. She was a trained soprano, which was a talent that she exhibited to people who visited the Brundage home. She had a strong interest in classical music. This interest might not have been fully shared by her husband, who said that a performance of Wagner's Die Walküre "started at 7 o'clock, at 10:00 pm I looked at my watch and it registered exactly 8 o'clock". Elizabeth died at age 81 in 1971.

In 1973, Brundage married Princess Mariann Charlotte Katharina Stefanie von Reuss. He had no children with either of his two wives. However, during his first marriage Brundage fathered two sons out of wedlock with his Finnish mistress, Lilian Dresden. His affair with Dresden was one of many. The children were born in 1951 and 1952, at precisely the time that Brundage was being considered for the presidency of the IOC. Though he privately acknowledged paternity, Brundage took great pains to conceal the existence of these children; he was concerned that the truth about his extra-marital relationships might damage his chances of election. He requested that his name be kept off the birth certificates. Brundage visited his two sons periodically in the 1950s, visits that tailed off to telephone calls in the 1960s and nothing in his final years. He did establish a trust fund for the boys' education and start in life, but after his death, unnamed in his will, they sued and won a small settlement of $62,500 each out of his $19 million estate.

After its founding in 1915, a large source of the Avery Brundage Company's business was wartime government contracts. Brundage, who applied for a commission in the Army Ordnance Corps but was rejected, in the postwar period became a member of the Construction Division Association, composed of men who had built facilities for the military, and later became its president from 1926 to 1928.

In the 1920s, Brundage and his company became very active in constructing high-rise apartment buildings in Chicago. He used rapid construction methods, allowing clients to begin realizing income from their investments quickly—the Sheridan-Brompton Apartments (1924) overlooking Lincoln Park, were built in five months, allowing the start of $40,000 in monthly rental income, offsetting a monthly mortgage payment of $15,000. Often, the Brundage Company was involved in the ownership of the apartments: 3800 Sheridan Road (1927), a 17-story building costing $3,180,000, was owned by a company which had as its president and treasurer Chester Brundage, Avery's younger brother. It was constructed in eight months, through the Chicago winter, using an onsite concrete mixing plant. This temporary structure also provided office space for the construction. Another source of income for Brundage and his company was hotel construction, for which he was often paid in part with stock in the new facility. One president of an engineering firm specializing in large structures called Brundage's methods on the Shoreham Hotel "progressive, snappy, [and] up-to-date" and "straightforward and honest".

In 1923, Brundage constructed a massive assembly plant on Torrence Avenue on Chicago's South Side for the Ford Motor Company. At $4 million in cost and bringing 16 acres (6.5 ha) under one roof, it was the largest industrial plant built by Brundage. Constructed in ten months, the new facility helped meet the national demand for Model T cars in the 1920s, and in 1950, produced 154,244 vehicles. A plant for Hubbard & Co. was erected in 125 days despite an unusually harsh Chicago winter. Despite later statements from Brundage that he avoided public works due to corruption, he built the 23rd Street viaduct as part of the South Shore Development project; Brundage's viaduct extended Chicago's shoreline into Lake Michigan at a cost of two million dollars. By 1925, the Avery Brundage Company was acclaimed for speed, innovation and quality, and had a payroll of $50,000 a week.

Although the start of the Depression in 1929 was a major setback for Brundage, he rebuilt his wealth by investments in real estate, also accepting interests in buildings he had constructed in lieu of payments the owners were unable to make. He later stated that "you didn't have to be a wizard" in order to "buy stocks and bonds in depressed corporations for a few cents on the dollar—and then wait. I was just a little lucky." According to historian and archivist Maynard Brichford, Brundage "emerged from the difficult depression years with a substantial annual income, a good reputation, and excellent investments". His foresight resulted in a fortune which by 1960 was estimated at $25,000,000.

A major Brundage investment was Chicago's La Salle Hotel, which had been built in 1908. Located in the heart of The Loop and the city's financial district, Brundage first leased it in 1940, later purchasing it. When the hotel was seriously damaged by fire in 1946, Brundage spent about $2.5 million remodeling and modernizing it. As Brundage made a home there during his time as IOC president, the hotel became famous in international sports as his residence. He sold the hotel in 1970, but later reclaimed it when the purchaser failed to make required payments.

Brundage's interest in Asian art stemmed from a visit he made to an exhibition of Chinese art at the Royal Academy in London in early 1936, after the Winter Olympics in Garmisch-Partenkirchen. Brundage stated of the experience, "We [his first wife Elizabeth and himself] spent a week at the exhibition and I came away so enamored with Chinese art that I've been broke ever since." He did not begin active collecting until after the Brundages' two-week visit to Japan in April 1939, where they visited Yokohama, Kyoto, Osaka, Nara and Nikko. They followed up Japan with visits to Shanghai and Hong Kong, but due to the war between Japan and China, were unable to explore further on Avery Brundage's only visit to mainland China—this disappointment bothered him his whole life.

On his return to the United States after the June 1939 IOC session in London, Brundage systematically set about becoming a major collector of Asian art. The unsettled conditions caused wealthy Chinese to sell family heirlooms, and prices were depressed, making it an opportune moment to collect. He bought many books on Asian art, stating in an interview that a "major library is an indispensable tool". After the US entered World War II, stock owned by Japanese dealers in the United States was impounded; Brundage was able to purchase the best items. Dealers found him willing to spend money, but knowledgeable and a hard bargainer. Brundage rarely was fooled by forgeries, and was undeterred by the few he did buy, noting that in Asian art, fake pieces were often a thousand years old. In his 1948 article on Brundage for Life, Butterfield noted that "his collection is regarded as one of the largest and most important in private hands in this country".

Brundage engaged the French scholar René-Yvon Lefebvre d'Argencé, then teaching at the University of California, as full-time curator of his collection and advisor on acquisitions. The two men made a deal—no piece would be purchased unless both men agreed. They built a collection of jade which ranged from the neolithic period to the modern era; and hundreds of Chinese, Japanese and Korean bronzes, mostly Buddhas and Bodhisattvas. The painter whom Brundage admired the most was Huizong, 12th-century Chinese emperor of the Song dynasty; the collector never was able to obtain any of his work. Brundage several times bought pieces smuggled out of their lands of origin to restore them there. When Brundage sold a piece, it was most likely because he no longer favored it artistically, rather than to realize a profit. In 1954, a financial statement prepared for Brundage listed the value of his collection as more than $1 million. In 1960, Robert Shaplen, in his article on Brundage for The New Yorker, noted that Brundage, during his travels as IOC president, always found time to visit art dealers, and stated that the collection was valued at $15 million.

By the late 1950s, Brundage was increasingly concerned about what to do with his collection. His homes in Chicago and California were so overwhelmed with art that priceless artifacts were kept in shoeboxes under beds. In 1959, Brundage agreed to give part of his collection to the city of San Francisco. The following year city voters passed a bond issue of $2,725,000 to house the donation. The result was the Asian Art Museum of San Francisco, which opened in 1966 in Golden Gate Park, initially sharing space with the M. H. de Young Memorial Museum before moving to its own facility near the Civic Center in 2003. Brundage made another major donation in 1969 (despite a fire which destroyed many pieces at his California home, "La Piñeta" near Santa Barbara in 1964), and left the remainder of his collection to the museum in his will. Today, the museum has 7,700 pieces from Brundage among the 17,000-plus objects which make up its collection.

Brundage connected the world of art and that of amateur sports in his own mind. In a speech to the IOC session in Tokyo in 1958, he discussed netsuke, used at one time by Japanese men to anchor items, typically inro wallets, hung from kimono belts. Brundage owned several thousand netsuke, and held two in his hands as he spoke. He told the members that a netsuke was at one time carefully carved by the man who wore it, building "something of himself into the design", and although a class of professional netsuke makers arose later, whose work might have been more technically adept, it was "ordinarily cold, stiff, and without imagination. ... Missing was the element of the amateur carver, which causes these netsuke to be esteemed so much higher by the collector than the commercial product carved for money." Brundage later commented about his speech, "Here was the difference between amateurism and professionalism spelled out in a netsuke."

In May 2012, The Independent dubbed him "The ancient IOC emperor, anti-Semite and Nazi sympathiser bent on insulating the Games from the meddlesome tentacles of the real world." The Orange County Register stated that Brundage's "racism and anti-Semitism are well documented", and the New York Daily News averred that Brundage "admired Hitler and infamously replaced two Jewish sprinters on the 4-by-100 relay team because it could have further embarrassed Hitler if they won". David Miller described him as "despotic" in The Official History of the Olympic Games and the IOC (2012).

In 2021, San Francisco’s Asian Art Museum removed a bust of Brundage that had sat prominently in its foyer for five decades, which had been dedicated to him for his donating his sizable collection. Museum director Jay Xu wrote that Brundage "espoused racist and anti-Semitic views".

Writing for The Nation, Dave Zirin and Jules Boykoff criticized him for his controversial policies and statements, concluding, "Brundage’s 'contributions' to Olympic history need to be understood. But he has long forfeited a place of honor and respect".

Brundage, the only American and only non-European to serve as IOC president, left a mixed legacy. Guttmann notes that in the 1960s, Brundage may have been better-known as an art collector than for his sports activities, and "there are those who maintain that he will be remembered not for his career in sports but for his jades and bronzes." Andrew Leigh, a Member of the Australian House of Representatives, criticizes Brundage for expelling the two athletes in Mexico City, calling him "a man who'd had no difficulty with the Nazi salute being used in the 1936 Olympics". Dick Pound believes Brundage to have been one of the IOC's great presidents, along with de Coubertin and Samaranch, but concedes that by the end of his term, Brundage was out of touch with the world of sports. While Pound credits Brundage with holding the Olympic movement together in a period when it was beset by many challenges, he notes that this might not be fully appreciated by those who remember Brundage for the final years of his term, and for Munich.

Alfred Senn suggests that Brundage remained too long as IOC president:

After Munich, Brundage departed the Games, which had grown beyond his comprehension and his capacity to adjust. The NOCs and the [ISFs] were revolting against his arbitrary administration; violence had invaded his holy mountain and was giving every indication of returning; despite all his efforts to reach out to the world through athletics, he stood accused of bigotry and both race and class prejudice, not to mention the denunciations proclaiming him politically naive ... Few mourned his departure from the Olympic scene, and the International Olympic Committee turned to his successor, who, its members hoped, would be better suited to handle the new items on its agenda.

Books

• Espy, Richard (1981). The Politics of the Olympic Games. Berkeley: University of California Press. ISBN 978-0-520-04395-4.
• Guttmann, Allen (1984). The Games Must Go On: Avery Brundage and the Olympic Movement. New York: Columbia University Press. ISBN 978-0-231-05444-7.
• Hilton, Christopher (2008). Hitler's Olympics: The 1936 Berlin Olympic Games (revised ed.). Stroud, UK: Sutton Publishing Limited. ISBN 978-0-7509-4293-5.
• Large, David Clay (2007). Nazi Games: The Olympics of 1936. New York: W.W. Norton & Company. ISBN 978-0-393-05884-0.
• Maraniss, David (2008). Rome 1960: The Olympics That Changed the World. New York: Simon & Schuster. ISBN 978-1-4165-3407-5.
• Miller, David (2003). Athens to Athens: the Official History of the Olympic Games and the IOC, 1894–2004. Edinburgh: Mainstream Publishing. ISBN 978-1-84018-587-4.
• Pound, Dick (2004). Inside the Olympics. Mississauga, Ont.: John Wiley & Sons Canada Limited. ISBN 978-0-470-83454-1.
• Senn, Alfred (1999). Power, Politics, and the Olympic Games. Champaign, Ill.: Human Kinetics. ISBN 978-0-88011-958-0.

Other sources

• Brichford, Maynard (1994). "Avery Brundage: Money and Olympic Ideology" (PDF). London, Ont.: International Centre for Olympic Studies. pp. 61–66. Retrieved May 20, 2012.
• Brichford, Maynard (Winter 1998). "Avery Brundage: Chicago Businessman" (PDF). Journal of the Illinois State Historical Society. 91 (4). Springfield, Ill.: Illinois State Historical Society: 218–232. JSTOR 40193288. Archived from the original (PDF) on December 16, 2013.
• Busch, Noel (October 1968). "Avery Brundage: Olympian of Asian Art". Reader's Digest. Pleasantville, NY: Reader's Digest Association. Article reprint (from a press kit style folder in the Avery Brundage Collection, University of Illinois). Page numbers are for the reprint, not the original article.
• Butterfield, Roger (June 14, 1948). "Avery Brundage". Life. New York: Time, Inc. Retrieved May 9, 2012.
• Chan, Gerald (Autumn 1985). "The "Two-Chinas" Problem and the Olympic Formula". Pacific Affairs. 58 (3). Vancouver: University of British Columbia: 473–490. doi:10.2307/2759241. JSTOR 2759241.
• Ellis, Jack (September 7, 1972). "'Games must go on', says Brundage". Stars & Stripes. Retrieved May 20, 2012.
• Hinkley, David (May 1, 2012). "'Jesse Owens' on PBS eyes racism's high hurdles". New York Daily News. Archived from the original on February 8, 2013. Retrieved May 20, 2012.
• Johnson, William Oscar (August 4, 1980). "Avery Brundage: The Man Behind the Mask". Sports Illustrated. New York: Time, Inc. pp. 49–63. Archived from the original on September 19, 2020. Retrieved May 8, 2012.
• Lapchick, Richard E. (September 1979). "South Africa: Sport and Apartheid Politics". Annals of the American Academy of Political and Social Science. 445. Philadelphia: American Academy of Political and Social Sciences: 155–165. doi:10.1177/000271627944500116. JSTOR 1042963. S2CID 146606395.
• Lennartz, Karl (December 2001 – January 2002). "The 2nd International Olympic Games in Athens 1906" (PDF). Journal of Olympic History. Fochteloo, Netherlands: International Society of Olympic Historians: 10–27. Archived from the original (PDF) on May 15, 2012. Retrieved May 24, 2012.
• Marvin, Carolyn (April 1982). "Avery Brundage and American Participation in the 1936 Olympic Games". Journal of American Studies. 16 (1). Cambridge: Cambridge University Press: 81–105. doi:10.1017/s002187580000949x. JSTOR 27554090. S2CID 143749220.
• Reid, Scott M. (May 18, 2012). "Once again, IOC fails to exhibit courage". The Orange County Register. Retrieved May 25, 2012.
• Shaplen, Robert (July 23, 1960). "Amateur". The New Yorker. pp. 28–71.
• Steen, Rob (May 20, 2012). "John Carlos: A salutary lesson in the power of sport". The Independent. Retrieved May 20, 2012.
• Wallace, William N. (January 4, 2001). "Marty Glickman, announcer and blocked Olympian, 83". The New York Times. Retrieved May 12, 2012.
• "History". San Francisco: Asian Art Museum. Archived from the original on May 13, 2012. Retrieved May 20, 2012.
• "Brundage quits Olympic post". The Bend (Ore.) Bulletin. December 1, 1953. Retrieved May 20, 2012.
• "Avery: self-made millionaire and Olympic 'stuffed shirt'". Chicago Sun-Times. May 9, 1975. p. 73.
• "Athletic purist Brundage dead at 87". Chicago Sun-Times. May 9, 1975. p. 136.
• "Martin honoured with MacArthur Award". Southampton, U.K.: International Sailing Federation (ISAF). Retrieved May 24, 2012.
• "South Africa's long Olympic wait is over". Moscow-Pullman Daily News. July 23, 1992. Retrieved May 19, 2012.
• "Brundage re-elected unanimously". Sarasota Herald-Tribune. August 14, 1960. Retrieved May 19, 2012.

• Avery Brundage Collection at University of Illinois at Urbana-Champaign
• I.N.A.: Avery Brundage in opening ceremony, Grenoble, 1968 (in French)
• Transcript of Brundage press conference from 1970
• Avery Brundage's complicated Olympic legacy
• Avery Brundage at the USATF Hall of Fame (archived)
• Avery Brundage at the Team USA Hall of Fame (archive July 20, 2023)
• Avery Brundage at Olympics.com
• Avery Brundage at Olympedia (archive)