The history of the Jews in Gaza City was intermittent, spanning from the second century BCE until the 1929 Palestine riots and the 1948 Arab–Israeli War. The Jewish community in the city produced rabbis and notable figures throughout its history. The Jewish presence in Gaza City was characterized by periods of coexistence, economic challenges, and occasional tensions with other communities.

During the Middle Ages, for around three centuries, the area of Gaza had a thriving Jewish community, until the Crusaders destruction of Gaza and its proximate cities in the 12th century. In the Ottoman period, under the leadership of Nathan of Gaza in the 17th century, Gaza became a centre for Jewish mysticism and the birthplace of the biggest modern Jewish Messianic movement. The events of the 1948 Arab–Israeli War marked a significant turning point, leading to the evacuation of the Jewish population from Gaza. The Israeli Jewish communities established in the Gaza Strip post-1967 Six-Day War, which were later evacuated in 2005, did not encompass a Jewish community within Gaza City itself.

Samson, the last of the judges of the ancient Israelites mentioned in the Book of Judges (chapters 13 to 16) traveled to Gaza, where he sees a prostitute (Hebrew: אִשָּׁ֣ה זוֹנָ֔ה) and visits her. His enemies wait at the gate of the city to ambush him, but he tears the gate from its very hinges and frame and carries it to "the hill that is in front of Hebron."

He was later captured by Philistines and taken to Gaza City to grind grain at a mill. After regaining his physical strength, he was taken to the temple of Dagon, where he brought down the columns, collapsing the temple, killing himself as well as all of the Philistines.

The final mention of Gaza in the Hebrew Bible was when king Hezekiah attacked Gaza and its outposts (II Kings 18:1-8). However, he was unable to conquer this city.

Gaza City, situated along the Mediterranean coast, was part of the Seleucid Empire during the Hellenistic period, and later came under Roman rule. During the Hellenistic period, which began with the conquests of Alexander the Great in the late 4th century BCE, there was a large Jewish population in nearby Judea, and Jewish communities also existed in other parts of the region. This period was characterized by the influence of Hellenistic culture and the interaction between different ethnic and religious groups. The existence of a Jewish community in Gaza is first mentioned during this time.

In 145 BCE, the Jewish Hasmonean leader Jonathan Apphus besieged Gaza while serving in the army of the Seleucid king Antiochus VI. He burned the fields, and the people of the city surrendered. Gaza remained hostile to the Hasmoneans until its destruction by the Hasmonean king Alexander Jannaeus in 96 BCE. Gaza was rebuilt by Roman General Pompey Magnus, and granted to Herod the Great thirty years later. Throughout the Roman period, Gaza continued to prosper, receiving grants from several Roman emperors. A 500-member senate governed the city, which then had a diverse population of Greeks, Romans, Jews, Egyptians, Persians and Nabateans. During the reign of Herod (37–4 BCE), Gaza was under his authority. After his death in 4 BCE, it was annexed to the province of Syria. In 66 CE, during the early phases of the First Jewish–Roman War, the city was attacked by Jewish rebels and  was burned down during their rebellion against the Romans.

On one of the pillars in the Great Mosque of Gaza, an inscription in both Hebrew and Greek read "Hananiah son of Jacob." Above it was carved a menorah with a shofar on one side and an etrog on the other. During the late 19th century, the column was part of an old synagogue in Caesarea Maritima and was brought to the mosque for its perceived religious value, as the Caesarea church had appeared in the Madaba Map. The fact that this Jewish symbol was preserved throughout the decades inside the mosque was described as demonstrating "peaceful coexistence" by scholar Ziad Shehada. It was heavily damaged in February 1987 during the First Intifada.

According to all rabbinic opinions, the Gaza Strip is included in the borders that ‘olei Mitzrayim’ were commanded to conquer in the times of Joshua, and it was included in the inheritance of Judah, as it is written: “And Gaza with its surrounding towns and settlements, as far as ‘Nachal Mitzrayim’ and the border at the Mediterranean Sea” (Joshua 15:47).

Towards the end of the Byzantine period, Gaza was an important Jewish center in southern Palestine. Gaza, together with Tiberias, became an alternative pilgrimage site for Jews when Jewish access to Jerusalem was forbidden. According to a Karaite source, during this period, "before the Arab conquest of the Land of Israel, they could not come to Jerusalem, and from the four corners of the land, they came to Tiberias and Gaza for the desire of the Temple." This settlement continued until the end of the Byzantine period, as indicated by the Arab conquest of Gaza and its vicinity (in February 634, after the siege of the city's surroundings, it was besieged, culminating in the capture of the city in July) in a Syriac chronicle: "And about four thousand poor peasants from the Land of Israel were killed there: Christians, Jews, and Samaritans. And the Arabs devastated the entire land."

Near the port of Gaza, remnants of the ancient synagogue of southern Gaza were discovered, built around the year 508 CE during the Byzantine period. On the mosaic floor of the synagogue, various African animals are depicted, along with medallions, and at the entrance, there is a portrayal of a figure playing a musical instrument surrounded by animals, with the name "David" above. The central inscription on the mosaic in Greek reads: "We, Menahem and Yeshua, sons of Jesse, wood merchants, as a mark of respect for the holiest site, donated this mosaic in the month of Louos, year 569" (the Jewish counting of the year in Gaza began with the expulsion of Gabinus). The community of Gaza is also mentioned in the Cairo Geniza documents.

The area of Gaza including the city of Rafah had a thriving Jewish community for nearly 300 years during the Middle Ages, with a number of Jewish relics such as letters and correspondence by the community members later being found in the Cairo Genizah, attesting to what is described as a "flourishing" Jewish presence until the 12th century with the Crusaders destruction of Gaza's area.

With the rise of Islam in the 7th century, Gaza was conquered by the army of the Rashidun Caliphate. The conquests were characterized by a certain level of religious tolerance, allowing Jews and Christians to practice their religions under the conditions of dhimmi, a protected status for non-Muslims.

Gaza became part of the Umayyad and later the Abbasid Caliphates. During the Arab conquest, the Arab conquerors did not expel the Jewish and Christian residents of the city, and throughout the entire Islamic rule, the Jewish settlement in Gaza persisted continuously until the mid-11th century when the city was devastated.

Notable figures from this period include Rabbi Moshe al-Azati the punctuator (circa 800–825) and Rabbi Ephraim ben Shemariah al-Azati, leaders of the Jewish community in Fustat, Egypt, who originated from Gaza. Correspondence from this time between the sages of Fustat and those of Gaza survives, addressing inheritance disputes at the heart of a legal dispute between them. A letter signed by fifteen leaders of the city, headed by Yeshua ben Nathan, "Yeshua, the member of the Great Sanhedrin, son of Rabbi Nathan, descendant of the Lion Cub," attributed to leaders from the tribe of Judah, is a part of this correspondence. A manuscript discovered in the Cairo Geniza testifies to a dispute between David ben Daniel from the prominent Jewish community of Babylonia and the Gaon family concerning whether Gaza and Ashkelon were included in the borders of the Land of Israel. This dispute indicates the existence of a Jewish community in Gaza during this period.

Uncertainty exists regarding the existence of small Jewish settlements around Gaza in the early second millennium. According to Shmuel Asaf, dispersed Jews around Gaza during this period were forced to move into the city. However, according to Benjamin Zeev Kedar, this assumption lacks sufficient evidence. A piyyut (liturgical poem) from that time, written by the poet Rabbi Shmuel ben Yerubi Haushana, refers to a decree that affected the Jews in the region during the same period: "And the sons of Gaza were cut off, and the congregation of its courtyards was driven away." The version "courtyards" supports the existence of a Jewish settlement around Gaza. However, a parallel version reads, "and the congregation of its courtyards (ḥatzerim) was driven away," and "ḥatzerim" is an ancient Hebrew term for Rafah, suggesting that the poet may have referred to the Jews of Rafah and did not address the existence of a Jewish settlement around Gaza, raising doubts about its existence.

In 1077, Atsiz ibn Uvaq, suppressing a rebellion against the Seljuks, captured Gaza and completely destroyed the city. The population was only renewed after two generations. Jewish travelers who visited the Land of Israel during this period, such as Benjamin of Tudela and Petachiah of Regensburg, do not mention Gaza at all. Even Yehuda Alharizi, who passed through Gaza in 1218 on his way from Egypt to Jerusalem, does not mention encountering Jews in the city.

With the Crusader conquest and the establishment of the Kingdom of Jerusalem in the early 12th century, the residents of Gaza fled their city, and it was handed over to the order of the Knights Templar, who resettled it. There is no information about Jews in the city during that period, leading to the assumption that their fate was similar to that of the other Muslim residents. The Jewish settlement in Gaza ceased to exist during this time until the Mamluk period.

In the 13th century, Rabbi Avraham Abulafia mentioned "the Torah commentary composed by Rabbi Tzedaka Halevi from the city of Gaza." However, there is a scholarly debate about whether this indicates a Jewish settlement in the city during that period.

With the Mamluk conquest, Jews returned to Gaza, and a thriving Jewish community emerged in the city. In 1384, a Christian Italian traveller named Goti reported on a Jewish settlement in Gaza, noting that the Jews were engaged in wine production, stating, "and their wine is good." Another Christian traveller, De Angeli, who visited the city in 1395, described the division of the city into neighbourhoods or streets, with Jews residing in one street and Samaritans in another, and so forth. To distinguish between residents of different faiths, the inhabitants of Gaza were required to wear head coverings in different colours: Muslims wore white turbans, Christians wore turbans in a blueish colour, Samaritans wore white turbans with a light red shade, and Jews wore yellow head coverings.

Around 1432, the Burgundian spy Bertrandon de la Brocquière reported encountering Sicilian Jews in Gaza. While this mention alone does not confirm a constant Jewish presence in the city at the beginning of the 15th century, it seems that a consistent Jewish presence persisted, as indicated by references from the latter half of that century.

A Jewish traveller from Florence, Solomon Molcho, described Gaza in 1481 during his journey to the Holy Land, noting a thriving city with good land, cattle, bread, and wine produced by the Jews. He reported a large population, including around 70 Jewish families and four Samaritan families. According to his account, Jews resided in the upper part of the city, referred to as the "Yehudika" (Jewish quarter), with the house of Delilah located there, associated with the biblical Samson. He also observed a large courtyard, presumably destroyed, that he considered significant even in his time. Meshullam of Volterra (1481) reported 60 Rabbinite and four Samaritan Jewish households, and noted that the Jews were responsible for cultivating the wine in the region. Obadiah of Bertinoro (1488) noted 70 Rabbinite and two Samaritan households, and mentions the presence in Gaza of a Rabbi Moses of Prague, who had moved there from Jerusalem.

In the 16th century, there is evidence of a Jewish population in Gaza, likely larger than in the 15th century. According to Ottoman tax records from 1525 to 1526, 95 Jewish families lived in Gaza during that year.

In the early 17th century, the Jewish community in Gaza was served by renowned Kabbalist and poet Rabbi Israel Najara until his death and burial in Gaza's Jewish cemetery in 1625. He was succeeded as the community's rabbi by his son, Rabbi Moshe ben Israel Najara. Samuel ben David founded a synagogue in the city in 1641.

In the 1660s Gaza became a hub for the mystical Jewish Kabbalah teachings and the place where the worldwide Sabbateans Jewish Messianic movement was born, under the leadership of Nathan of Gaza, who moved from Jerusalem to live with his wife's family among Gaza's Jewish community. Nathan's move to Gaza was the turning point in his awareness of "adhering to an ascetic lifestyle, dabbling in mysticism and delving further" into the writings of rabbinic interpreters; after being well versed in the Torah interpretations as well as Jewish commentary since childhood in Jerusalem. In Gaza Nathan was quickly surrounded by young apostles due to his "persistence, daring and original theology, eloquent expression and social skill", with the apostles described as "fervently singing" the age-old piyyutim liturgical poems by Gaza's figure Rabbi Israel Najara and "dancing ecstatically" on Gaza's beach. Nathan's teachings at the seashore of Gaza were described by an apostle as "“The Prophet’s” penchant for preaching the secrets" of the Zohar, the fundamental book of Jewish mysticism. During the Jewish holiday of Shavuot in the 1660s, Sabbatai Zevi travelled from Jerusalem to Gaza to receive a healing penance by Nathan, who proclaimed him as the Messiah, founding the biggest Jewish Messianic movement in the modern era.

In 1674, a Jesuit priest who visited Gaza reported that Jews comprised a quarter of the city's population. Another Christian priest who arrived in Gaza in 1726 mentioned a Jewish community engaged in trade and serving as interpreters.

In February 1799, with the French forces led by Napoleon capturing the city, and the emergence of a plague, most of the Jewish community fled from Gaza.

By the early 19th century, little remained of the Jewish community, with only 90 Jews recorded in 1903. In 1869, two families of Moroccan Jewish origin settled in Gaza, marking the initial core of the new Jewish settlement in the city. During the First Aliyah (1880s), attempts were made to renew the Jewish population in Gaza. As part of these efforts, Zalman David Levontin visited the city in 1882 and reported that he did not encounter any Jews. He proposed establishing the Jewish settlement in the Ali Montar area.

In 1885, Zev Klonimus Wissotzky, a leader of the Ḥovevei Zion movement, called for the establishment of Jewish settlements in the heart of Arab cities, including Gaza, Lod, and Nablus, to provide an additional source of livelihood for immigrants, in addition to the agricultural settlements that were established at the time. In 1886, Yechiel Brill [he] reported that there were already 50 Jewish families residing in Gaza. Members of the renewed community mainly came from the old Jewish community of Jerusalem, primarily from the Sephardic community, alongside some Russian Jewish immigrants.

The synagogue that served the local community in the 18th century, if it was in an independent building, either collapsed or came under Muslim control. The renewed community initially conducted its prayers in an apartment designated for this purpose. By 1895, a small Jewish community remained in the city, consisting of about twelve families who maintained excellent relations with the Muslim population.

Starting in 1907, Rabbi Nissim Ohana served as the rabbi of the Gaza community. He established a school for studying the Tanakh and Talmud in Hebrew and initiated the construction of a mikveh for women, as well as the establishment of a cemetery.

During World War I, many Jews from Gaza were expelled, and some were even deported from the Land of Israel. Prior to this wave of displacement, a significant portion of Gaza's remaining Jewish population was involved in trade, and they held a monopoly on exporting watermelons ("khandol") through the port of Gaza to the port of Hamburg. The watermelon crop was cultivated by Bedouins in the Negev sands and also served as a popular remedy for constipation. Another agricultural branch, the cultivation of barley for the brewing industry, attracted many Jewish families until 1916, the year in which Ottoman and German forces evacuated all residents of the area in preparation for the confrontation with British and Allied forces. After the war, Jews returned to Gaza (approximately 50 Jews were registered in 1922), but they were forced to leave again due to the 1929 Palestine riots.

In the 1945 village survey, there were 150 Jews counted in the Gaza district, with 80 in the city itself. However, after the 1948 Palestine war no Jewish settlement remained in the city. Gaza City was briefly under Israeli military administration led by Lt. Col. Haim Gaon during the Suez Crisis between November 2-3, 1956 and March 7, 1957.

Following the 1967 Six-Day War, Israel occupied the Gaza Strip, and new Israeli settlements, deemed illegal under international law, were constructed in the Gaza District, including Gush Katif, Kfar Darom, and Netzarim, and the northern settlements of Elei Sinai, Nisanit, and Dugit. However, there was no Jewish settlement within Gaza City.

In 1987, Israeli architects Gershon Tzapor and Benjamin Edelson designed the expansion of Al-Shifa Hospital. The expansion and renovation aimed to improve the living conditions of Gaza residents.

After the 2005 Israeli disengagement from Gaza, Israel closed off the Gaza Strip to Israelis. (Egypt has mostly kept its crossing closed and argues Israel must decide who may enter).

Israeli troops invaded and occupied Gaza City during the 2023 Israel–Hamas war in which they temporarily installed menorahs for the Hanukkah holiday in the Shuja'iyya neighborhood and in the nearby city of Beit Hanoun.