Haifa East railway station was built in 1904 by the Ottoman authorities as the western terminus of their Jezreel Valley railway. In the 1920s, under the British Mandate, Palestine Railways extended the Eastern Railway north to connect with the Jezreel Valley railway at Haifa station. At this time, Haifa was the only place in the world to have direct railway services to three continents: Istanbul in Europe, Kantarah in Africa, and Medina in Asia. To this day, the distances on the Israel Railways main line are measured from Haifa East station.

As the first railway station in Haifa, it was originally named just "Haifa Station", and changed its name in 1937 when the new Haifa Center railway station opened.

Until July 1940, Haifa East railway station housed the head office of Palestine Railways; but in 1940, citing the unfavourable physical environment in the busy downtown and the physical vulnerability next to the port of Haifa in time of war, the administration started moving its branches into Khoury House at the intersection of Nevi'im st. and Khoury st. During the Battle of Haifa (1948), Khoury House was burnt down together with all records of Palestine Railways; due to this, the headquarters of the emerging Israel Railways were established at Haifa Center railway station.

The original station building at Haifa East was heavily damaged by an Irgun bombing on 20 September 1946, and only a small part of it is preserved. Currently, it houses some Israel Railways offices; the five-track former engine shed, situated across the tracks from the station building and built in c.1918, houses the Israel Railway Museum; and the station itself is used as a marshalling yard and a stabling point.