The term "forgotten borough" cannot be earlier than 1898 (when the city unified as five boroughs). However, the term seems to be an early and persistent one.
15 July 1928, New York Times, pg. RE1:
Urging Staten Island operators to be cautious about pricing their realty, W. Burke Harmon, President of the Harmon National Real Estate Corporation, yesterday declared that sudden price increases on properties at this time might well result in halting the normal development of what he calls "this forgotten borough that has suddenly stepped into the limelight."
19 October 1950, New York Times, pg. 35:
Speaking at a borough-wide rally last night in the Boulevard Hotel, Grant CIty, S. I., Mr. Corsi described Staten Island as the "forgotten borough" by the present Tammany administration at City Hall.
(Edward Corsi, Republican candidate for Mayor - ed.)
23 November 1958, New York Times, pg. R1:
Staten Island rates high as the possible scene of New York's next boom in industrial construction, according to city and Chamber of Commerce officials. The fact that the commerce group's members come largely from Richmond is not expected to lengthen the odds against "the forgotten borough," as some of them have named it.
26 March 1996, New York Times, pg. B3:
Staten Islanders and their political leaders say the landfill is smellier than ever and offers conclusive proof that they are being treated as the city's forgotten borough.