The term "forgotten borough" cannot be earlier than 1898 (when the city unified as five boroughs). However, the term seems to be an early and persistent one, and has been cited in print since at least 1922.
In 2018, Staten Island officially branded itself as "The Unexpected Borough."
Wikipedia: Staten Island
Staten Island (/ˌstætən ˈaɪlənd/) is one of the five boroughs of New York City, in the U.S. state of New York. Located in the southwest portion of the city, the borough is separated from New Jersey by the Arthur Kill and the Kill Van Kull and from the rest of New York by New York Bay. With an estimated population of 479,458 in 2017, Staten Island is the least populated of the boroughs but is the third-largest in land area at 58.5 sq mi (152 km2). The borough also contains the southern-most point in the state, South Point.
5 September 1922, The Evening World (New York, NY), "What Did You See To-day?," pg. 18, col. 7:
IS RICHMOND A "FORGOTTEN" BOROUGH?
Where will you find such service as this except in New York City's forgotten borough of Staten Island? -- Maurice Vogt, Iona Street, Graham Beach, Staten Island.
15 July 1928, New York (NY) Times, pg. RE1:
Urging Staten Island operators to be cautious about pricing their realty, W. Burke Harmon, President of the Harmon National Real Estate Corporation, yesterday declared that sudden price increases on properties at this time might well result in halting the normal development of what he calls "this forgotten borough that has suddenly stepped into the limelight."
1 November 1933, Daily News (New York, NY), "Around the Clock with Candidates" by George Dixon, pg. 4, col. 5:
Staten Island, the forgotten borough, turned out in its best bib and tucker last night to welcome the first visit from a major candidate for Mayor -- John P. O'Brien.
19 October 1950, New York (NY) Times, pg. 35:
Speaking at a borough-wide rally last night in the Boulevard Hotel, Grant CIty, S. I., Mr. Corsi described Staten Island as the "forgotten borough" by the present Tammany administration at City Hall.
(Edward Corsi, Republican candidate for Mayor - ed.)
23 November 1958, New York (NY) Times, pg. R1:
Staten Island rates high as the possible scene of New York's next boom in industrial construction, according to city and Chamber of Commerce officials. The fact that the commerce group's members come largely from Richmond is not expected to lengthen the odds against "the forgotten borough," as some of them have named it.
26 March 1996, New York (NY) Times, pg. B3:
Staten Islanders and their political leaders say the landfill is smellier than ever and offers conclusive proof that they are being treated as the city's forgotten borough.
OCLC WorldCat record
Staten Island slayings : murderers & mysteries of the forgotten borough
Author: Patricia M Salmon
Publisher: Charleston : The History Press, 2014.
Edition/Format: eBook : Document : English
Staten Island saw its share of violence and murder as it transformed from a sleepy community to an urban outer borough. The 1920 discovery of a woman's body by two young boys walking their dog remains unsolved. An inmate at Sailors' Snug Harbor--a retirement home for seamen--shot a preacher in cold blood. Shocking and horrific stories of killers and their victims such as these plague Staten Island's otherwise pleasant past. From the handsome soldier convicted of his Russian wife's shooting in New Dorp Beach to the New Brighton guard beaten to death while protecting seized whiskey during Prohibition, local historian Patricia Salmon uncovers Staten Island's most chilling tales of infamous and long-forgotten acts of violence.
OCLC WorldCat record
Mass Transit, New Towns, and National Parks in New York City's 'Forgotten Borough'
Author: Patrick Nugent
Publisher: Rockefeller Archive Center 2016-01-01
Edition/Format: Downloadable archival material : English
My dissertation insists that Staten Island is central not only to the history of twentieth-century New York City, but to postwar urban planning and politics more broadly. Despite its absence from almost every major historical work on the postwar urban crisis, the borough was considered by many planners and politicians to be New York City's greatest asset and most volatile liability. Set against the rest of the boroughs' declining populations and shrinking tax revenues, Staten Island's large swaths of vacant acres provided a blank slate onto which urbanists mapped their conflicting critiques and cure-alls for the American city. Amongst a long list of influential politicians, environmentalists, and planning organizations that debated the future of Staten Island, Governor Nelson Rockefeller and the Regional Plan Association (RPA) stand out as having particular interest in and influence on the borough. My research at the Rockefeller Archive Center was integral to tracking and contextualizing both Rockefeller's and the RPA's planning approaches to the "forgotten borough" -- philosophies which by the early 1970s had come into tension with one another. While Rockefeller would move, in the late 1960s, toward encouraging a dense, socially-diverse, mass-transit oriented Staten Island, the RPA's uncompromising support of Gateway National Recreation Area would forestall precisely the type of new town planning project -- the South Richmond Development Corporation designed by James Rouse -- that was capable of densifying and integrating the borough's overwhelmingly white, middle-class southern shore.