THe term “second city” usually refers to a nation’s second most populous city. New York City has always been the first city after every United States census. From 1890 until 1990, Chicago was the second city in the United States. Los Angeles came close in 1980 and officially became the second most populous city after the 1990 census. Wikipedia has a list of “Largest cities in the United States by population by decade.”
 
The Oxford English Dictionary records the “second city” term in 1621—before the United States was formed.
 
“Chicago is now an important city—the second city of the lakes” (after Buffalo) was cited in print in 1852. “Chicago is the Second City” was a headline in the New York (NY) Times in August 1890.
 
American writer A. J. Liebling (1904-1963) wrote three essays for Thew New Yorker, with the first published on January 12, 1952 under the heading “Second City.” The essays were published under the title Chicago: The Second City (1952) and helped cement the “Second City” nickname. Chicago’s Second City theater troupe was founded in 1959.
   
Chicago’s “Second City” nickname was still used after 1990, although with less popularity.
     
 
Wikipedia: Chicago
Chicago (/ʃᵻˈkɑːɡoʊ/ or /ʃᵻˈkɔːɡoʊ/; shi-kah-goh) is the third most populous city in the United States. With over 2.7 million residents, it is the most populous city in the state of Illinois and the Midwestern United States, and the county seat of Cook County. The Chicago metropolitan area, often referred to as Chicagoland, has nearly 10 million people and is the third-largest in the U.S.
 
(Oxford English Dictionary)
second city, n.
The second largest city in a particular country or region; (also) a city which is often regarded as the second most important in a country or region.
Usually with in or of, except when second city (freq. with the and with capital initials in this context) has become something of a fixed name for a particular city within a given country, as, for example, Chicago (in the United States), Birmingham (in England), or Glasgow (in Scotland).
1621   J. Reynolds Triumphs Gods Revenge: 1st Bk. ii. 40   Pauia (the second City of the Dutchy of Millan).
1952   New Yorker 12 Jan. 29 (heading)  Second City… Chicago seems a big city instead of merely a large place.
 
Chronicling America
30 April 1852, The Republic (Washington, DC), pg. 1, col. 6:
General CASS said in the Senate Chamber,, in 1848: “It is now twenty-five years ago that I sat all night in a canoe at the head of a pond at Chicago, there being no human habitation in which we could obtain shelter from the mouth of the Illinois to the mouth of the Chicago river.” And yet, sir, Chicago is now an important city—the second city of the lakes.
 
Chronicling America
2 January 1890, Wichita (KS) Daily Eagle, pg. 4, col. 4:
Nevertheless Chicago is now the second city of the union, contending with New York for the world’s fair on the anniversary of the four hundredth century of the discovery of the continent by Columbus.
 
3 March 1890, Saginaw (MI) Evening News, “Chicago’s Position in the WOrld’s Fair,” pg. 2, col. 1:
“We want to show them that Chicago is the second city in the United States and I am confident that when the fair is held we will have as great a population as the census showed New York to have in 1880.”
(Spoken by Joseph Medill.—ed.)
 
Chronicling America
25 June 1890, Wheeling (WV) Daily Intelligencer, pg. 2, col. 2:
SINCE the census demonstrated that Chicago is the second city in the country in point of population, the windy city is happy and her rivals are correspondingly down in the mouth.
 
8 August 1890, New York (NY) Times, pg. 1, col. 7:
CHICAGO IS THE SECOND CITY.
WASHINGTON, Aug. 7.—He official rough count as announced by the Census Bureau to-day shows the population of Chicago to be 1,098,576. This is an increase during the decade of 595,391, or 118.32 per cent. This shows that Chicago has a population of about 53,682 in excess of Philadelphia, and is therefore the second city in population in the United States.
 
Chronicling America
11 August 1890, Rock Island (IL) Argus, pg. 2, col. 1:
The official count of the census bureau shows that Chicago’s population is 1,098,576, an increase of 595,391 in en years. Chicago is now the second city in population in the United States, having an excess of 53,682 over Philadelphia.
   
OCLC WorldCat record
History of Chicago, Illinois
Author: John Moses; Joseph Kirkland
Publisher: Chicago and New York, Munsell & Co., 1895.
Edition/Format:   Print book : English
Contents: v. 1. Pre-historic agencies ; Rise and fall of French dominion ; First permanent settlement ; The massacre ; Rudimentary—
The log cabin age ; From 1830 to 1837—
The balloon age ; From 1837 to 1849—
The period of collapse, stagnation and revival ; 1849-1860 ; 1860-1870 ; Chicago in the Civil War ; 1870-1880—
From destruction to reconstruction ; 1880-1890—
The second city ; 1891-1893 ; Trade and commerce ; Railroad history ; Banks and banking ; Real estate and insurance ; Biographical—
 
OCLC WorldCat record
Chicago’s record of accomplishment under Mayor Edward J. Kelly : the remarkable regeneration of America’ second city under its Democratic mayor.
Publisher: [1940]
Edition/Format:   Print book : Biography : English
     
The New Yorker
PROFILES JANUARY 12, 1952 ISSUE
SECOND CITY.
By A. J. Liebling
 
OCLC WorldCat record
Chicago: the second city ... Drawings by Steinberg.
Author: A J Liebling; Saul STEINBERG
Publisher: Alfred A. Knopf: New York, 1952.
Edition/Format:   Print book : English
 
Wikipedia: The Second City
The Second City is an improvisational comedy enterprise, best known as the first ever on-going improvisational theater troupe based in Chicago. It also has programs that run out of Toronto and Los Angeles. The Second City Theatre opened on December 16, 1959 and has since become one of the most influential and prolific comedy theatres in the world.
 
16 December 1962, The Sunday Courier and Press (Evansville, IN), “William the Conqueror” by Tom Fox, pg. 3-C, col. 2:
They’re calling Los Angeles the “New Second City” and if that doesn’t give Chicago the St. Vitus dance, then Walter O’Malley is St. Vincent de Paul.
 
27 June 1978, Newsday (Long Island, NY), “The best cities for women” by Warren Berry, pg. 9A, col. 3:
To a real New Yorker, Chicago will always be “the Second City.” Los Angeles is Queens with palm trees.
 
11 September 1980, Dallas (TX) Morning News, “Third World boosts LA into Second City,” pg. 37A, cols. 4-5:
Although Los Angeles has a long history of self-promotion and boosterism, the news that it has passed Chicago to become the nation’s Second City is not likely to bring much local applause: Many residents already believe they live in the Second City and have considered New York as their natural rival.
 
Chicago (IL) Reader
July 29, 1993
Why They Call It the Second City
A.J. Liebling and the Chicago He Knew and Hated
By J. Weintraub
(...)
In October 1949, A.J. Liebling leased an apartment with his second wife and her teenage daughter on the city’s near north side. Known then primarily for his New Yorker profiles, his war correspondence from North Africa and France, and his criticism of the press, Liebling had recently left his staff position at the New Yorker to write a series of articles on Henry Luce and Colonel Robert McCormick for Collier’s magazine. The assignment was a lucrative one, and Liebling hoped eventually to convert the material into a profitable book. But the owners of Collier’s apparently found the caustic tone of his first installment unsuitable for their audience, and terminated the contract before he got to McCormick. Family responsibilities forced Liebling to remain in Chicago until the summer of 1950, when he returned to the east coast to resume his position at the New Yorker. In January 1952 he published the first of three articles describing his experiences in and impressions of Chicago. Appearing in consecutive issues of the New Yorker, the articles were entitled “So Proud to Be Jammy-jammy,” “At Her Feet the Slain Deer,” and “The Massacree.”
     
Chicago (IL) Tribune
A.J. Liebling’s classic ‘Chicago: The Second City’ put Chicago in its place
By Patrick T. Reardon
MAY 3, 2016, 10:42 AM
A.J. Liebling, that caustic, sarcastic, witty New Yorker magazine writer, was no fan of Chicago as he made clear in his 1952 book, “Chicago: The Second City.”
(...)
The term “Second City,” although employed today as a name of honor by the city’s world-famous comedy troupe, was particularly galling in the 1950s because, throughout much of its history, Chicago had expected to grow into the largest and greatest city in the world. Those rosy predictions, however, had fizzled.