Castle Williams, on Governors Island, was designed and erected between 1807 and 1811. It was called a “cheese box” (or “cheesebox”) in the 19th century.
 
“It (Castle Williams—ed.) looks like a tower, or like a cheese box that isn’t on a raft” was cited in print in 1889.
 
   
Wikipedia: Castle Williams
Castle Williams is a circular fortification of red sandstone on the northwest point of Governors Island, part of a system of forts designed and constructed in the early 19th century to protect New York City from naval attack. It is a prominent landmark in New York Harbor. Together with Fort Jay (Fort Columbus), it is managed by the National Park Service as part of Governors Island National Monument.
 
Chronicling America
23 August 1889, The Salt Lake Herald (Salt Lake City, UT), pg. 6, col. 2:
Castle Williams.
Castle Williams, not William, is on the western extremity of Governor’s island in New York harbor. It looks like a tower, or like a cheese box that isn’t on a raft.
 
4 April 1897, New York (NY) Times. “Fate of Castle Williams,” pg. 24, col. 2:
To the thousands of persons who pass up and down New York’s inner harbor daily, the picturesque old “cheese-box” shaped fortress of Castle Williams, situated at the northwest point of Governors Island, is more or less an object of curiosity.
 
26 June 1904, Cleveland (OH) Plain Dealer, “Old Defenses Now Useless: Governors Island Has Seen Great Changes in the Art of War.” pt. 3, pg. 1, col. 6:
“Then they turned their attention to Castle Williams, that big brick cheesebox that stood that stood on the corner of the island nearest to New York.”
 
Google Books
New York City Guide:
A comprehensive guide to the five boroughs of the metropolis: Manhattan, Brooklyn, the Bronx, Queens, and Richmond
By Federal Writers’ Project (N.Y.)
New York, NY: Random House
1939
Pg. 414:
CASTLE WILLIAMS, popularly known as “the cheese box” because of its circular shape, was begun in 1807 and completed in 1811 after the designs of Lieutenant-Colonel Jonathan Williams.
   
Google Books
The Landmarks of New York
By Barbaralee Diamonstein-Spielvogel
New York, NY: Abrams
1988  
Pg. 32:
In 1811 Castle Williams — originally called “The Tower” and nicknamed “The Cheesebox” — was erected. This impressive, circular red sandstone bastion was built to guard the waterway between Governors Island and New York City.
 
Google Books
Governors Island
By Susan L. Glen
Charleston, SC: Arcadia Pub.
2006
Pg. 10:
1811: Cattle Williams, dubbed “Cheesebox,” is completed.
 
Ephemeral New York
The Civil War prison in New York Harbor
September 21, 2009
(...)
Regular troops, however, went to Castle Williams—nicknamed the “Cheesebox” because of its circular design.
 
Google Books
AIA Guide to New York City
Fifth Edition
By Norval White, Elliot Willensky and Fran Leadon
New York, NY: Oxford University Press
2010
Pg. 950:
Appearing from the harbor to be fully circular in shape (hence its onetime nickname, The Cheesebox), this 200-foot-diameter red sandstone fortification is actually chevron-shaped in plan on its inland side.
(Castle Williams—ed.)