New York City pizzerias usually (though not always) sell pizza by the slice. The practice originated with the pizza itself, in Naples, Italy.
 
The origin of the square pizza is less clear, but see the citations below.
 
   
New Yawk Tawk:
A Dictionary of New York City Expressions 
by Robert Hendrickson
New York: Checkmark Books
1998
 
Pg. 148:
slice
Usually used by New Yorkers as shorthand for a slice-a (a piece of) pizza when ordering New York’s favorite fast food in a pizza parlor: “Gimme a slice anna small Coke.” Whether it’s a square cut of thick Sicilian or pie slice-shaped regular (Neopolitan) pizza, it’s always a slice. When a whole pizza is ordered it’s “Gimme a pie.” Piece is rarely heard.
 
 
May 1901, Century Illustrated Magazine, “Breakfast in Naples” by Mary Scott-Uda, pg. 18:
The famous pizzerie of Naples, some of which boast a hundred years of existence, are devoted exclusively to the manufacture and sale of a sort of rustic pie, or short-cake made out of risen dough, sharply beaten till quite thin, and seasoned on top with a great deal of lard, tomatoes, and grated cheese, or, on fast-days, with olive-oil, fresh anchovies, and a touch of garlic. The brisk tapping and slapping of the pizze can be heard a block away, and is as characteristic as the sonorous call of the sellers: “Have some breakfast! Have some breakfast!” You can buy a slice in the street from one of the runners, or, if you prefer, can enter the shop, stand by while your pizza is being vigorously thumped and slapped, can see it cooked in the glowing open oven under the fierce heat of a lateral fire of wood shavings, whisked out on an iron shovel in three minutes’ time, and served to you in popular style on a tin plate, all or three cents.
 
 
3 March 1941, Washington Post, “Ringside Talbe” with Mary Harris, pg. 6:
Pie is not pie in Ciro’s Famous Village. “It’s pizza, a pie that beats anything gramma ever made. It’s made of cheese, tomatoes and whatnots. Cut yourself a slice and be happy.
 
 
7 November 1952, Long Beach (CA) Independent, pg. 25 photo caption:
Montgomery Payoff, entered in the five-gaited division of the National Horse Show in New York, is served a bottle of beer by Trainer William Dearing, and a slice of pizza pie by Winnifred Freeman after his Madison Square Garden performance Thursday.
 
 
5 March 1955, Chicago Defender, pg. 16:
PIZZA PIE. Maybe this is a case of squaring the circle, but believe us, a square pizza pie tastes just as good as the traditional round variety, and it goes just as well with a mug of steaming coffee, too. Instead of the usual pizza dough, the shell is plain pastry from a packaged mix. The advantage of the square pan is that it is available inexpensively.
 
 
14 July 1956, New York Times, pg. 14:
Pizza is usually eaten in triangular slices with napkin between hand and the sizzling pastry. The apex of the triangle is most often the first part sacrificed to the yawning chasm which has replaced the once tight-lipped mouth and the base (to be more exact, the arc of the great circle) is saved for last. For of all the parts of the pizza pie, the crunchiest, and for many its crowning glory, is the outer part of the diameter known as “the crust.” Eating pizza calls for a certain peripatetic philosophy, the essence of which is nibbling on the run.
(...)
Consistently the shape of the pizza has been circular. Not square, nor rectangular, nor trapezoidal. There is no record of a pizzaiuolo ever having thrown a square pizza shell or any other shape for that matter except the circle, and so the circle has become a part of the pizza legerdemain and been accepted without embarrassing questions or geometrical speculations.
 

10 June 1957, New York Times, pg. 43:
Although there was no immediate count of those who bought $1.50 tickers for the lunch or how many bought slices of pizza for 15 cents, bottles of soda pop for the same price or bottles of wine for $1.25, it was agreed that the show surpassed all expectations.
 
 
6 December 1958, Chicago Defender, pg. 16:
Pizzas Are
Tasty, Good
Reducer
 
NEW YORK—Weight-watchers, take note—soon we’ll be able to reduce on pizza.
 
Vince Mastro, who does the world’s largest business in pizza equipment, is developing a new low-calorie dough for the pies.
(...)
He also originated the square pizza—50 per cent bigger than the round type.