As of the census of 2000, there were 3,686 people, 1,306 households, and 1,000 families residing in the town. The population density was 90.8 people per square mile (35.1/km²). There were 1,366 housing units at an average density of 33.6 per square mile (13.0/km²). The racial makeup of the town was 97.86% White, 0.38% African American, 0.54% Native American, 0.19% Asian, 0.05% from other races, and 0.98% from two or more races.  Hispanic or Latino of any race were 0.49% of the population.

There were 1,306 households out of which 38.4% had children under the age of 18 living with them, 63.5% were Married couples living together, 7.9% had a female householder with no husband present, and 23.4% were non-families. 16.0% of all households were made up of individuals and 5.3% had someone living alone who was 65 years of age or older. The average household size was 2.82 and the average family size was 3.14.

In the town the population was spread out with 29.3% under the age of 18, 7.4% from 18 to 24, 32.9% from 25 to 44, 22.2% from 45 to 64, and 8.2% who were 65 years of age or older. The median age was 35 years. For every 100 females there were 102.5 males. For every 100 females age 18 and over, there were 103.0 males.

The median income for a household in the town was $43,170, and the median income for a family was $46,190. Males had a median income of $30,104 versus $25,150 for females. The per capita income for the town was $17,229. About 5.3% of families and 6.9% of the population were below the poverty line, including 8.7% of those under age 18 and 2.0% of those age 65 or over.

Communities and locations in Palermo

• Catfish – A hamlet in the northeast part of the town.
  
• Clifford – A hamlet north of Palermo village. It was once called "Dentons Corners."
• Cribbs Corners – A hamlet northeast of Palermo village on State Route 3.
  
• East Palermo – A hamlet east of Palermo village.
• Farley Corners – A location at the west town line.
  
• Loomis Corner – A hamlet south of Palermo village on State Route 264.
  
• Mungers Corners – A hamlet northeast of Palermo village.
• Palermo – The hamlet of Palermo is on Route 3. It is also called "Palermo Center" and was once known as "Jennings Corners."
• Peat Corners – A hamlet south of East Palermo.
• Russ Mills – A hamlet north of East Palermo.
• Russ Pond – A small lake north of Russ Mills.
• Sayles Corners – A location in the northwest part of the town.
• Sutton Corner – A hamlet near the south town line at Routes 49 and 264.
• Upson Corner – A hamlet north of East Palermo.
• Vermillion – A hamlet at the north town line on Catfish Creek.