Cape May Point has the oldest population in the entire state. A new study of Census data by NJ.com shows the median age of a resident in Cape May Point is 67.8 years old, the oldest figure in the entire state. Stone Harbor is ninth on the list, and Avalon is tenth. The youngest town in New Jersey? New Brunswick city, where the median age is 23.1 years of age, thanks to the location of the state’s university.