According to neighbours, the violence erupted from a domestic dispute in a small, white ranch house near the Bayonne Bridge.
A fifth person was rushed to Bayonne Medical Center and was being treated for non-life-threatening injuries.
The suspect is believed to have first fired at two adults, then the infant, and then turned the gun on himself.
“It is a tragedy and a senseless loss of life for the family and the community as a whole,” Mayor Mark Smith said.
The mayor said two other children were in the home at the time of the shooting, but were unharmed.
The shootings occurred around 8:00 p.m. When police turned up a woman was outside the house suffering from a gunshot wound.
The woman spoke little English but was able to convey to the officers that there were more people shot inside the home, officials said.
Police then entered the house and found a man and a woman shot dead.
According to officials, a boy, who was aged one to one-and-a-half years old, had also been shot and was rushed to the hospital, but did not survive.
Police have not identified the victims.
It was not clear whether he was dead when police arrived or whether he shot himself after police had entered the home.
Around 15 to 20 shots were fired, and police believe a gun recovered at the scene was the one used in the killings.
Around the neighbourhood, stunned residents were trying to get a handle on what had happened.
Has Jones, 30, said he heard the gunfire: “I heard four shots and then it stopped, and then I heard screaming and then three more shots.”
Has Jones, who lives about 50ft from the house where the shooting took place, said the person screaming was a woman.
Nicole Poesl, 33, who walked several blocks from her home to the scene tonight, said she saw a police officer exit the house, cradling a young girl in his arms.
She said she asked police: “What’s going on? Murder-suicide?” and an officer shook his head, “yes”.
Some residents said a “lady with two kids” lived in the home.
At least 158 people have died in Spain's worst flooding disaster in generations. On October…
Google has been fined two undecillion (a two followed by 36 zeroes) roubles by a…
Embarking on a home remodel is an exciting journey, promising enhanced comfort, increased property value,…
The US presidential candidates continued to campaign across key swing states on October 20. Footage…
Elon Musk has said he will give away $1 million a day to a registered…
India and Canada have decided to expel their top envoys along with other diplomats as…