According to neighbours, the violence erupted from a domestic dispute in a small, white ranch house near the Bayonne Bridge.
A fifth person was rushed to Bayonne Medical Center and was being treated for non-life-threatening injuries.
The suspect is believed to have first fired at two adults, then the infant, and then turned the gun on himself.
“It is a tragedy and a senseless loss of life for the family and the community as a whole,” Mayor Mark Smith said.
The mayor said two other children were in the home at the time of the shooting, but were unharmed.
The shootings occurred around 8:00 p.m. When police turned up a woman was outside the house suffering from a gunshot wound.
The woman spoke little English but was able to convey to the officers that there were more people shot inside the home, officials said.
Police then entered the house and found a man and a woman shot dead.
According to officials, a boy, who was aged one to one-and-a-half years old, had also been shot and was rushed to the hospital, but did not survive.
Police have not identified the victims.
It was not clear whether he was dead when police arrived or whether he shot himself after police had entered the home.
Around 15 to 20 shots were fired, and police believe a gun recovered at the scene was the one used in the killings.
Around the neighbourhood, stunned residents were trying to get a handle on what had happened.
Has Jones, 30, said he heard the gunfire: “I heard four shots and then it stopped, and then I heard screaming and then three more shots.”
Has Jones, who lives about 50ft from the house where the shooting took place, said the person screaming was a woman.
Nicole Poesl, 33, who walked several blocks from her home to the scene tonight, said she saw a police officer exit the house, cradling a young girl in his arms.
She said she asked police: “What’s going on? Murder-suicide?” and an officer shook his head, “yes”.
Some residents said a “lady with two kids” lived in the home.
The US House Ethics Committee has voted to release its report on former Republican Representative…
ABC News has agreed to pay $15 million to President-elect Donald Trump to settle a…
South Korea’s parliament has voted to impeach President Yoon Suk Yeol over his failed attempt…
Israeli war planes have carried out more than 100 air strikes in Syria on December…
President-elect Donald Trump has threatened to impose 100% tariffs on the BRICS countries if they…
Syrian troops have withdrawn from the city of Aleppo following an offensive by rebels opposed…