Armstrong was born on October 13, 1717, in Brookeborough, County Fermanagh, Ireland. He was educated there and became a civil engineer before emigrating to Pennsylvania. Armstrong came to Pennsylvania as a surveyor for the Penn family, who owned the colony. In 1750 he laid out the first plat or plan for the town of Carlisle, Pennsylvania, and was one of its first settlers. He was later appointed surveyor for the newly established Cumberland County.
During the French and Indian War, a combined force of Delaware (Lenape) Indians and Frenchmen attacked and sacked Fort Granville (near present-day Lewistown) in June 1756, taking a number of prisoners back along the Kittanning Path to the their fortified village of Kittanning on the Allegheny River (present-day Kittanning, Pennsylvania). Governor John Penn ordered provincial troops stationed in Cumberland County to respond. Colonel Armstrong led the Kittanning Expedition, a bold raid deep into hostile territory that destroyed Kittanning on 8 September 1756. The action earned Armstrong life-long fame as the "Hero of Kittanning."
In 1758, Colonel Armstrong led 2,700 Pennsylvania provincial troops on the Forbes Expedition, the approach of which compelled the French to vacate and blow up Fort Duquesne. Armstrong became a good friend to the other militia commander in this expedition, Colonel George Washington.
In the early stages of the Revolutionary War, Armstrong was a brigadier general in the Pennsylvania militia. On March 1, 1776, the Congress appointed him to that same rank in the Continental Army. He was sent south to begin preparations for the defense of Charleston, South Carolina. He contributed his engineering talents to the construction of defenses that enabled them to withstand the Siege of Charleston later that year. When General Charles Lee arrived to take command, he returned to his duties with the main army and with the Pennsylvania militia. Pennsylvania named him Major General in charge of the state militia. This ended his service in the Continental Army, but not the war or his cooperation with General Washington.
At the Battle of Brandywine on September 11, 1777, Armstrong's militia held the far left of the American line. They were also to guard the Army's supplies. After a hard day's fighting the Americans were forced to withdraw or face being surrounded. Armstrong brought the supplies and his militia out from Pyle's Ford after dark.
In the Battle of Germantown on October 2, General Armstrong led the American right. His mission was to skirt the British left flank and attack there and in their rear. Despite delays and the troubles some units had in moving, the overall attack was going well, until the center was held up at the Chew House. Then it collapsed after a fog inspired friendly fire incident in which General Adam Stephen's men fired on Anthony Wayne's troops causing their withdrawal. Armstrong, whose men had advanced nearly to the center of Germantown, but were not greatly involved in the fight later complained that it was "....a glorious victory fought for and eight tenths won, ....mysteriously lost, for to this moment no one man can ....give any good reason for the flight."
After Germantown, Armstrong was granted permission to give up active command. His health, at sixty, was not what it had been, and old wounds were troubling him. Returning home to Carlisle, he was elected to the Continental Congress by the Pennsylvania Assembly. As a delegate from 1777 to 1780 he was a strong supporter of Washington and the Army. Armstrong was firm in his support for a new United States Constitution, and was returned to the Congress during its final days in 1787 and 1788.
Throughout his life Armstrong served in a number of local or civic offices. One of these, the Carlisle school board, led him to originally oppose Dr. Benjamin Rush's proposal to start a college in the town. He later relented, and became a member of the first Board of Trustees for Dickinson College. John died at home in Carlisle, Pennsylvania, on March 9, 1795, and is buried in the Old Carlisle Cemetery. In 1800, when Pennsylvania created a new county at Kittanning, it was named Armstrong County in his honor. His son, John Armstrong, Jr. also served in the Army and the Congress.