From the rear of the boat his captain, Mike Gansas, knew something was drastically wrong and tried to call Smith, first on his walkie-talkie, and then simply yelling his name "Richie" and shouting at him to correct the situation.
There was no response. One city official described the scene: "[Smith] is sitting down. He's unresponsive. [Gansas] goes over and tries to take control of the ferry and it hits the pier."
As the boat slammed into the harbour, killing 10 passengers and injuring dozens, Mr Smith suddenly became animated by the consequences of his inaction. Leaving his post so quickly that he forgot his house keys, he disappeared into the stampede of terrified passengers.
He made the three miles back to his home on the island, broke in and barricaded himself into the bathroom. There he slashed his wrists, and shot himself in the head and the chest with a pellet gun. At 4.20, exactly an hour after he had stared into space as the ferry he was supposed to be steering headed towards chaos and carnage, a man called emergency services and told them the boat's pilot had tried to kill himself.
Mr Smith, 55, was taken to St Vincent's hospital where he is in critical condition after surgery and under police custody. The pilot, whose house was guarded by police preventing reporters from speaking to his family on Wednesday night, gave initial comments to police but was not subjected to further questioning once he had reached the hospital.
It is not difficult to see Mr Smith's distress. Yesterday's was the deadliest transport accident in the city since 1928, when 16 people died following a subway derailment in Times Square.
Eddie McCabe, who lives on his block, said: "If you'd seen all the people that got killed or maimed how would you feel? That's gonna be on his mind for the rest of his life."
What is less clear is why an experienced ferry operator like Mr Smith would have been at the centre of such a tragedy. He had been commended twice for his work in his 15 years on the Staten Island ferry service and made the journey thousands of times. By all accounts he was a quiet family man whose interests in recent years had switched from antique cars to gardening and the piano.
"There's nothing in his record that we have seen so far that would indicate a problem," said city commissioner Iris Weinshall.
But on Wednesday something snapped. "There are conflicting reports," said one senior city official. "Did he pass out? Did he fall asleep? Is he just not there? We just don't know."
Investigators from the National Transportation Safety Board, who will be in charge of the inquiry into the crash, were trying to determine yesterday whether Mr Smith had been drinking or taking drugs or might have been incapacitated as a result of a medical condition.
The NTSB's chairwoman, Ellen Engleman, said investigators would be "looking into all records of the captains themselves, as well as the prior 72 hours in their lives". The inquiry could take a year. Meanwhile, ferry services resumed with passenger numbers already back to normal.
Witnesses said the boat did not appear to slow before it hit a maintenance pier, hundreds of feet from where the ferry normally docks.
"The scene was total chaos," said passenger Frank Corchado, of Staten Island, recounting a tableau of horrific sights: a decapitated man, a legless woman, a passenger bleeding from his eyes.
"There was a lady without legs, right in the middle of the boat," he said. "She was screaming. You ever see anything like that?"
New York's mayor, Michael Bloomberg, said most of the deaths apparently occurred when pilings ripped through the ship.
"What apparently happened is people who were sitting there as the ferry docked were hit by the pilings that came through the side of the boat," he said.
The dead, one woman and nine men, were aged from 25 to 52, police said.
Mr Gansas took over control of the ferry during the final moments, in a desperate attempt to steer it away from danger amid choppy waters and 45mph gusts. Were it not for his efforts, say some, the death toll would have been far higher.
"People are saying he's a hero but then he's saying, 'But 10 people are dead,' " said a relative of Mr Gansas.
The crew will to be interviewed and some have already been tested for drugs and alcohol, routine after major transport accidents.
Free service that never stops
· A public service since 1905, the Staten Island ferry carries over 19 million passengers annually between the St. George terminal in Staten Island and the Whitehall terminal in lower Manhattan, a distance of 5.2 miles. The service runs 24 hours a day, every day of the year, and is free.
· During the week five boats transport more than 65,000 passengers every day, making 104 trips a day. In the rush hour boats operate at 15-minute intervals. At the weekend three boats make 64 trips. More than 33,000 trips are made annually.
· The 25 minute journey passes the Statue of Liberty and Ellis Island. It is the only way for pedestrians to reach Staten Island.
· The ferry has not always been free. Between 1897 and 1972 passengers were charged 5 cents. The fare was increased to 10 cents in 1972, 25 cents in 1975, and 50 cents in 1990. It became free in July 1997.
Research: Isabelle Chevallot