Stephen Harper says staffers such as Ray Novak enjoy his confidence

Stephen Harper says the people who work for him enjoy his confidence, otherwise they wouldn't be on his staff.

That was the prime minister's answer when he was asked if he stands by chief of staff, Ray Novak.

Recent testimony in the Mike Duffy fraud trial has more closely linked Mr. Novak to the controversial $90,000 payment to Duffy in 2013 by Harper's previous chief of staff, Nigel Wright.

Mr. Harper reiterated Thursday during a campaign stop in King Township, Ont. north of Toronto that he holds only Mr. Duffy and Mr. Wright responsible for any wrongdoing.

"I've been very clear. When people are working for me, they have my confidence. If they didn't have my confidence, they wouldn't be working for me. And in this case I have held the two people accountable who are responsible," Harper said.

"And I'm not going to go around holding everybody else accountable for their actions."

The revelations from the Duffy trial have forced Mr. Harper to answer questions about the affair for the last week.

On Thursday, Mr. Harper said his party has moved to bounce questionable members of the upper chamber from his team.

In response to a question on leadership, Mr. Harper said senators appointed by various prime ministers have encountered difficulties because the Senate is an "unaccountable" institution.

Mr. Harper said the difference is that Conservatives have ensured there is accountability.

"We have every right to expect better," Mr. Harper said. "I have a right to expect better. "

His two main rivals, NDP Leader Thomas Mulcair and Liberal Leader Justin Trudeau, are campaigning in British Columbia later in the day.