Mr Hammond coordinated a show of strength from Tory Remainers earlier this week as he attacked Boris Johnson's Brexit strategy and vowed to fight the UK leaving the EU without an agreement.
But his outspoken opposition to No Deal has reportedly gone down badly with some of the members of the Conservative Association in his Runnymede and Weybridge seat.
Party insiders told The Telegraph that many activists were 'not best pleased' at Mr Hammond working against Mr Johnson and there was 'no doubt' tensions would boil over in the near future.
It came as senior allies of Mr Johnson savaged Mr Hammond and accused him of being 'patronising and misleading'.
Iain Duncan Smith, who served as Mr Johnson's campaign chairman during the Tory leadership race, said Mr Hammond was talking 'utter nonsense' over his attempts to interpret what people were voting for at the EU referendum in 2016.
Mr Hammond made his first major public intervention earlier this week after he quit as chancellor when Mr Johnson took office last month.
He penned a letter signed by 20 of his Tory colleagues in which he accused Mr Johnson the Prime Minister of torpedoing any chance of a Brexit deal because of his hardline negotiating demands.
The letter suggested the PM had 'set the bar so high that there is no realistic' chance of a new divorce agreement being struck with Brussels.
Meanwhile, in a separate newspaper column for The Times the former chancellor took aim at Mr Johnson's top aide Dominic Cummings who he accused of 'pulling the strings' to steer the government towards a No Deal split from the EU.
Philip Hammond penned a letter to Boris Johnson earlier this week in which he accused the PM of making a new Brexit deal with the EU impossible
Iain Duncan Smith, pictured on August 13 in Downing Street, said Mr Hammond was talking 'utter nonsense'
One source said that it was 'no secret' some activists were angry at Mr Hammond because of his Brexit 'interference'.
An insider said: 'There are people in the association in the executive committee, who are not best pleased with some of the things that Philip has said.'
Another source said a 'reasonably sized group' of hard Brexiteers were 'not too chuffed' with Mr Hammond and there was 'no doubt they will be expressing those opinions during the course of September'.
While the result of such a vote would not be binding on Mr Hammond it would be hugely embarrassing for him if he was to lose one.
Mr Hammond's intervention this week sparked a furious Tory Brexiteer backlash which continued today as Mr Duncan Smith took aim at the ex-chancellor.
Writing in The Telegraph, the Tory former leader said: 'Other Remainers have intoned similarly (and endlessly) that no one voted to lose their jobs or to be poorer, as though they had some incredible insight into not just what motivated those who voted Leave, but also into the consequences of our departure.
'This is, of course, utter nonsense. The only thing we know the majority of people didn't vote for was to remain in the EU.
The 20 Tory MPs who signed Philip Hammond's anti-No Deal letter to Boris Johnson
Philip Hammond's letter to Boris Johnson attacking the PM's Brexit strategy was signed by 20 of his Tory colleagues.
These are the Tory MPs who backed Mr Hammond:
Sir Nicholas Soames
Dame Caroline Spelman
'The question on the ballot paper was simple: we were asked whether we wanted to leave the EU or remain in the EU.
'The attempt by Mr Hammond and others to re-interpret the referendum result is patronising and deliberately misleading.'
He added: 'The message to those like Mr Hammond who want to block Brexit and to those who would do deals with Mr Corbyn to block Brexit is this: leaving with no deal isn't a betrayal of the British people, but trying to stop it is.'
Mr Hammond's letter to Mr Johnson was signed by numerous former Cabinet ministers including David Gauke, David Lidington and Rory Stewart.
The letter said: 'We are alarmed by the "red lines" that you have drawn which, on the face of it appear to eliminate the chance of reaching agreement with the EU.
'Any deal necessarily has to be a compromise, and many commentators feel that you have set the bar so high that there is no realistic probability of a deal being done.
'We would therefore greatly appreciate your confirmation that you remain committed to doing a deal; that you accept any such deal will most likely require compromise and that it remains your view that the chance of No Deal is "less than a million to one".'
'Everyone knows that the ex-chancellor's real objective was to cancel the referendum result,' the source said.