LONDON (AFP) - British Prime Minister Gordon Brown faced growing calls Sunday for a leadership contest as more than a dozen lawmakers from his governing Labour party attacked his performance. While only a small minority of Labour members of parliament have so far called for Brown to put his authority on the line, the dissenters insist momentum is building ahead of a party conference starting Saturday. Fiona Mactaggart, a former Home Office Minister, said it was time to give the party a chance to replace Brown, who has seen support for Labour plummet in the 15 months since he took over from Tony Blair. "I think we should give a chance to someone else to take over, I really do," she told BBC television. She said the British electorate "seem to have lost a sense of what it is we're trying to do and where we're trying to take Britain". Mactaggart attributed Brown's lack of credibility as prime minister to the fact that he had succeeded Blair in June last year without a leadership contest. "Gordon Brown was elected without anyone standing against him and in a way I think that has made it harder for him as well as harder for us as a whole party," she said, adding that she had received dozens of messages of support for her stance. Her comments came after another MP, Joan Ryan, was sacked as Labour's vice-chair after joining calls for a leadership contest, and government whip Siobhain McDonagh - whose job it was to enforce voting discipline in parliament - was fired on Friday for starting the calls. Other disgruntled backbenchers weighed in at the weekend, pointing to opinion polls showing the opposition Conservatives leading Labour by more than 20 points with a general election looming by 2010. Barry Gardiner, the government's special envoy on forestry, said the public had lost confidence in Brown. "He is not a popular prime minister, but he would continue to have my support if he showed sound judgment, international leadership and political vision," he wrote in The Sunday Times newspaper. "Instead, we have vacillation, loss of international credibility and timorous political manoeuvres that the public cannot understand." The rebels appear to be some way short of the support of 20pc of Labour lawmakers - equating to 71 MPs - needed to trigger a leadership contest but they hope to make their voices heard at the party conference in Manchester. Significantly, no member of the cabinet has yet thrown their weight behind the rebels. Foreign Secretary David Miliband, the 43-year-old tipped as a future party leader, said Sunday he did not support calls for a leadership contest and insisted Brown was capable of winning the next election. "I have said I expect Gordon to lead us into the next general election and I will support him in doing so," Miliband told BBC television. Miliband said: "I don't support their (rebels) argument that we should trigger a leadership contest. "I've said I expect Gordon to lead us into the next general election. I will support him in doing so."  The British Foreign Secretary's support for the Prime Minister is particularly significant as he has been seen as a contender for the top job. Business Secretary John Hutton said he did not dismiss the rebels' concerns, but also refused to join their calls for a change of leader. "I'm not going to criticise any of my colleagues who want Labour to do better and neither am I going to criticise those who say, for example, that we do need to set out a stronger vision of what we are doing," he told the BBC. "It is a difficult political climate for us. There is no question at all about that. "And that is a challenge to all of us in the Cabinet to do better - not just to Gordon as prime minister, but to all of us to do better, to make our arguments more convincing and clearer."