Labor will determine its position on asylum seekers at a federal caucus meeting today and all frontbenchers will have to promote it, Opposition Leader Simon Crean said yesterday.

He warned that Carmen Lawrence, who was "ashamed" by Labor's pre-election policy stance, would have to support the caucus position when addressing a pro-refugee rally planned at Parliament House tomorrow.

Today's caucus position would be binding on Ms Lawrence and must be reflected in her public statements, he said. "What she has to do is present the Labor Party position, which will be determined at the caucus meeting on Monday."

His warning came as Prime Minister John Howard questioned if UN Human Rights Commissioner Mary Robinson was running "some agenda" by asking for an envoy to be allowed into Woomera Detention Centre to investigate possible human rights abuses.

Labor's new stance is likely to call for an end to the "Pacific solution" of processing asylum seekers offshore but to maintain support for mandatory detention.

Detainees must be processed more quickly, said Mr Crean, who has previously called for women and children to be removed from such

Mr Crean also said Mr Howard's visit to Indonesia had failed as no framework had been laid out for next month's Bali conference on people smuggling.

Mr Howard defended his visit but continued to refuse to guarantee Ms Robinson's envoy would be allowed inside Woomera. The government had already allowed representatives of the UN High Commissioner for Refugees access to detention centres, he said.

"Why is it that she also wants access?" Mr Howard said. "I mean, you might start to wonder whether there's some agenda being run." The government would not give an "advance blank cheque" on the issue and would discuss the request over the next few days, he said.

Human rights lawyers yesterday said a petition to the UN from Woomera detainees described the centre as enveloped in despair and unsuitable for children.