The Australian reports a recent baptism, captured on video and dated September 21, has been posted on the YouTube internet site.
The clip shows resident priest Fr Terry Fitzpatrick baptising a young child with the words, "We baptise you in the name of the creator, sustainer and liberator of life", adding "who is also father, son and spirit".
The priest then added: "That's good, nice and cool" and invited "everyone to put water on him".
Catholic commentator and editor of Annals magazine, Fr Paul Stenhouse, told the paper that he doubted the baptism was valid "because the form of the Sacrament cannot be varied".
"It sounds like green theology to me," Fr Stenhouse said.
The legitimate form of baptism is: "I baptise you in the name of the Father and of the Son and of the Holy Spirit."
The two priests at St Mary's, Father Fitzpatrick and Peter Kennedy, were ordered to revert to the correct form of baptism in 2004 after it emerged that hundreds of children had been baptised using the wrong formula.
In March, a statement released with approval from the Pope outlawed baptisms being conferred with the words "I baptise you in the name of the creator, and of the redeemer, and of the sanctifier" or "I baptise you in the name of the creator, and of the liberator, and of the sustainer".
The fresh controversy has emerged after Archbishop Bathersby wrote to St Mary's in August warning the church could be closed down "by practices that separate it from communion with the Roman Catholic Church".
The letter was prompted by complaints to the Vatican about a Buddhist statue in the church.
A spokesman for the Catholic Church in Brisbane said: "Once those with liturgical expertise have had an opportunity to view the YouTube footage and discuss it with others, they will be able to provide some advice to the Archbishop as to the validity."The Youtube video is now only available for viewing by "friends" of the person who posted the video to the site.
Suburban Catholic church tests the tolerance of Rome (The Australian, 4/10/08)
Brisbane church defies Pope on baptism (The Australian, 3/10/08)