I cry every time I teach one of my art lessons. The kids are really mean. They have those little games going, such as at certain times someone gives a signal and the whole class dive under the table (minus three who still behave well). We don't do detentions at my school, nor punishment work, and in art we are not even allowed to give homework. There's nothing I can use to threaten the pupils, not even to get them to finish an assignment at home.
What you said
You need support from your department head or senior leadership team. If no sanctions are available, then run for the hills now! Praise the three who are working and explain the individual behaviour to the rest. Several of my pupils go pale at the very mention of phoning their families: it has far more impact on their lives if they're grounded or banned from the internet at home than if they miss part of lunch at school (which you can't even do).
If, as a form tutor and class teacher, I found out that my pupils were behaving with such disrespect towards a colleague, I would go ballistic. As others have said, your colleagues are vital in a situation like this. Kids need boundaries, so even if your school isn't into the idea of detentions and so on (which is nuts), they still have a responsibility towards you and the pupils to ensure they are guided in terms of their behaviour.
The expert view
The school doesn't allow detentions or other forms of sanction? Seriously? If that's true, I despair for the school, for you and most of all for the kids. Unless we expect all children to be perfect, we need consequences, and that means rewards and sanctions. If you had any leeway to do so then you could solve this in a fortnight. There must be some system for poor behaviour, surely?
What do other people in your school do? Find out. Look to other teachers in your faculty area to see what strategies they use. But you're disadvantaged immediately because you're new to the classes and haven't built up a relationship yet.
Surely you can at least call home and ask the parents to help you, being careful to speak to them in a way that suggests you're both working for the same goal, the kid's well-being, which you are, whether they know it or not.
Or you could speak to line management about what strategies they suggest - and don't be fobbed off by any ideas that revolve around "they need to get to know you" because that's a coward's answer. I'm astounded that a school would abandon a new member of staff so much. This isn't your fault, but theirs.
Tom Bennett is author of The Behaviour Guru and Not Quite a Teacher. Read more from Tom on his blog, behaviourguru.blogspot.com, or follow him on Twitter at @tombennett71. His latest book, Teacher, is out now, published by Continuum
Post your questions at www.tes.co.ukbehaviour.