In all these groups there are fantastically helpful reviewers. There are also reviewers whose comments just don't ring true with me, and whose opinions just don't carry the weight that other reviewers do.
I think the trick to getting something out of a critique group is to find the people who you trust. It may take some time. You also have to be ready to take criticism. When I joined my first group, I don't think I was. I was still convinced my book was brilliant and perfect and anyone who disagreed was a moron. It took a few months of critiquing other people for me to have enough perspective on my own work to know some of the comments I was getting were right.
Now I have a solid group of reviewers whose feedback resonates with me. I know I can trust their judgement, and if they say something doesn't work, well, I believe it. These are my go-to-guys. When I'm stuck somewhere, they're the ones who can unstick me. I still get critiques from people who don't get my work, or are just copying the comments the person before them made, but it doesn't bother me. I just thank them politely, and move on. And every now and then, one of them points something out I've never thought of before, and none of my other reviewers has mentioned.
As a writer your work is going to be read and misunderstood by (hopefully) a wide range of readers. A critique group can be your first introduction to the diversity of your audience. And yeah, it's kind of scary!