Hey, Ash, if you have PCOS, you should talk further with your OB or RE about your metabolism. If you are insulin-resistant on PCOS (many are, but not everyone), you may really need Metformin to help regulate and really lose weight.
Avoiding refined sugar and carbs is a diet that helps a number of PCOS women w/ insulin resistance, but keeping weight/struggling to lose weight is one of the more common features of the disorder (remembering that PCOS is a very broad-based disorder and not all women will be affected the same way).
If you haven't already found it, I suggest soulcysters.com as an excellent, excellent resource for you. You'll find the most current research, support and help in navigating the disorder.
It can really, really throw off your metabolism and ability to lose weight. I've been tested because of my weight (but exhibit 0 symptoms outside late ovulation, which is likely due to my weight period - fat stores estrogen, the higher the levels of estrogen, the slower the follicles mature, because the hormone feedback is artificially disrupted). I have a couple of friend with PCOS - one of them has had a really hard struggle with weight because of the disorder. She works out harder and eats better than anyone I know, but until her doc put her on Met, she simply didn't lose weight, no matter how much she tried. Once her insulin resistance was dealt with, she was able to begin losing weight in a normal, healthy manner.
Don't discount that as a potential issue in weight-loss.