Eating too many sweets?

Is it okay to have a few small sweets every now and then? The ones I have are usually like small chewy ones like refreshers, I have about 3-4 sometimes a day. I'm trying to stop, and I'm cutting back on a lot of food, but do you think these sweets are doing me bad? Thank you & any help is appreciated xx