Diplomacy is always good -- especially when dealing with creative people.

If something is vastly different than what I had discussed at a production meeting, I speak to the designer (PRIVATELY) and ask "is this what we had originally planned?"

If the designer becomes defensive (or out & out lies) then I respond politely with "I'm sorry, but I was picturing something very different. Can this be changed?"

If yes, then great. If the designer again disagrees, (No time! No money! No materials!) Rather than trying to bargain for more time or money (which is never available) I merely say, "Well, maybe I'll have to wait until it's all done to see what your concept is. But if it still becomes a distraction, then I'm sorry, but it will need to be fixed."

At this point the designer realizes I might just designate someone else to fix the problem and they become more agreeable. "What is it that is distracting to you?" they ask.

It's important you are SPECIFIC in your answers, rather than just "I don't like it."   The color is too bright and needs to be toned down. The wallpaper is uneven or too complicated ... etc, etc.     

Offer SOLUTIONS rather than point out problems. Can we do a quick sponge wash with a darker shade?   Would another trim color soften the palette? Can we hide the problem areas with art, drapery or furniture?

Sometimes the designers are "too close" to the project to be able to look at it through another person's eyes.

My mother is a set designer. I remember she was working on A Christmas Story and the director came up to her several times and mentioned that he didn't like the wallpaper here or the paint here, etc. Yet each time she told him to wait 'til she was done. And he did. And he liked it each time once she was finished.

If you really like the work of your set designer and you trust them, wait
'til it's done. As dexter74656 said, it is completely in your jurisdiction but perhaps you haven't gotten the whole effect yet. "All of the parts, none of the whole".

But if the set designer is just completely off base and not on the same page as you, you will regret not speaking up!

   Ideally of course, the director should meet with the technical staff prior to any creative work beginning.  This way all people involved should be of the same mindset. 

   I am reminded of a show I directed where the set builder had the set constructed prior to our first rehearhal.  Normally this would be great, but when we started to block we quickly discovered that the set simply was impossible to work on.  I had to go to the set builder and ask him to redo a large portion of his construction. 

As a director, I am responsible for every aspect of the show coming together as a unified artistic vision.  I do a lot of research up front, and I usually know what I want (or what I think I want) prior to talking to my designers.  And I talk a lot to my designers before the first penny is spent.  I want to make sure we are all on the same page and that they understand where I am going with the project.  I then want them to come back to me with their interpretation and ideas.  They will come up with stuff I never thought of - both good and bad.  I am always willing to listen to new ideas even if it wasn't part of my original plan.  But the final decision is mine.

As a result I've not had to deal with a lot of 'that's not what I wanted'.  When it does occasionally happen, I talk to them about in in specific terms of why it doesn't work for me and ways to fix it.  Very open dialog, and nobody is ever pissy, including me.

And any artistic director that says that you as a director don't have control of what goes on your stage should be fired post haste.   Artistic Directors are there to guide the vision of the theatre as a whole, by supporting the individual shows within the season, and helping the director realize the shared vision of the season within that particular show.