Ms Caldwell, 27, was murdered by Iain Packer in 2005.
The serial rapist was convicted last year and was handed a life sentence with at least 36 years behind bars.
The Scottish government has ordered a judge-led public inquiry into the original police investigation amid accusations of failings that allowed Packer to escape justice for a quarter of a century.
Following the killer’s conviction, Sky News heard testimony from several former sex workers who raised concerns to police about his dangerous sexual behaviour years before Ms Caldwell was murdered.
Speaking following a meeting with solicitor general Ruth Charteris KC on Wednesday, the family said Justice Secretary Angela Constance had previously “invited” them to give input for the terms of reference (ToR), which sets out the areas the inquiry will cover.
During the meeting in Glasgow, the family said they learned a draft ToR had been drawn up and circulated with a number of parties to the inquiry without their knowledge.
Margaret Caldwell, Ms Caldwell’s mother, described her “sheer disappointment and fatigue” at having to wait so long for the inquiry to start.
She said: “I just want it to go ahead. I want what I’ve always wanted – justice for Emma. It’s all I’ve ever asked for from day one.”
The family’s solicitor, Aamer Anwar, read a statement to the media following the meeting.
He said: “The family put their trust in the Scottish government and today, sadly, they feel betrayed yet again.
“When we met with the justice minister, she invited the family to input into the ToR.
“Today, we were told by the solicitor general that she had been provided with a suggested potential draft ToR. None of those have been provided to the family.
“When we asked the solicitor general and those in Crown Office to provide those to us, we were told those are the property of Crown Office and that they cannot be provided to us.
“The family feel sadly patronised by those within the justice department and treated like a box-ticking exercise.
“The solicitor general said today, quite rightly, that the most important person in this inquiry is Emma’s mother, Margaret Caldwell.”
Read more from Sky News:
Man dies and woman injured in UK park shooting
Woman shot by husband after trying to escape relationship
Mr Anwar called for the inquiry to begin as soon as possible and urged for it to look at the Crown Office amid accusations of “misconduct” by some of the officers and lawyers involved in the case.
A spokesperson for the Crown Office said: “We have a duty to act on the lord advocate’s instruction of an independent investigation into potential criminality by police officers in this case and we also support the public inquiry established by Scottish ministers.
“These processes are separate and they can proceed in parallel. The Crown Office and Procurator Fiscal Service is clear that both should happen as quickly as possible.”
Justice Secretary Angela Constance added: “My thoughts remain with Margaret Caldwell and Emma’s family.
“I was grateful to them for meeting me last month and to be able to update them that as well as the chair being in position, the terms of reference for the public inquiry are being finalised.”
