The Israeli government weighed in on the issue with a statement that the controversy was harming Poland's image and straining a "well-developing Polish-Jewish dialogue."

Also in Israel, directors of the Yad Vashem Holocaust Memorial museum have said the crosses violate an international accord under which it is illegal to erect political, ideological or religious symbols at the site.

In Poland, the country's leading Catholic clergyman called on the radical religious groups to stop erecting the crosses. Cardinal Josef Glemp issued the call after fringe Catholic groups, backed by hard-line priests, ignored Jewish protests.

The Polish government, asked for its help in the issue, this week asserted that it would not work to have the crosses removed. This prompted an outcry from the Los-Angeles based Simon Wiesenthal Center, which responded by sending a letter to Jerry Kozminski, the Polish ambassador to the United States.

"The installation of these crosses in Auschwitz sends a clear signal to Jewish communities throughout the world that your government, in effect, cedes its international responsibility on this matter to a group of Polish extremists who intend to impose the symbols of Christianity upon Jewish suffering," said the letter, according to a Wiesenthal Center statement. "These extremists are actually attempting to write Jews out of Polish history."

The letter was signed by Rabbi Abraham Cooper, associate dean of the center, and Mark Weitzman, director of the center's national task force against hate.

It continued that "the government of Poland, as custodian of Auschwitz, must accept the responsibility of ensuring that no one individual, group or organization, impose and alter the character of the site…"

Therefore, the letter said, the center demands "swift action" to restore the site to "a silent but powerful reminder of man's inhumanity to man" and "to ensure that the memory of the 1.5 million Jews martyred there is not further demeaned at the very site of their murder."

This isn't the first time that crosses at Auschwitz have been the source of controversy.

In December of last year, the Polish government ordered the removal of eight crosses and 11 Stars of David from the site. The crosses had been placed at Auschwitz in 1983 by a Polish scouting group, while the Jewish symbols had been placed by unknown parties.