The Kohen Gadol wore his eight special vestments when offering the various sacrifices, as he did on all other days. As detailed in Shemot, chapter 28, they were: the ketonet, mikhnasayim, mitznefet, avnet, ḥoshen, ephod, me’il, and tzitz. The ketonet, mikhnasayim, and mitznefet were white, while the other vestments were various colors, some also threaded with gold. (Therefore, they are sometimes called the “golden vestments.”) The ephod and ḥoshen were blue, purple, scarlet, and white, with gold woven in (Shemot 28:6, Rashi ad loc.). The bells on the me’il were made of gold, as were the tzitz, the straps and rings of the ḥoshen, and the ephod. The settings of the stones on the ḥoshen were also made of gold. Thus, the Kohen Gadol’s vestments were rich in color, expressing the spread of sanctity in this very diverse world. Each vestment represented a specific idea and atoned for a corresponding sin (section 3 above). Therefore, if the Kohen Gadol was not wearing every single one of these items, his avoda was compromised and therefore invalid.
In contrast, for the part of the Yom Kippur avoda performed in the Kodesh Ha-kodashim, he wore only his four white linen vestments, as we read, “Thus only shall Aharon enter the holy place…. He shall be dressed in a sacral linen tunic, with linen breeches next to his flesh, and be girt with a linen sash, and he shall wear a linen miter. They are sacral vestments” (Vayikra 16:3-4). If he performed this avoda while wearing his golden vestments, the avoda was invalid. For the special avoda of Yom Kippur, the Kohen Gadol had to raise himself entirely beyond this-worldly concerns. Even though every aspect of this world has positives, alongside these positives are negatives and sins. To atone for them, he had to raise himself to the level of simple unity, beyond the world’s diversity. This unity is symbolized by the color white (Maharal, Gevurot Hashem, ch. 51, and Netivot Olam, Netiv Ha-Torah ch. 10).
There is another reason why the Kohen Gadol changed out of his golden vestments before entering the Kodesh Ha-kodashim. “Why doesn’t the Kohen Gadol enter the Kodesh Ha-kodashim on Yom Kippur while wearing his golden vestments? Because the prosecutor cannot become the defender” (Rosh Ha-shana 26a). Gold is the most glorious of metals. This is the reason that Temple items were made of it – to reveal the glory of sanctity in this world. However, along with its glory, gold also leads people to chase after money and desires, as we see from the episode of the Golden Calf. The Sages tell us that it was the wealth and gold that Israel acquired when they left Egypt that led them to follow their evil inclination and look for a physical representation of the divine, which they then worshipped (Berakhot 32a). Therefore, when the Kohen Gadol entered the Kodesh Ha-kodashim to purge any impurity from Israel’s faith, he needed to remove his golden vestments and wear his white vestments instead.