Yanai was one of the Chashmonai kings (Rashi and Tosfos here state that). The Gemara tells us how an especially evil man (ish letz, lev ra, ubliyaal) Elazar ben Po'ira convinced Yanai that the Prushim were not fond of Yanai and that led to Yanai killing many, many chachamim. What was the action which Elazar used to show Yanai that the Prushim opposed him? To wear the Tzitz at the same time as the royal crown.
The Maharsha gives a beautiful hashkafic explanation to the problem of Yanai wearing the Tzitz and Keser Malchus. He interprets the Mishna in Avos "There are 3 crowns, the crown of Torah, the crown of kehuna and the crown of malchus, and the crown of good name is on them" as literally referring to Tefillin, the Tzitz and the royal crown, and all have the name of Hashem (keser shem tov) on them. The Gemara says there is room on the head for two tefillin shel rosh at one time (there is a dispute if it means side by side or in front and in back. I know a Persian man who wears 2 battim shel rosh at the same time, with one in front and one in back, and he keeps the second bayis & its retzuos covered in a very large kippah cap, almost like the old chazzanim used to wear). The Maharsha explained that when Yanai wore the royal crown and the Tzitz, he necessarily had to remove his Tefillin, because there was no room for them on his head. This was the complaint - isn't it enough to be melech because you already have to wear tefillin and to be Kohen gadol you also need to wear the tzitz, but there's not enough room for all of them.
With this Maharsha we can give another answer to Tosfos in Avodah Zarah 34a. The Gemara asks what did Moshe wear during the seven inaugural days of the mishkan, during which Moshe served as Kohen Gadol? A white garment. Why didn't Moshe wear the regular bigdei kohen gadol? Tosfos (s.v. Bameh) brings two answers: First, there was no law of bigdei kehuna yet, since the pasuk says "and you shall make holy garments for Aharon and his sons" - they were for Aharon and his sons, and not for anyone else. Second, R. Yaakov of Orleans answered that during the seven inaugural days the Mishkan had the status of a bamah (a large communal altar) but did not have the status of mikdash, and bigdei kehuna may not be worn at a bamah. With the Maharsha we may give a third answer (though it's a very hashkafic answer to a very halachik question): Moshe had the status of melech (Zecachim 102 and Rambam Hil. Beis Habechira 6:1, I'm looking at Griz on Vayeilech). He had to wear tefillin. And if he wore bigdei kehuna gedolah, he would have to wear the tzitz also, but there wasn't room for it. So he couldn't wear the bigdei kehuna.
Why does the Maharsha say this was a problem by Yanai, but other Chashmonai kings were also kohanim gedolim? Perhaps they always took off the Tzitz immediately after the avodah, which Yanai did not do (see Rabbeinu Tam in Tosfos s.v. Hakeim). Side lesson - we see that just because something is mutar (i.e., to wear the tzitz shelo b'shaas avodah), it doesn't mean you have to do it. If Yanai would have taken off the tzitz, this episode would not have happened. To leave off with a question, is the statement of Yehuda ben Gididya similar to Rabi Zecharia ben Avkulas who, according to Rabi Yochanan in Gittin 56a, caused the Beis Hamikdash to be destroyed in the Kamtza & Bar Kamtza incident (as we see in Bava Basra 4a when Hordus realized the chachamim were good people after he killed so many of them, Bava Ben Buta said you killed "ohr - the chachamim" so fix "ohr - the Beis Hamikdash") in that Yehuda ben Gidiyah's statement caused many chachamim to be killed)?