The news means that three out of the four original judges from the last series will not return to the show, following the decision by Simon Cowell and Cheryl Cole to switch to the US version of the show.
Minogue will be replaced by N-Dubz singer Tulisa Contostavlos, who is expected to sign a contract to join the show this week.
Contostavlos will join a new panel, which already includes Take That frontman Gary Barlow, who is set to replace Cowell, the show's creator and key judge.
Speculation is now bound to turn to the replacement for arguably the most popular judge, Girls Aloud singer and tabloid darling, Cole.
Sources at 'The X Factor' said the announcement of Cole's replacement would be made just before the show’s auditions kick off in two weeks' time, and said the announcement of the new judge will be "incredibly exciting".
In an official statement, Minogue said: "I am so disappointed that I can't be a part of The 'X Factor' this year. I'm going to miss the show and the amazing team who work so incredibly hard on it.
"During discussions for me to return [to 'The X Factor'] it became clear that unfortunately, this year, 'The X Factor' audition dates in the UK clash with the live shows of 'Australia's Got Talent' during June and July. For this reason, I am unable to return.
"I want to wish the new judges every success for this year's show, and can't wait to see who the British public choose as their next big star. Thanks to everyone who has shown me so much love and support, and brought me so much joy during my four incredible years working on the show."
The announcement of the full judging panel for the new series will be announced "in due course", according to ITV.
Danni Minogue was widely expected to be in a strong position to keep her position on 'The X-Factor' judging panel, following her success with former painter and decorator Matt Cardle, who won the show last year. Her spot was further bolstered by the departure of Cole, who was the only other female judge on the show.