In an interview with the Oxford Union, Whedon spoke about his involvement in the MCU, saying he was a consultant on all of the Phase 2 films, but decided to solely focus on "Age of Ultron" when it went into production -- with the intention that the "Avengers" sequel would be his last movie with Marvel.
Here's Whedon's explanation, courtesy of Cosmic Book News:
Will you have any involvement in the wider Marvel Cinematic Universe?
Whedon: No. You know I was their sort of Consigliere for a while... We do not discuss our thing... [audience laughter] But, I sort of had my finger in all of the films in the second phase, but then I just had to concentrate only on Ultron, and sort of know when it was done I was just going to stop. So I made a completely clean break - not because we had a falling out - just because I was like, "I can't..." If I was still there going, "Well, here are my thoughts on this film," I'd be there every day. I wouldn't do anything else because there are a lot of films, and it is a lot of fun. It's very seductive. When you can put your little fairy dust on things and just improve them slighty, and they actually listen to you... I was a script doctor for a long time, and the part where they listen to you was very rare; so it was very important for my own self to go "we can still be friends," but...
You can watch the full interview with Whedon below:
While Whedon won't be back, the MCU forges on, with "Captain America: Civil War" hitting screens on May 6, followed by "Doctor Strange" on November 4. The next "Avengers" film, "Infinity War - Part 1," is slated for a May 4, 2018 release.