Rian Johnson has revealed some of his thoughts on why he didn't end the Jedi in a movie that is called The Last Jedi. As it turns out, the director was getting ready to give Star Wars fans something that they didn't expect, which is something that Johnson tried to do throughout the entire movie and did successfully. Some of those twists have been criticized by fans and The Last Jedi has split the fan community in half with some agreeing with Rian Johnson's risks and others flat out calling for the movie to be stricken from the official Star Wars canon.
The Last Jedi is very much about the past and the character's relationship with it. Kylo Ren is looking to demolish the past and start fresh and reinvent himself while Luke Skywalker is looking to avoid his past on his exile to Ahch-To after becoming disillusioned with the Jedi. Luke discovers that the Jedi were arrogant and allowed the Empire to grow to power, so he decides to go to a remote part of the galaxy to die and end the Jedi. However, Rian Johnson says that was never going to happen in the movie.
According to The Art of Star Wars: The Last Jedi, Rian Johnson never saw the end of the Jedi as a good idea for the Star Wars franchise. Johnson went on to talk about embracing one's past to move forward, which is ultimately what happened in The Last Jedi. In the end, Johnson decided that ending the Jedi was never a "valid choice." He had this to say.
"When Rey shows up, the first and foremost thing is she needs a mentor. In looking at this grand plan from ten miles up in the air, Luke is missing the thing right in front of his nose. Here's somebody who needs you, who needs your help. If you think you are throwing away the past, you are fooling yourself. The only way to go forward is to embrace the past, figure out what is good and what is not good about it. But it's never going to not be a part of who we all are. And that includes Rey, who grew up hearing the legends about the Jedi. So the notion of, 'Nope, toss this all away and find something new,' is not really a valid choice, I think."
Rian Johnson never intended The Last Jedi to be something that fans would anticipate and though many didn't think that the Jedi would end with Luke, it sure looked as if it could. Luke Skywalker is broken, but later through Rey, he realizes his place and becomes one with The Force, accepting his role. Rey takes the ancient sacred Jedi texts, something that Luke never even bothered to read during his exile on Ahch-To, and it is believed that Rey will lead or teach the new Jedi in the ways of the Force.
It isn't clear what J.J. Abrams has planned for Star Wars 9, but the continuation of the Jedi is something that will have to be addressed. In addition, Kylo Ren will more than likely still be on his quest to destroy the past, which is something that will now include Rey, who is a new Jedi. As far as Rian Johnson is concerned, there was never any choice on whether to end the Jedi or not, it was just something that you can't do in the Star Wars universe. Not yet, anyway. You can read more about Rian Johnson's comments on ending the Jedi via Abrams Books.