I disagree
I've always been a firm believer than one should train in both a striking and a grappling art if they have the means to.
When doing both you'll start to see similar situatoins and positions and many things, like an understanding of body positioning,anatomy, and leverage will really start to have positive effects on both.
The only caveat would be if you were competing regularly with dreams of nationals and beyond.
That said, were I you I would drop judo and study BJJ. A lot of the submissions will feel familiar, you wont magically forget your throws or takedowns and can still work them, you'll most likely get to work with wrestlers and MMA guys(will will be more than willing to help you blend tkd and BJJ even more as well as exposing you to more fighting styles helping your adaptibility), and you'll have the focus on groundfighting that comes with BJJ.
BJJ is a good grappling style, but I personally feel its one of the best complement arts due to the fact that it fits nearly every other background without having to worry about confusing overlaps (i.e. we had a lady who could kick the hell out of the bag the way she was taught in her CMA, but not the thai way)