Even though many of us do not use TeX directly, knowing TeX can certainly help you use LaTeX better. For one thing, you can write your own macros.
For many years TeX by Topic by Victor Eijkhout is one of the best—if not the best—reference books on TeX since it is organized by, urr, topic. 
And here is the good news: Victor has generously put the full text online after the publisher reverted the rights to him. Kudos!
P.S. Reading TeX by Topic by itself can be very unproductive since it’s a reference book. For a book on TeX that you should read cover to cover, nothing beats Knuth’s The TeXbook.
