Whether or not you're a fan of Dr. Seuss, you aren't likely to forget his books for one simple reason—they rhyme. Each line carries a cadence that drives the sing-song deep into the memory where it survives and prospers, and it does this most effectively in the minds of children. The rhythms of language impart a love of words kids are unlikly to abandon when they're older.
It's pretty common to equate "rhyme" and "poetry." There are a couple things wrong with this assumption: not all poetry rhymes, and poetry is more about meter than it is about vowel-chime. Meter is the beat of each line, the "da-da-da-da-da-da" that makes poetry distinct from prose. Rhyming books aren't necessarily poetry (at least, not in the Robert Frost or John Milton sense), but they are incredibly musical.
Kids aren't born automatically in love with language. If their first encounters with reading (either being read aloud to or learning how to read themselves) throw them into contact with primarily dull books, there's a good chance they'll grow up with a distaste for words. Even if they encounter brilliant writing later, they're already predisposed to reject it, largely because they don't understand it or have the tools to appreciate it.
Rhyming books are a great place to start to help raise readers who are entranced by well-constructed sentences. If you suppose someone like Bill Peet is a shoddy craftsman because he writes relatively simple narratives for kiddos, you need to re-read The Caboose Who Got Loose. Writers who can create fun stories using a form as difficult as metered couplets deserve not only our admiration, but our readership.
We love words here at Exodus Books. We love the way the printed page looks. We offer many first-rate titles. Peruse them when you're feeling idle.