It's tempting to say that classic comic plays aren't just slapstick and goofiness, but that's exactly what some of them are. Shakespeare's A Comedy of Errors is little more than a simple case-of-mistaken-identity tale with plenty of funny violence and eyebrow-raising jokes. It's plot is also stolen directly from Roman playwrite Plautus' Menaechmi, complete with the same characters and the same one-liners.
Does this thievery make Shakespeare's play less appealing? Certainly not. For one thing, it demonstrates the continuity of literature; for another, it shows that even the most original writer has to get his ideas from somewhere. A Comedy of Errors came at the beginning of the Bard's illustrious career, and even if plagiarism is a crime we can probably forgive him this one as he was just developing as a writer.
Comedy in the dramatic sense isn't simply intended to denote humor. A comedy is any play that ends well for all or most of the characters. If someone dies, it's the bad guy; if someone ends up in exile or divorced or under a spell it's because the audience will be satisfied with no other outcome. Until fairly recently (within the last 150 years or so), "dark comedy" was a misnomer; if a play was dark, it wasn't a comedy, and vice versa. Sure, tragedies had jokes, but the words comedy and jokes weren't used synonymously.
The Greeks maintained a definite distinction between which plays contained humor and which ones didn't, however. A tragedy like Oedipus Rex isn't funny at all, while one of Aristophanes' satires (notably The Clouds and The Birds) keeps up a stream of jokes from beginning to end. Plays became less compartmentalized when they became a little more realistic.
Not that anything by Shakespeare or Ben Jonson is realistic in the sense that it portrays everyday life accurately (the way writers of the Realist school used the term). Their plays were realistic in the sense that "tragedy" and "comedy" came to be seen more in terms of outcome. The Greeks wrote tragedies in which the tragic moment defined the play, whereas Shakespeare often led the audience along with hints of happiness and a possibly happy ending before killing everyone off or having them go blind.
Comedy underwent a similar reimagining. All of life wasn't seen as funny or ridiculous in every comedy; happiness was simply shown to be a very real element of human existence that often befell people (whether they deserved it or not). Later writers like Oscar Wilde resurrected to some degree the idea that comedies should be funny clear through, but guys like George Bernard Shaw reestablished a sense of balance.
Many of the plays in this section are funny; some are not. Not all of them end well—satires aren't really about "happy" or "sad" endings, they're about pointing out the foibles and sins of a culture by exaggerating every element of the society under consideration. In many ways, the plays in this category represent the relativism of literature, the way in which terms are used differently in different contexts, and how these uses shape and change a given genre.
Yet one constant has remained: comedy has always appealed to the part of the human psyche that desires and needs comfort. Whether that comfort comes from a "happy ending" (which for Shakespeare was usually marriage), or from recognizing your neighbor in a satirical caricature, or simply from laughter induced by absurd jokes and antics, comic plays often retain their appeal for centuries or millennia because people, who will always have tragedy in their own lives, continue to need a better (and, for Christians, more accurate) view of the world than unrelenting sadness offers.