Villains are a tricky lot. A lot of times in writing the villain is around to just serve as an obstacle that the protagonist is meant to overcome. They are put in place to create conflict for dramatic tension. The story may have less of an impact if there wasn't someone working against the hero. In our every day lives we tend to not have villains. Sure it may feel like there are people who are actively working against us, but more often than not, it's just someone going about their life and their method happens to conflict with ours. So the conflict in our life comes from other sources, usually in the form of situations that are possibly beyond our control. While stories about those types of situations are compelling, adding a antagonistic human element to the mix can make things even more interesting. I've talked about how perception shapes what is constituted as evil. For the most part we believe what we're doing is right and if not right, at least acceptable to us for whatever reasons we come up with. We could easily become the villain in someone's story and not even know it.
Sherlock Holmes was written as a brilliant, but flawed detective that always figured out what or who was behind the mystery at hand. This type of character is very interesting to read at first, but like with many characters they begin to take on an almost superhuman ability to overcome everything put in front of them. If the hero always wins then it's hard to create tension because the audience knows the hero will win somehow. The same thing has been a problem for Superman in the comics. His abilities have made him god-like and if he can do anything then how do you create a situation that would seem even remotely threatening? With Sherlock Holmes it was decided to end his adventures by going up against essentially an evil version of himself. Professor Moriarty appears in The Final Problem and is revealed to be the mastermind behind several of the cases that Holmes had taken on. Up until that point each of the mysteries could be taken as independent of each other and having no real connection. With the creation of Moriarty it was shown that many of the problems Holmes solved were originally part of a larger issue constructed by Moriarty. Many people now consider Professor Moriarty to be the archenemy of Sherlock Holmes, even though he only appears in two stories and is briefly mentioned in a few others.
That leads to some of the problems that come with villains. The first is too much exposure. In the story Red Dragon there is a brief appearance by a character named Hannibal Lecter. He wasn't the focus of the story and was really used as a way to help the main character track down the real villain, The Tooth Fairy. Hannibal has always seemed to be just another spin on the Moriarty-type villain. Brilliant, but twisted in a way that makes them fascinating. Because Hannibal worked so well in the first novel, he was used in very much the same in the second one, which most people know, Silence of the Lambs. His role is almost exactly the same as it was in Red Dragon, except it's expanded a bit. Then it was taken too far. Harris wrote a third novel which focused entirely on Hannibal and sucked away all the mystery surrounding the character. In order to cash in before it was too late, even a prequel was made to further explain what made Hannibal into the man we knew him to be. It's a fine line with villains. You want them to be more than a one note act, but you can't go too far in the other direction and explain too much because the villain can lose his impact.
As I mentioned, we often don't have full on villains in our lives. Instead we just have people doing their thing. Now in war we have an enemy. We may know who our enemy is, but we may not really know who our enemy is. In World War II America knew that Germany was causing serious problems in Europe, but that was Europe's problem. When the Japanese attacked Pearl Harbor the war came home. Japan, and Germany because they were allies, became our enemy. I'm guessing most soldiers didn't know a lot about the Japanese or German people and probably didn't want to know about them. There were no longer people. They were the enemy. The enemy becomes this almost faceless idea that must be fought against, not related to. It's only when soldiers on both sides came face to face with each other did they realize that the enemy is just another person. Knowing that may not change the outcome or desire to win, but it stops being an abstract at that point. With a villain the same thing is true. They start off as this idea. They are the negative counterpoint in the conflict, but only because we've been introduced to the hero as the hero. In theory the story could be told from a different vantage point in such a way that the hero is the antagonist. It's not easy to do because villains tend to do villainous things that many people couldn't relate to. Superman stands for truth, justice, and the American way (whatever that is). Lex Luthor stands pretty much for himself. His belief is that mankind will never achieve greatness with someone like Superman standing watch. Maybe he's right, but the part that he's not telling most people is that he himself wants to be the one that leads mankind into greatness.
There have been books and movies that follow something of an anti-hero through an adventure. This could be a reluctant hero or even a straight up villain himself. The tendency is to set them up against something worse. Riddick goes up against space monsters. Danny Ocean steals from the ruthless Benedict. So we end up rooting for them because given the situation, they are the hero and what they're up against is the villain. In most other circumstances you wouldn't want to run up against those types of "heroes". Predator hunts humans and takes their skulls as trophies. It seems a bit cruel because humans are clearly outmatched in nearly every way. Predators are larger, faster, stronger, and more technologically advanced than humans and it almost seems unsporting for them to be hunting people. If deer could make movies then humans would be the horrifying creature that not only kills, but skins its victims and mounts pieces of them as trophies.
Speaking of which, monsters are easy villains because you don't have to explain their motivations. The reason they do what they do is hidden behind the fact that they're monstrous. As I've mentioned, not all monsters need to be literal monsters. More often than not the monster is just a person. In the movie Se7en Somerset warned Mills that labeling John Doe as crazy was comfortable. John Doe was twisted, but he believed what he was doing was righteous. In his mind the victims were the villains of society and needed to be punished as a warning to everyone else like them. In the end though that's all the explanation we need from him. Anything more would have watered down the character. We understood that he is just a person who has taken extreme measures to create a world that he believes is correct. In the grand scheme of things his crimes were an isolated event that impacted only those in his immediate area.
That leads to the other problem with villains. In addition to being overexposed, they can often be given far too much credit. Moriarty is a prime example. He went from being a brilliant criminal mastermind to someone that was involved in nearly everything Holmes went up against. Villains suffer from the same problem that heroes do. While Superman has an array of powers that make it difficult to create situations where anyone can truly be worried about the outcome, villains too have to constantly increase their threat level to make sure they're still relevant. In the early Spider-Man comics there was a somewhat goofy character named the Green Goblin. At first he was just another throw away bad guy meant to be flashy and colorful. Then he started to be a mastermind leading several criminal gangs and became more of a threat. It was revealed that the Green Goblin was actually the father to Peter Parker's best friend Harry. This really created conflict because even if the hero won, he would still lose. Eventually the Green Goblin kidnapped and killed Peter's girlfriend, Gwen Stacy. This lead to a final fight between Spider-Man and Green Goblin that resulted in the Goblin being impaled by his own glider. That should have been it. A great villain had a lasting impact on Spider-Man from that point on. Twenty years later the Green Goblin not only returns from the dead, but reveals that he has been responsible for manipulating things from behind the scenes. The Green Goblin, and more directly, Norman Osborn became an arch-nemesis to Spider-Man in a way that no other character had done before. It wasn't enough to simply have him be an especially smart and threatening villain, it was decided that he would be responsible for so many of the problems plaguing Spider-Man. Sounds a bit like the Moriarty-Effect at work here.
Villains can be very interesting because they not only represent conflict personified, but can show the flip side of humanity. An effective villain doesn't have to be one that we identify with or even fully understand. They only need to have fully realized motivations for what they do in order to make them believable. By doing that the villain becomes a very real threat.
