There are a lot of definitions that get thrown around about “the deep web” and “the dark web.” It is frustrating how people use the terms without a clue as to what they mean. The deep web and dark web are NOT synonyms!
Starting with defining "The Internet," think of all the wires and connections as a highway system. When I talk about the general term of the internet, I am speaking about the technologies that move packets of information along wires from one destination to another, specifically the family of protocols known as TCP/IP (transmission control protocol - internet protocol).
The "World Wide Web” represents the many destinations that are connected together using the public highway system of the internet. When I talk about the general term of the World Wide Web, I am speaking about the technologies that create websites and webservers such as HTTP (hypertext transfer protocol) and HTML (hypertext markup language).
Where it gets confusing is how you apply the usage of the terms. Sometimes when people say "the internet" they are not describing just the highway system, but they are using the term to represent all the websites in existence. Likewise, often when people say “The World Wide Web” they use it to mean all the websites in existence.
The technology that the internet uses on the public highway, things like the internet protocols like TCP/IP and World Wide Web components HTTP and HTML, can also be used to take us to private destinations as well. This collection of private destinations is known as the "Deep Web." Computer scientist Michael Bergman, founder of search indexing specialist company Bright Planet is credited with coining the term deep web in 2001 as part of a research study.
In 2014, a Forbes article, "Insider Trading On The Dark Web"(1), completely confuses the terms and misquotes the definitions of BrightPlanet CEO Michael Bergman and incorrectly describes Bright Planet as "a firm that harvests data from the Dark Web." In response to confusion about the terms Deep Web versus Dark Web BrightPlanet published the article, "Clearing Up Confusion – Deep Web vs. Dark Web." (2)
The link to the BrightPlanet article is listed at the end of this article, but here are a few points from that article which define the main points.
- "The Surface Web is anything that can be indexed by a typical search engine like Google, Bing or Yahoo."
- "...the Deep Web is anything that a search engine can’t find."
- "The Dark Web then is classified as a small portion of the Deep Web that has been intentionally hidden and is inaccessible through standard web browsers."
- "The key thing to keep in mind is the Dark Web is a small portion of the Deep Web."
Why does the "deep web" have much more content than the "regular web" since it's used by far fewer people?
Here's an analogy that might help you understand why there is so much more information "below the surface" on private networks, than above the surface on public networks.
Go to the downtown of an average city where you can find a variety of commercial office buildings. Some of the buildings have a lobby, where you can go inside and walk around. Some buildings might actually have a common area where the general public can walk around freely and access various bits of information, like the lobby of a bank or insurance company. But on the floors above the lobby are offices which require special privileges to access, you must have a need to get into these rooms.
Likewise, you might have a government building where the first floor might contain a post office or some other public service agency that anyone can access. But the floors above it could contain other types of offices where admission is restricted, or accessed by invitation only.
In your downtown area, how many of the buildings can you walk around freely, and how many have controlled access? Are there buildings that you can not walk around in at all because they are privately owned and don't allow access to the general public?
I could expand the analogy further, but hopefully you start to see that in the "real world" of your downtown area there will places that are open to the public, and other areas with various degrees of access limitations. Likewise in the virtual world of the web, there there will places that are open to the public, and other areas with various degrees of access limitations.
The deep web does not mean some dark and mysterious place of evil, it is simply a term describing an area of controlled access rather than free and open access.
What is the dark web and how do you access it?
Going back to the analogy that the deep web represents the buildings in your town that don't allow access to the general public, the dark web represents all the back alley doorways that are not clearly marked and are accessed by knowing what to say to the doorman to gain access to what is inside.
The worldwide network known as “the dark web”uses specially configured servers designed to work with custom configured web browsers with the purpose of hiding your identity. You will see the term Tor servers and web browers to describe this private network. Tor originally stood for "The Onion Router."
Tor receives funding from the American government but operates as an independent nonprofit organization. The dark web is an interesting place as described in a Washington Post article that explains how the NSA is working around the clock to undermine Tor's anonymity while other branches of the federal government are helping fund it.(3)
A Wired article explains how WikiLeaks was launched with documents intercepted from Tor.(4) You can follow this link to an interview with former government contractor Edward Snowden (5) explaining how Tor is used to create private communications channel.
What can you find on the dark net?
The mysterious dark web, sometimes called the dark net, is the fuel for spy movies. it helped to create WikiLeaks run by the super spy Julian Assange and it allows cyber snitches like Edward Snowden share secret information.
Because the dark net is hidden, and the people that are hiding are doing their best not to be found, knowing the what goes on in the dark can be as mysterious as the name implies. For example one study that claims that nearly half of the sites on the dark net are not doing anything illegal.(6) But a different study that claims that 80% of dark net traffic is related to child abuse and porn sites.(7)
Various names have been used to describe the dark net such as the black internet, to suggest it is the home of online black markets. And the claims of the black internet are supported when a well know online drug black market gets busted. (8)
But does anyone really know what we could find on the dark net? What could you find in your city if you started knocking on doors in dark alleys? Would you want to guess?
(1) Insider Trading On The Dark Web https://www.forbes.com/sites/realspin/2014/03/25/insider-trading-on-the-dark-web/
(2) Clearing Up Confusion – Deep Web vs. Dark Web. https://brightplanet.com/2014/03/clearing-confusion-deep-web-vs-dark-web/
(3) The NSA is trying to crack Tor. The State Department is helping pay for it.
(4) WikiLeaks Was Launched With Documents Intercepted From Tor
(5) This is What a Tor Supporter Looks Like: Edward Snowden
(6) Research suggests the dark web is not as dark as we think
(7) Study claims more than 80% of 'dark net' traffic is to child abuse sites
(8) "End Of The Silk Road: FBI Says It's Busted The Web's Biggest Anonymous Drug Black Market"