Concise Bytes: What is the Deep Web?

Concise Bytes: What is the Deep Web?

Henry Dalziel | Concise Bytes | October 19, 2013

This is part of our Concise Bytes Series where we take a look at common InfoSec concepts or indeed any important IT technology, concept or process that can be associated within Information Security. Many within our community, and visitors to our blog, are students hence the reason why we are offering this ‘Bite Size’ resources section! If you are a professional and know the answer then we’d really appreciate your comments below to correct or append this post!

This Concise Bytes blog post will discuss and give a very brief overview of the Deep Web!

What is the Deep Web? The Deep Web (or the ‘Dark Web’ as it is often incorrectly associated with) has received a lot of press recently due to the take down of the Silk Road website. The Silk Road never had a traditional ‘Silk Road’ dot com domain, rather, they had a .onion domain which can only be accessed via the Tor Network – more on that later. Anyways – the main purpose and inspiration of this particular Concise Byte post was the interest that has been generated by the Silk Road and the mysterious ‘Deep Web’!

The ‘Deep Web’, ‘Deepnet’ or the ‘Hidden Web’ is content that is not part of the ‘normal Internet’. By ‘normal Internet’ we are referring to any content that can be (and is able to be) indexed by Search Engine spiders and bots. The ‘Deep Web’ – as it will be referred to in this post, has been around forever, in fact, it’s been around longer than the Internet.

If you do any research into this subject – which by the way you really should because it is absolutely fascinating – you will see many analogies of the ‘Deep Web’ with the Ocean and for good reasons, not least because it makes it a whole lot easier to understand. Think of it like this: Google, Bing or Yahoo – drag their fishing nets across the surface of the ocean and haul in their enormous catch of fish. They then proceed to categorize, rank, and index the ‘fish’ that they have caught. ‘Fish’ refers to ‘content’ of course! Just as fisherman will toss overboard trash their nets accidentally caught, so will the search engines ‘toss overboard’ spam. Discussing spam here is a slight digression – so back to the nets and ocean – and the Deep Web! The major search engines’ nets can only reach a certain depth – say 100 meters – and there is plenty of fish at that depth that keeps the Internet alive and what it is today. However, there are more, a lot more, fish that the Google fishing nets simply can’t reach. Those ‘fish’ belong and live in the Deep Ocean – also known as, the ‘Deep Web.’ made an excellent 30 minute video demonstration on the Deep Web titled: “The Hidden Internet – Exploring The Deep Web” which we highly recommend.

Accessing The Deep Web!

Again – just to emphasize, go ahead and watch the video (link above) because it really does an excellent job at explaining the Deep Web, how it works and how to access it. However – if you’d rather read our blog post then here’s a brief overview of how you can access the Deep Web. As mentioned in the first section of this post, the Internet, as we know it, uses the http protocol through known port numbers to trawl and index content. These same processes are, for the most part, unable to scan and access the Deep Web. (If you are interested in search engines that can, or have attempted to access the Deep Web, you should check out projects like DeepPeep, Intute and Scirus (which is actually set to retire in early 2014).

So – how do we access the Deep Web?

Tor (short for The Onion Router) is your friend. There are other ways but in essence, Tor is the best known. Tor, also referred to as “Onion Routing” is structured around layers of encryption to access data. You can read more about Tor here – but a simple search on YouTube and you’ll find a plethora of resources.

In summary
A post about the Deep Web, in our opinion, must always be accompanied with a health warning. Owing to the ‘secretive’ nature of the Deep Web some have taken this as an opportunity to hide vile content, so please be careful about that.

Do you access the Deep Web and if you don’t mind sharing with us, what for? Let us know, we’d love to hear from you. Also, any tips, advice or general suggestions for this post (critical or otherwise!) are warmly and gratefully received!

Go and explore, have fun – but be safe!

Leave a comment or reply below...thanks!