Surface Web, Deep Web, And Dark Web Explained

Blog
  • 1631810200206-human1.jpg.webp
    When you think about the internet, what’s the first thing that comes to mind ? Online shopping ? Gaming ? Gambling sites ? Social media ? Each one of these would certainly fall into the category of what requires internet access to make possible, and it would be almost impossible to imagine a life without the web as we know it today. However, how well do we really know the internet and its underlying components ?

    Let’s first understand the origins of the Internet

    The “internet” as we know it today in fact began life as a product called ARPANET. The first workable version came in the late 1960s and used the acronym above rather than the less friendly “Advanced Research Projects Agency Network”. The product was initially funded by the U.S. Department of Defense, and used early forms of packet switching to allow multiple computers to communicate on a single network - known today as a LAN (Local Area Network).

    The internet itself isn’t one machine or server. It’s an enormous collection of networking components such as switches, routers, servers and much more located all over the world - all contacted using common “protocols” (a method of transport which data requires to reach other connected entities) such as TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). Both TCP and UDP use a principle of “ports” to create connections, and ultimately each connected device requires an internet address (known as an IP address which is unique to each device meaning it can be identified individually amongst millions of other inter connected devices).

    In 1983, ARPANET began to leverage the newly available TCP/IP protocol which enabled scientists and engineers to assemble a “network of networks” that would begin to lay the foundation in terms of the required framework or “web” for the internet as we know it today to operate on. The icing on the cake came in 1990 when Tim Berners-Lee created the World Wide Web (www as we affectionately know it) - effectively allowing websites and hyperlinks to work together to form the internet we know and use daily.

    However, over time, the internet model changed as a result of various sites wishing to remain outside of the reach of search engines such as Google, Bing, Yahoo, and the like. This method also gave content owners a mechanism to charge users for access to content - referred to today as a “Paywall”. Out of this new model came, effectively, three layers of the internet.

    Three “Internets” ?

    To make this easier to understand (hopefully), I’ve put together the below diagram
    1626271657-557191-interneticeberg.webp

    The “Surface Web”

    Ok - with the history lesson out of the way, we’ll get back to the underlying purpose of this article, which is to reveal the three “layers” of the internet. For a simple paradigm, the easiest way to explain this is to use the “Iceberg Model”.

    The “internet” that forms part of our everyday lives consists of sites such as Google, Bing, Yahoo (to a lesser extent) and Wikipedia (as common examples - there are thousands more).

    The “Deep Web”

    The next layer is known as the “Deep Web” which typically consists of sites that do not expose themselves to search engines, meaning they cannot be “crawled” and will not feature in Google searches (in the sense that you cannot access a direct link without first having to login). Sites covered in this category - those such as Netflix, your Amazon or eBay account, PayPal, Google Drive, LinkedIn (essentially, anything that requires a login for you to gain access)

    The “Dark Web”

    The third layer down is known as the “Dark Web” - and it’s “Dark” for a reason. These are sites that truly live underground and out of reach for most standard internet users. Typically, access is gained via a TOR (The Onion Router - a bit more about that later) enabled browser, with links to websites being made up of completely random characters (and changing often to avoid detection), with the suffix of .onion. If I were asked to describe the Dark Web, I’d describe it as an underground online marketplace where literally anything goes - and I literally mean “anything”.

    Such examples are

    • Ransomware
    • Botnets,
    • Biitcoin trading
    • Hacker services and forums
    • Financial fraud
    • Illegal pornography
    • Terrorism
    • Anonymous journalism
    • Drug cartels (including online marketplaces for sale and distribution - a good example of this is Silk Road and Silk Road II)
    • Whistleblowing sites
    • Information leakage sites (a bit like Wikileaks, but often containing information that even that site cannot obtain and make freely available)
    • Murder for hire (hitmen etc.)

    Takeaway

    The Surface, Dark, and Deep Web are in fact interconnected. The purpose of these classifications is to determine where certain activities that take place on the internet fit. While internet activity on the Surface Web is, for the most part, secure, those activities in the Deep Web are hidden from view, but not necessarily harmful by nature. It’s very different in the case of the Dark Web. Thanks to it’s (virtually) anonymous nature little is known about the true content. Various attempts have been made to try and “map” the Dark Web, but given that URLs change frequently, and generally, there is no trail of breadcrumbs leading to the surface, it’s almost impossible to do so.
    In summary, the Surface Web is where search engine crawlers go to fetch useful information. By direct contrast, the Dark Web plays host to an entire range of nefarious activity, and is best avoided for security concerns alone.

  • @phenomlab some months ago I remember that I’ve take a look to the dark web……and I do not want to see it anymore….

    It’s really….dark, the content that I’ve seen scared me a lot……

  • @justoverclock yes, completely understand that. It’s a haven for criminal gangs and literally everything is on the table. Drugs, weapons, money laundering, cyber attacks for rent, and even murder for hire.

    Nothing it seems is off limits. The dark web is truly a place where the only limitation is the amount you are prepared to spend.


  • 6 Votes
    7 Posts
    154 Views

    @phenomlab

    yep but I use it since several month and I haven’t see any bugs or crash
    In any case, I only use him anymore 🙂

    Tabby offers tabs and a panel system, but also themes, plugins and color palettes to allow you to push the experience to the limit. It can support different shells in the same window, offers completion, has an encrypted container for your passwords, SSH keys and other secrets, and can handle different connection profiles.

    Each tab is persistent (you can restore them if you close one by mistake) and has a notification system, which will let you know if, for example, a process is finished while you are tapping in another tab.

    It’s really a great terminal that will easily replace cmd.exe for Windowsians or your usual terminal. And it can even work in a portable version for those who like to carry their tools on a USB key.

    –> To test it, you can download it, but there is also a web version. Handy for getting an idea.

    https://app.tabby.sh

  • 4 Votes
    8 Posts
    278 Views

    @phenomlab
    Sorry to delay in responding, yes as i mentioned above, i had to remove my redis from docker and reinstall a new image with this command

    docker run --name=redis -p 127.0.0.1:6379:6379 -d -t redis:alpine

    and now when i test my ip and port on
    https://www.yougetsignal.com/tools/open-ports/

    the status of my redis port is closed. I think which to configure firewall in droplet digital ocean is a good idea too, and i will configure soon.
    Thanks for the help!

  • Blog Setup

    Solved Customisation
    17
    8 Votes
    17 Posts
    421 Views

    Here is an update. So one of the problems is that I was coding on windows - duh right? Windows was changing one of the forward slashes into a backslash when it got to the files folder where the image was being held. So I then booted up my virtualbox instance of ubuntu server and set it up on there. And will wonders never cease - it worked. The other thing was is that there are more than one spot to grab the templates. I was grabbing the template from the widget when I should have been grabbing it from the other templates folder and grabbing the code from the actual theme for the plugin. If any of that makes sense.

    I was able to set it up so it will go to mydomain/blog and I don’t have to forward it to the user/username/blog. Now I am in the process of styling it to the way I want it to look. I wish that there was a way to use a new version of bootstrap. There are so many more new options. I suppose I could install the newer version or add the cdn in the header, but I don’t want it to cause conflicts. Bootstrap 3 is a little lacking. I believe that v2 of nodebb uses a new version of bootstrap or they have made it so you can use any framework that you want for styling. I would have to double check though.

    Thanks for your help @phenomlab! I really appreciate it. I am sure I will have more questions so never fear I won’t be going away . . . ever, hahaha.

    Thanks again!

  • Nodebb as blogging platform

    General
    10
    5 Votes
    10 Posts
    315 Views

    @qwinter I’ve extensive experience with Ghost, so let me know if you need any help.

  • 1 Votes
    1 Posts
    148 Views

    1622031373927-headers-min.webp

    It surprises me (well, actually, dismays me in most cases) that new websites appear online all the time who have clearly spent an inordinate amount of time on cosmetics / appearance, and decent hosting, yet failed to address the elephant in the room when it comes to actually securing the site itself. Almost all the time, when I perform a quick security audit using something simple like the below

    https://securityheaders.io

    I often see something like this

    Not a pretty sight. Not only does this expose your site to unprecedented risk, but also looks bad when others decide to perform a simple (and very public) check. Worse still is the sheer number of so called “security experts” who claim to solve all of your security issues with their “silver bullet” solution (sarcasm intended), yet have neglected to get their own house in order. So that can you do to resolve this issue ? It’s actually much easier than it seems. Dependant on the web server you are running, you can include these headers.

    Apache <IfModule mod_headers.c> Header set X-Frame-Options "SAMEORIGIN" header set X-XSS-Protection "1; mode=block" Header set X-Download-Options "noopen" Header set X-Content-Type-Options "nosniff" Header set Content-Security-Policy "upgrade-insecure-requests" Header set Referrer-Policy 'no-referrer' add Header set Feature-Policy "geolocation 'self' https://yourdomain.com" Header set Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=()" Header set X-Powered-By "Whatever text you want to appear here" Header set Access-Control-Allow-Origin "https://yourdomain.com" Header set X-Permitted-Cross-Domain-Policies "none" Header set Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" </IfModule> NGINX add_header X-Frame-Options "SAMEORIGIN" always; add_header X-XSS-Protection "1; mode=block"; add_header X-Download-Options "noopen" always; add_header X-Content-Type-Options "nosniff" always; add_header Content-Security-Policy "upgrade-insecure-requests" always; add_header Referrer-Policy 'no-referrer' always; add_header Feature-Policy "geolocation 'self' https://yourdomain.com" always; add_header Permissions-Policy "geolocation=(),midi=(),sync-xhr=(),microphone=(),camera=(),magnetometer=(),gyroscope=(),fullscreen=(self),payment=();"; add_header X-Powered-By "Whatever text you want to appear here" always; add_header Access-Control-Allow-Origin "https://yourdomain.com" always; add_header X-Permitted-Cross-Domain-Policies "none" always; add_header Strict-Transport-Security "max-age=63072000; includeSubdomains;" always;

    Note, that https://yourdomain.com should be changed to reflect your actual domain. This is just a placeholder to demonstrate how the headers need to be structured.

    Restart Apache or NGINX, and then perform the test again.


    That’s better !

    More detail around these headers can be found here

    https://webdock.io/en/docs/how-guides/security-guides/how-to-configure-security-headers-in-nginx-and-apache

  • 0 Votes
    1 Posts
    140 Views

    1622032930658-hacked_listen-min.webp

    I’ve been a veteran of the infosec industry for several years, and during that time, I’ve been exposed to a wide range of technology and situations alike. Over this period, I’ve amassed a wealth of experience around information security, physical security, and systems. 18 years of that experience has been gained within the financial sector - the remaining spread across manufacturing, retail, and several other areas. I’ve always classed myself as a jack of all trades, and a master of none. The real reason for this is that I wanted to gain as much exposure to the world of technology without effectively “shoehorning” myself - pigeon holing my career and restricting my overall scope.

    I learned how to both hack and protect 8086 / Z80 systems back in 1984, and was using “POKE” well before Facebook coined the phrase and made it trendy (one of the actual commands I still remember to this day that rendered the CTRL, SHIFT, ESC break sequence useless was

    POKE &bdee, &c9

    I spent my youth dissecting systems and software alike, understanding how they worked, and more importantly, how easily they could be bypassed or modified.

    Was I a hacker in my youth ? If you understand the true meaning of the word, then yes - I most definitely was.

    If you think a hacker is a criminal, then absolutely not. I took my various skills I obtained over the years, honed them, and made them into a walking information source - a living, breathing technology encyclopedia that could be queried simply by asking a question (but not vulnerable to SQL injection).

    Over the years, I took an interest in all forms of technology, and was deeply immersed in the “virus era” of the 2000’s. I already understood how viruses worked (after dissecting hundreds of them in a home lab), and the level of damage that could be inflicted by one paved the way for a natural progression to early and somewhat infantile malware. In its earliest form, this malware was easily spotted and removed. Today’s campaigns see code that will self delete itself post successful execution, leaving little to no trace of its activity on a system. Once the APT (Advanced Persistent Threat) acronym became mainstream, the world and its brother realised they had a significant problem in their hands, and needed to respond accordingly. I’d realised early on that one of the best defences against the ever advancing malware was containment. If you “stem the flow”, you reduce the overall impact - essentially, restricting the malicious activity to a small subset rather than your entire estate.

    I began collaborating with various stakeholders in the organisations I worked for over the years, carefully explaining how modern threats worked, the level of damage they could inflict initially from an information and financial perspective, extending to reputation damage and a variety of others as campaigns increased in their complexity). I recall one incident during a tenure within the manufacturing industry where I provided a proof of concept. At the time, I was working as a pro bono consultant for a small company, and I don’t think they took me too seriously.

    Using an existing and shockingly vulnerable Windows 2003 server (it was still using the original settings in terms of configuration - they had no patching regime, meaning all systems were effectively vanilla) I exhibited how simple it would be to gain access first to this server, then steal the hash - effortlessly using that token to gain full access to other systems without even knowing the password (pass the hash). A very primitive exercise by today’s standards, but effective nonetheless. I explained every step of what I was doing along the way, and then explained how to mitigate this simple exploit - I even provided a step by step guide on how to reproduce the vulnerability, how to remediate it, and even provided my recommendations for the necessary steps to enhance security across their estate. Their response was, frankly, shocking. Not only did they attempt to refute my findings, but at the same time, dismissed it as trivial - effectively brushing it under the carpet so to speak. This wasn’t a high profile entity, but the firm in question was AIM listed, and by definition, were duty bound - they had a responsibility to shareholders and stakeholders to resolve this issue. Instead, they remained silent.

    Being Pro Bono meant that my role was an advisory one, and I wasn’t charging for my work. The firm had asked me to perform a security posture review, yet somehow, didn’t like the result when it was presented to them. I informed them that they were more than welcome to obtain another opinion, and should process my findings as they saw fit. I later found out through a mutual contact that my findings had been dismissed as "“unrealistic”, and another consultant had certified their infrastructure as “safe”. I almost choked on my coffee, but wrote this off as a bad experience. 2 months later, I got a call from the same mutual contact telling me that my findings were indeed correct. He had been contacted by the same firm asking him to provide consultancy for what on the face of it, looked like a compromised network.

    Then came the next line which I’ll never forget.

    “I don’t suppose you’d be interested in……”

    I politely refused, saying I was busy on another project. I actually wasn’t, but refused out of principle. And so, without further ado, here’s my synopsis

    “…if you choose not to listen to the advice a security expert gives you, then you are leaving yourself and your organisation unnecessarily vulnerable. Ignorance is not bliss when it comes to security…”

    Think about what you’ve read for a moment, and be honest with me - say so if you think this statement is harsh given the previous content.

    The point I am trying to make here is that despite sustained effort, valiant attempts to raise awareness, and constantly telling people they have gaping holes in systems for them to ignore the advice (and the fix I’ve handed to them on a plate) is extremely frustrating. Those in the InfoSec community are duty bound to responsibly disclose, inform, educate, raise awareness, and help protect, but that doesn’t extend to wiping people’s noses and telling them it wasn’t their fault that they failed to follow simple advice that probably could have prevented their inevitable breach. My response here is that if you bury your head in the sand, you won’t see the guy running up behind you intent on kicking you up the ass.

    Security situations can easily be avoided if people are prepared to actually listen and heed advice. I’m willing to help anyone, but they in return have to be equally willing to listen, understand, and react.

  • 2 Votes
    6 Posts
    252 Views

    @kurulumu-net CSS styling is now addressed and completed.

  • 0 Votes
    1 Posts
    158 Views

    1631812610135-security1.webp
    The recent high profile breaches impacting organisations large and small are a testament to the fact that no matter how you secure credentials, they will always be subject to exploit. Can a password alone ever be enough ? in my view, it’s never enough. The enforced minimum should be at least with a secondary factor. Regardless of how “secure” you consider your password to be, it really isn’t in most cases – it just “complies” with the requirement being enforced.

    Here’s classic example. We take the common password of “Welcome123” and put it into a password strength checker
    1564764162-304322-password1.png
    According to the above, it’s “strong”. Actually, it isn’t. It’s only considered this way because it meets the complexity requirements, with 1 uppercase letter, at least 8 characters, and numbers. What’s also interesting is that a tool sponsored by Dashlane considers the same password as acceptable, taking supposedly 8 months to break
    1564764192-579936-password2.png
    How accurate is this ? Not accurate at all. The password of “Welcome123” is in fact one of the passwords contained in any penetration tester’s toolkit – and, by definition, also used by cyber criminals. As most of this password combination is in fact made up of a dictionary word, plus sequential numbers, it would take less than a second to break this rather than the 8 months reported above. Need further evidence of this ? Have a look at haveibeenpwned, which will provide you with a mechanism to test just how many times “Welcome123” has appeared in data breaches
    1564764241-350631-hibp.png

    Why are credentials so weak ?

    My immediate response to this is that for as long as humans have habits, and create scenarios that enable them to easily remember their credentials, then this weakness will always exist. If you look at a sample taken from the LinkedIn breach, those passwords that occupy the top slots are arguably the least secure, but the easiest to remember from the human perspective. Passwords such as “password” and “123456” may be easy for users to remember, but on the flip side, weak credentials like this can be broken by a simple dictionary attack in less than a second.

    Here’s a selection of passwords still in use today – hopefully, yours isn’t on there
    1564764251-257407-passwordlist.jpeg
    We as humans are relatively simplistic when it comes to credentials and associated security methods. Most users who do not work in the security industry have little understanding, desire to understand, or patience, and will naturally choose the route that makes their life easier. After all, technology is supposed to increase productivity, and make tasks easier to perform, right ? Right. And it’s this exact vulnerability that a cyber criminal will exploit to it’s full potential.

    Striking a balance between the security of credentials and ease of recall has always had it’s challenges. A classic example is that networks, websites and applications nowadays typically have password policies in place that only permit the use of a so-called strong password. Given the consolidation and overall assortment of letters, numbers, non-alphanumeric characters, uppercase and lowercase, the password itself is probably “secure” to an acceptable extent, although the method of storing the credentials isn’t. A shining example of this is the culture of writing down sensitive information such as credentials. I’ve worked in some organisations where users have actually attached their password to their monitor. Anyone looking for easy access into a computer network is onto an immediate winner here, and unauthorised access or a full blown breach could occur within an alarmingly short period of time.

    Leaked credentials and attacks from within

    You could argue that you would need access to the computer itself first, but in several historical breach scenarios, the attack originated from within. In this case, it may not be an active employee, but someone who has access to the area where that particular machine is located. Any potential criminal has the credentials – well, the password itself, but what about the username ? This is where a variety of techniques can be used in terms of username discovery – in fact, most of them being non-technical – and worryingly simple to execute. Think about what is usually on a desk in an office. The most obvious place to look for the username would be on the PC itself. If the user had recently logged out, or locked their workstation, then on a windows network, that would give you the username unless a group policy was in place. Failing that, most modern desk phones display the name of the user. On Cisco devices, under Extension Mobility, is the ID of the user. It doesn’t take long to find this. Finally there’s the humble business card. A potential criminal can look at the email address format, remove the domain suffix, and potentially predict the username. Most companies tend to leverage the username in email addresses mainly thanks to SMTP template address policies – certainly true in on premise Exchange environments or Office 365 tenants.

    The credentials are now paired. The password has been retrieved in clear text, and by using a simple discovery technique, the username has also been acquired. Sometimes, a criminal can get extremely lucky and be able to acquire credentials with minimal effort. Users have a habit of writing down things they cannot recall easily, and in some cases, the required information is relatively easily divulged without too much effort on the part of the criminal. Sounds absurd and far fetched, doesn’t it ? Get into your office early, or work late one evening, and take a walk around the desks. You’ll be unpleasantly surprised at what you will find. Amongst the plethora of personal effects such as used gym towels and footwear, I guarantee that you will find information that could be of significant use to a criminal – not necessarily readily available in the form of credentials, but sufficient information to create a mechanism for extraction via an alternative source. But who would be able to use such information ?

    Think about this for a moment. You generally come into a clean office in the mornings, so cleaners have access to your office space. I’m not accusing anyone of anything unscrupulous or illegal here, but you do need to be realistic. This is the 21st century, and as a result, it is a security measure you need to factor in and adopt into your overall cyber security policy and strategy. Far too much focus is placed on securing the perimeter network, and not enough on the threat that lies within. A criminal could get a job as a cleaner at a company, and spend time collecting intelligence in terms of what could be a vulnerability waiting to be exploited. Another example of “instant intelligence” is the network topology map. Some of us are not blessed with huge screens, and need to make do with one ancient 19″ or perhaps two. As topology maps can be quite complex, it’s advantageous to be able to print these in A3 format to make it easier to digest. You may also need to print copies of this same document for meetings. The problem here is what you do with that copy once you have finished with it ?

    How do we address the issue ? Is there sufficient awareness ?

    Yes, there is. Disposing of it in the usual fashion isn’t the answer, as it can easily be recovered. The information contained in most topology maps is often extensive, and is like a goldmine to a criminal looking for intelligence about your network layout. Anything like this is classified information, and should be shredded at the earliest opportunity. Perhaps one of the worst offences I’ve ever personally experienced is a member of the IT team opening a password file, then walking away from their desk without locking their workstation. To prove a point about how easily credentials can be inadvertently leaked, I took a photo with a smartphone, then showed the offender what I’d managed to capture a few days later. Slightly embarrassed didn’t go anywhere near covering it.

    I’ve been an advocate of securing credentials for some time, and recently read about the author of “NIST Special Publication 800-63” (Bill Burr). Now retired, he has openly admitted the advice he originally provided as in fact, incorrect

    “Much of what I did I now regret.” said Mr Burr, who advised people to change their password every 90 days and use obscure characters.

    “It just drives people bananas and they don’t pick good passwords no matter what you do.”

    The overall security of credentials and passwords

    However, bearing in mind that this supposed “advice” has long been the accepted norm in terms of password securuty, let’s look at the accepted standards from a well-known auditing firm

    It would seem that the Sarbanes Oxley 404 act dictates that regular changes of credentials are mandatory, and part of the overarching controls. Any organisation that is regulated by the SEC (for example) would be covered and within scope by this statement, and so the argument for not regularly changing your password becomes “invalid” by the act definition and narrative. My overall point here is that the clearly obvious bad password advice in the case of the financial services industry is negated by a severely outdated set of controls that require you to enforce a password change cycle and be in compliance with it. In addition, there are a vast number of sites and services that force password changes on a regular basis, and really do not care about what is known to be extensive research on password generation.

    The argument for password security to be weakened by having to change it on a frequent basis is an interesting one that definitely deserves intense discussion and real-world examples, but if your password really is strong (as I mentioned previously, there are variations of this which are really not secure at all, yet are considered strong because they meet a complexity requirement), then a simple mutation of it could render it vulnerable. I took a simple lowercase phrase

    mypasswordissimpleandnotsecureatall

    1564764311-893646-nonillion.png
    The actual testing tool can be found here. So, does a potential criminal have 26 nonillion years to spare ? Any cyber criminal who possesses only basic skills won’t need a fraction of that time as this password is in fact made up of simple dictionary words, is all lowercase, and could in fact be broken in seconds.

    My opinion ? Call it how you like – the password is here to stay for the near future at least. The overall strength of the password or credentials stored using MD5, bCrypt, SHA1 and so on are irrelevant when an attacker can use established and proven techniques such as social engineering to obtain your password. Furthermore, the addition of 2FA or a SALT dramatically increases password security – as does the amount of unsuccessful attempts permitted before the associated account is locked. This is a topic that interests me a great deal. I’d love to hear your feedback and comments.