Installing and Configuring CrowdSec in Cloudflare

Security
  • @phenomlab brings us a lot so here is a modest contribution to thank him and for the pleasure of sharing.

    🎅 Merry Christmas to everyone 🎅

    Installing and Configuring CrowdSec in Cloudflare

    We will see how to install and configure the Cloudflare bouncer to add another layer of security on top of our Cloudflare to protect your website against cyber threats.


    Cloudflare is a CDN (a global web content delivery network) that provides various services that allow among other things to manage DNS zones, distribute web content, secure applications and infrastructure.

    Several types of subscriptions are available. Today I’m going to show what’s achievable with CloudFlare’s free plan.

    CrowdSec is a FOSS security tool that works using collaborative security as users help each other by sharing information about attacks and blocking them both locally and among all other CrowdSec users.

    CrowdSec consists of two parts: the agent which analyzes log files, detects attacks and holds also the local API (lapi) as well as the bouncer which mitigates them. We install both components in this tutorial.

    One of the services that CrowdSec is able to protect is HTTP. Together with Cloudflare, it provides an extra layer of security against attacks such as aggressive mining, scanning/probing, path traversal, access to sensitive data, DDos L7 attacks, SQL injection and more again.

    CrowdSec is able to mitigate threats directly on Cloudflare by using the API provided by Cloudflare to block traffic or force suspicious users to go through a CAPTCHA challenge using the CrowdSec Cloudflare bouncer. We will therefore see how to configure the Cloudflare bouncer.

    Prerequisites

    https://support.cloudflare.com/hc/en-us/articles/200170786-Restoring-original-visitor-IPs

    Configuring the API token in CloudFlare

    First, here we generate a token to authenticate to the CloudFlare API with the right permissions.

    – Go to My Profile/API Tokens:

    01.png

    – Then create a Custom Token:
    02.png

    – Give it the name crowdsec then select the following permissions:

    Account - Account Filter Lists - Edit
    Account - Account Firewall Access - Edit
    Zone - Zone - Read
    Zone - Firewall Services - Edit
    

    03.png

    – Then validate via the Create Token button:
    04.png

    – We have a small summary with the token that is going well. Keep it preciously because impossible to find this one 😉

    25c83a0c-44b6-4afb-a317-635f99fca2cf-image.png

    – So let’s test access to the CloudFlare API to see if everything is OK on that side (Adapt with yourtokens):

    curl -X GET "https://api.cloudflare.com/client/v4/user/tokens/verify" \     
    -H "Authorization: Bearer 5hhTxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx6W0jzk" \     
    -H "Content-Type:application/json"
    

    25c83a0c-44b6-4afb-a317-635f99fca2cf-image.png

    07.png

    Installation and configuration of CrowdSec’s CloudFlare bouncer

    – We install the bouncer:

    sudo apt install crowdsec-cloudflare-bouncer
    

    08.png

    – We will configure the bouncer by editing its configuration file:

    sudo nano etc/crowdsec/bouncers/crowdsec-cloudflare-bouncer.yaml
    
    # CrowdSec Config
    crowdsec_lapi_url: http://localhost:8080/
    crowdsec_lapi_key: 7xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    crowdsec_update_frequency: 10s
    
    #Cloudflare Config. 
    cloudflare_config:
      accounts: 
      - id: [ID]
        token: [TOKEN]
        ip_list_prefix: crowdsec
        default_action: managed_challenge
        zones:
        - actions: 
          - managed_challenge # valid choices are either of managed_challenge, js_challenge, block
          zone_id: [ZONE_ID]
        
    
      update_frequency: 30s # the frequency to update the cloudflare IP list 
    
    # Bouncer Config
    daemon: true
    log_mode: file
    log_dir: /var/log/ 
    log_level: info # valid choices are either debug, info, error 
    log_max_size: 40
    log_max_age: 30
    log_max_backups: 3
    compress_logs: true
    
    prometheus:
      enabled: false
      listen_addr: 127.0.0.1
      listen_port: 2112
    

    – The things to focus on are:

    • [ID]: Account ID of your Cloudflare account

    • [TOKEN]: the token generated by CrowdSec that we kept preciously 🙂

    • [ZONE_ID]: ID of the Cloudflare zone you want to protect

    • prometheus: if we don’t care, let’s turn it off. (log analyzer and graphs)

      !! The information can be found from the “Overview” tab of the area. A more comprehensive guide here

    0588a715-c35f-4c62-888e-7a7584c47d71-image.png 297cf5c4-0eed-4a5d-9c32-9a8b9087247b-image.png

    – We save the file.

    IMPORTANT: Next, as the bouncer documentation states, we run the following command to clean up the existing CloudFlare components created by the bouncer:

    sudo crowdsec-cloudflare-bouncer -d
    

    58435d5e-0692-4265-9d2f-9367694b8910-image.png

    – If you don’t do this, you may get such errors in the bouncer logs:

    time="22-12-2022 01:06:47" level=info msg="Starting crowdsec-cloudflare-bouncer v0.2.1-debian-pragmatic-6b30687c25xxxxxxxxxxxxxxxxxxx59"
    time="22-12-2022 01:06:47" level=info msg="Using API key auth"
    time="22-12-2022 01:06:49" level=info msg="using existing  ip list crowdsec_managed_challenge 221xxxxxxxxxxxxxxxxxxxxxxxx4" account_id=53xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    time="22-12-2022 01:06:51" level=error msg="error filter at index (0) conflicts with an existing filter (10202) in creating firewall rule (ip.src in $crowdsec_managed_challenge)" account_id=53xxxxxxxxxxxxxxxxxxxxxxx zone_id=30xxxxxxxxxxxxxxxxxxxxxxxx
    time="22-12-2022 01:06:51" level=error msg="filter at index (0) conflicts with an existing filter (10202)" account_id=533xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    time="22-12-2022 01:06:51" level=error msg="filter at index (0) conflicts with an existing filter (10202)" account_id=533xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    time="22-12-2022 01:06:51" level=fatal msg="filter at index (0) conflicts with an existing filter (10202)"
    

    – And you will not be able to empty the list present in CloudFlare:

    b9a98a71-5083-4cc7-bea2-483ede980262-image.png

    f969908a-87f4-433b-8529-5a8c0699d916-image.png

    – Now, we restart the bouncer so that they can populate the IP address list and the WAF Firewall rule in CloudFlare:

    systemctl stop crowdsec-cloudflare-bouncer
    systemctl start crowdsec-cloudflare-bouncer
    
    

    74936946-e317-482a-a484-8eb11623d2e8-image.png

    a2488981-f24d-454a-8dc7-6ca40b78a014-image.png

    Managing multiple zones

    If we want to manage multiple zones, we need to slightly modify the configuration file:

    # Cloudflare Config.
    cloudflare_config:
      accounts:
      - id: [ID]
        token: [TOKEN]
        ip_list_prefix: crowdsec
        default_action: challenge
        zones:
          - zone_id: [ZONE1_ID]
            actions:
            - challenge
          - zone_id: [ZONE2_ID]
            actions:
            - challenge
          [...]
    

    – Assuming the zones belong to the same account.

    – If you have already started the service, you will find the cache file /var/lib/crowdsec/crowdsec-cloudflare-bouncer/cache/cloudflare-cache.json

    – I find it better to stop the service, delete the cache file and restart it, so as to force a configuration to reload.


    – Sources :

    https://www.crowdsec.net/blog/installing-and-configuring-crowdsec-into-cloudflare

    https://docs.crowdsec.net/docs/bouncers/cloudflare/

  • DownPWundefined DownPW marked this topic as a regular topic on
  • @DownPW thanks very much for this incredibly detailed guide. Anyone else reading this should certainly consider deploying this as part of their security platform.

    Remember - security isn’t a product, but a combination of best practice, common sense, strategy, and learning from others in the same situation. What better way to fortify your own castle with input and knowledge from a wider community?

    Knowledge is power, and so much more powerful when shared.

  • @phenomlab

    you are incredibly right. It’s just a stone to add on all the measures to be taken.

  • @DownPW yeah, I seem to spend a large amount of my time trying to educate people that there’s no silver bullet when it comes to security.


  • 0 Votes
    4 Posts
    152 Views

    @DownPW 🙂 most of this really depends on your desired security model. In all cases with firewalls, less is always more, although it’s never as clear cut as that, and there are always bespoke ports you’ll need to open periodically.

    Heztner’s DDoS protection is superior, and I know they have invested a lot of time, effort, and money into making it extremely effective. However, if you consider that the largest ever DDoS attack hit Cloudflare at 71m rps (and they were able to deflect it), and each attack can last anywhere between 8-24 hours which really depends on how determined the attacker(s) is/are, you can never be fully prepared - nor can you trace it’s true origin.

    DDoS attacks by their nature (Distributed Denial of Service) are conducted by large numbers of devices whom have become part of a “bot army” - and in most cases, the owners of these devices are blissfully unaware that they have been attacked and are under command and control from a nefarious resource. Given that the attacks originate from multiple sources, this allows the real attacker to observe from a distance whilst concealing their own identity and origin in the process.

    If you consider the desired effect of DDoS, it is not an attempt to access ports that are typically closed, but to flood (and eventually overwhelm) the target (such as a website) with millions of requests per second in an attempt to force it offline. Victims of DDoS attacks are often financial services for example, with either extortion or financial gain being the primary objective - in other words, pay for the originator to stop the attack.

    It’s even possible to get DDoS as a service these days - with a credit card, a few clicks of a mouse and a target IP, you can have your own proxy campaign running in minutes which typically involves “booters” or “stressers” - see below for more

    https://heimdalsecurity.com/blog/ddos-as-a-service-attacks-what-are-they-and-how-do-they-work

    @DownPW said in Setting for high load and prevent DDoS (sysctl, iptables, crowdsec or other):

    in short if you have any advice to give to secure the best.

    It’s not just about DDos or firewalls. There are a number of vulnerabilities on all systems that if not patched, will expose that same system to exploit. One of my favourite online testers which does a lot more than most basic ones is below

    https://www.immuniweb.com/websec/

    I’d start with the findings reported here and use that to branch outwards.

  • 4 Votes
    4 Posts
    73 Views

    @phenomlab said in TikTok fined £12.7m for misusing children’s data:

    Just another reason not to use TikTok. Zero privacy, Zero respect for privacy, and Zero controls in place.

    https://news.sky.com/story/tiktok-fined-12-7m-for-data-protection-breaches-12849702

    The quote from this article says it all

    TikTok should have known better. TikTok should have done better

    They should have, but didn’t. Clearly the same distinct lack of core values as Facebook. Profit first, privacy… well, maybe.

    Wow, that’s crazy! so glad I stayed away from it, rotten to the core.

  • 5 Votes
    4 Posts
    72 Views

    @DownPW here. Hostrisk is automated and doesn’t accept registrations.

  • 6 Votes
    7 Posts
    154 Views

    @phenomlab

    yep but I use it since several month and I haven’t see any bugs or crash
    In any case, I only use him anymore 🙂

    Tabby offers tabs and a panel system, but also themes, plugins and color palettes to allow you to push the experience to the limit. It can support different shells in the same window, offers completion, has an encrypted container for your passwords, SSH keys and other secrets, and can handle different connection profiles.

    Each tab is persistent (you can restore them if you close one by mistake) and has a notification system, which will let you know if, for example, a process is finished while you are tapping in another tab.

    It’s really a great terminal that will easily replace cmd.exe for Windowsians or your usual terminal. And it can even work in a portable version for those who like to carry their tools on a USB key.

    –> To test it, you can download it, but there is also a web version. Handy for getting an idea.

    https://app.tabby.sh

  • 5 Votes
    6 Posts
    653 Views

    Missed out on this deal ? Windscribe offer a limited free version. More about that here
    https://sudonix.org/topic/13/which-product-is-the-best-for-vpn/164?_=1652206628456

  • 4 Votes
    3 Posts
    488 Views

    @phenomlab

    No they have a free and pro console instance.
    We can see alert with IP, Source AS, scenario attack etc…

    Installation on the NODEBB server without problems. Very good tools

    cf7e5a89-84f4-435b-82eb-434c0bfc895e-image.png
    cc82a10e-a1f1-4fd8-a433-7c9b2d31f254-image.png

    1b7147b0-37c6-4d87-b4f1-a0fe92e74afd-image.png

    7c21fc10-1825-48e1-a993-92b84455f074-image.png


    We can also do research on IPs via the crowdsec analyzer

    I believe it’s 500 per month in the Free version

    43bc8265-a57c-4439-829c-0bb8602d99b4-image.png

  • 0 Votes
    1 Posts
    134 Views

    When you look at your servers or surrounding networks, what exactly do you see ? A work of art, perhaps ? Sadly, this is anything but the picture painted for most networks when you begin to look under the hood. What I’m alluding to here is that beauty isn’t skin deep - in the sense that neat cabling resembling art from the Sistine Chapel, tidy racks, and huge comms rooms full of flashing lights looks appealing from the eye candy perspective and probably will impress clients, but in several cases, this is the ultimate wolf in sheep’s clothing. Sounds harsh ? Of course it does, but with good intentions and reasoning. There’s not a single person responsible for servers and networks on this planet who will willingly admit that whilst his or her network looks like a masterpiece to the untrained eye, it’s a complete disaster in terms of security underneath.

    In reality, it’s quite the opposite. Organisations often purchase bleeding edge infrastructure as a means of leveraging the clear technical advantages, enhanced security, and competitive edge it provides. However, under the impressive start of the art ambience and air conditioning often lies an unwanted beast. This mostly invisible beast lives on low-hanging fruit, will be tempted to help itself at any given opportunity, and is always hungry. For those becoming slightly bewildered at this point, there really isn’t an invisible beast lurking around your network that eats fruit. But, with a poorly secured infrastructure, there might as well be. The beast being referenced here is an uninvited intruder in your network. A bad actor, threat actor, bad guy, criminal…. call it what you want (just don’t use the word hacker) can find their way inside your network by leveraging the one thing that I’ve seen time and time again in literally every organisation I ever worked for throughout my career - the default username and password. I really can’t stress the importance of changing this on new (and existing) network equipment enough, and it doesn’t stop at this either.

    Changing the default username and password is about 10% of the puzzle when it comes to security and basic protection principles. Even the most complex credentials can be bypassed completely by a vulnerability (or in some cases, a backdoor) in ageing firmware on switches, firewalls, routers, storage arrays, and a wealth of others - including printers (which incidentally make an ideal watering hole thanks to the defaults of FTP, HTTP, SNMP, and Telnet, most (if not all of) are usually always on. As cheaper printers do not have screens like their more expensive copier counterparts (the estate is reduced to make the device smaller and cheaper), any potential criminal can hide here and not be detected - often for months at a time - arguably, they could live in a copier without you being aware also. A classic example of an unknown exploit into a system was the Juniper firewall backdoor that permitted full admin access using a specific password - regardless of the one set by the owner. Whilst the Juniper exploit didn’t exactly involve a default username and password as such (although this particular exploit was hard-coded into the firmware, meaning that any “user” with the right coded password and SSH access remotely would achieve full control over the device), it did leverage the specific vulnerability in the fact that poorly configured devices could have SSH configured as accessible to 0.0.0.0/0 (essentially, the entire planet) rather than a trusted set of IP addresses - typically from an approved management network.

    We all need to get out of the mindset of taking something out of a box, plugging it into our network, and then doing nothing else - such as changing the default username and password (ideally disabling it completely and replacing it with a unique ID) or turning off access protocols that we do not want or need. The real issue here is that today’s technology standards make it simple for literally anyone to purchase something and set it up within a few minutes without considering that a simple port scan of a subnet can reveal a wealth of information to an attacker - several of these tools are equipped with a default username and password dictionary that is leveraged against the device in question if it responds to a request. Changing the default configuration instead of leaving it to chance can dramatically reduce the attack landscape in your network. Failure to do so changes “plug and play” to “ripe for picking”, and its those devices that perform seemingly “minor” functions in your network that are the easiest to exploit - and leverage in order to gain access to neighbouring ancillary services. Whilst not an immediate gateway into another device, the compromised system can easily give an attacker a good overview of what else is on the same subnet, for example.

    So how did we arrive at the low hanging fruit paradigm ?

    It’s simple enough if you consider the way that fruit can weigh down the branch to the point where it is low enough to be picked easily. A poorly secured network contains many vulnerabilities that can be leveraged and exploited very easily without the need for much effort on the part of an attacker. It’s almost like a horse grazing in a field next to an orchard where the apples hang over the fence. It’s easily picked, often overlooked, and gone in seconds. When this term is used in information security, a common parallel is the path of least resistance. For example, a pickpocket can acquire your wallet without you even being aware, and this requires a high degree of skill in order to evade detection yet still achieve the primary objective. On the other hand, someone strolling down the street with an expensive camera hanging over their shoulder is a classic example of the low hanging fruit synopsis in the sense that this theft is based on an opportunity that wouldn’t require much effort - yet with a high yield. Here’s an example of how that very scenario could well play out.

    Now, as much as we’d all like to handle cyber crime in this way, we can’t. It’s illegal 🙂

    What isn’t illegal is prevention. 80% of security is based on best practice. Admittedly, there is a fair argument as to what exactly is classed as “best” these days, although it’s a relatively well known fact that patching the Windows operating system for example is one of the best ways to stamp out a vulnerability - but only for that system that it is designed to protect against. Windows is just the tip of the iceberg when it comes to vulnerabilities - it’s not just operating systems that suffer, but applications, too. You could take a Windows based IIS server, harden it in terms of permitted protocols and services, plus install all of the available patches - yet have an outdated version of WordPress running (see here for some tips on how to reduce that threat), or often even worse, outdated plugins that allow remote code execution. The low hanging fruit problem becomes even more obvious when you consider breaches such as the well-publicised Mossack Fonseca (“Panama Papers”). What became clear after an investigation is that the attackers in this case leveraged vulnerabilities in the firm’s WordPress and Joomla public facing installations - this in fact led to them being able to exploit an equally vulnerable mail server by brute-forcing it.

    So what should you do ? The answer is simple. It’s harvest time.

    If there is no low-hanging fruit to pick, life becomes that much more difficult for any attacker looking for a quick “win”. Unless determined, it’s unlikely that your average attacker is going to spend a significant amount of time on a target (unless it’s Fort Knox - then you’ve have to question the sophistication) then walk away empty handed with nothing to show for the effort. To this end, below are my top recommendations. They are not new, non-exhaustive, and certainly not rocket science - yet they are surprisingly missing from the “security 101” perspective in several organisations.

    Change the default username and password on ALL infrastructure. It doesn’t matter if it’s not publicly accessible - this is irrelevant when you consider the level of threats that have their origins from the inside. If you do have to keep the default username (in other words, it can’t be disabled), set the lowest possible access permissions, and configure a strong password. Close all windows - in this case, lock down protocols and ports that are not essential - and if you really do need them open, ensure that they are restricted Deploy MFA (or at least 2FA) to all public facing systems and those that contain sensitive or personally identifiable information Deploy adequate monitoring and logging techniques, using a sane level of retention. Without any way of forensic examination, any bad actor can be in and out of your network well before you even realise a breach may have taken place. The only real difference is that without decent logging, you have no way of confirming or even worse, quantifying your suspicion. This can spell disaster in regulated industries. Don’t shoot yourself in the foot. Ensure all Windows servers and PC’s are up to date with the latest patches. The same applies to Linux and MAC systems - despite the hype, they are vulnerable to an extent (but not in the same way as Windows), although attacks are notoriously more difficult to deploy and would need to be in the form of a rootkit to work properly Do not let routers, firewalls, switches, etc “slip” in terms of firmware updates. Keep yourself and your team regularly informed and updated around the latest vulnerabilities, assess their impact, and most importantly, plan an update review. Not upgrading firmware on critical infrastructure can have a dramatic effect on your overall security. Lock down USB ports, CD/DVD drives, and do not permit access to file sharing, social media, or web based email. This has been an industry standard for years, but you’d be surprised at just how many organisations still have these open and yet, do not consider this a risk. Reduce the attack vector by segmenting your network using VLANS. For example, the sales VLAN does not need to (and shouldn’t need to) connect directly to accounting etc. In this case, a ransomware or malware outbreak in sales would not traverse to other VLANS, therefore, restricting the spread. A flat network is simple to manage, but a level playing field for an attacker to compromise if all the assets are in the same space. Don’t use an account with admin rights to perform your daily duties. There’s no prizes for guessing the level of potential damage this can cause if your account is compromised, or you land up with malware on your PC Educate and phish your users on a continual basis. They are the gateway directly into your network, and bypassing them is surprisingly easy. You only have to look at the success of phishing campaigns to realise that they are (and always will be) the weakest link in your network. Devise a consistent security and risk review framework. Conducting periodic security reviews is always a good move, and you’d be surprised at just what is lurking around on your network without your knowledge. There needn’t be a huge budget for this. There are a number of open source projects and platforms that make this process simple in terms of identification, but you’ll still need to complete the “grunt” work in terms of remediation. I am currently authoring a framework that will be open source, and will share this with the community once it is completed. Conduct regular governance and due diligence on vendors - particularly those that handle information considered sensitive (think GDPR). If their network is breached, any information they hold around your network and associated users is also at risk. Identify weak or potential risk areas within your network. Engage with business leaders and management to raise awareness around best practice, and information security. Perform breach simulation, and engage senior management in this exercise. As they are the fundamental core of the business function, they also need to understand the risk, and more importantly, the decisions and communication that is inevitable post breach.

    There is no silver bullet when it comes to protecting your network, information, and reputation. However, the list above will form the basis of a solid framework.

    Let’s not be complacent - let’s be compliant.

  • 5 Votes
    6 Posts
    292 Views

    @hari That’s it. Yes. Nothing more to do