Skip to content
Sudonix

Crowdsec: a replacement for Fail2ban

Security
  • DownPWundefined

    What is Crowdsec?

    35751b4b-7306-44e7-bb48-0e51a2e0750a-image.png

    CrowdSec is a free, open-source and collaborative IPS.
    Analyze behaviors, respond to attacks & share signals across the community.

    Imagined by the French Philippe Humeau, ex-founder of a high security host and by Thibault Koechlin, pentester, and DevSecOps, CrowdSec in 2019 is a 100% open source tool that combines 2 ideas well known to server administrators.

    The first idea is to be able to automatically ban an IP connecting to your server, a bit like a Fail2ban. And the second idea is to offer a database of these IPs to the community of CrowdSec users, a bit like AbuseIPDB does, in order to circulate the information quickly and so that everyone can be protected in turn.

    It is therefore a clever trade-off between the behavior of an IP address on your server and its reputation that determines whether it is a legitimate IP or that of an attacker. A bit like arriving in front of a nightclub where you have never set foot and the bouncer already knows your reputation.

    This allows you to detect all types of attacks and respond to them in a proportionate way at any level of your infrastructure (from the CDN to the application level)

    Developed in Go, this open source tool is inspired by the philosophy of Fail2ban, and uses Grok models and YAML syntax to analyze your server logs. Once the malicious IP is intercepted, CrowdSec is able to remediate the threats using different bouncers (block, 403, Captchas, etc.) and the blocked @IPs are then shared among all users to further improve a little more safety for everyone.

    So how does it work?

    Well, your server produces logs and different data sources (cloud trails, data from your applications, etc.) containing the IP addresses of the machines that access this service.

    A CrowdSec service, super simple to set up on your machine, then takes care of observing all of this in real time or retroactively for post-analysis on cold logs.

    Then, using parsers (Parsers) that you can set up quite easily either by writing them yourself or by downloading them from the CrowdSec site, the behavior of each IP address found in the logs is analyzed.

    The CrowdSec agent normalizes (using the GROK scheme) and then enriches this IP address with additional information such as the country of origin of the IP, its AS of origin, the range to which it belongs, then searches in the CrowdSec community database if this IP has already been noticed on another server. Obviously, if the answer is yes, it is blocked.

    If it is a new IP address with malicious behavior, different remediation scenarios can be offered to it: request a captcha, drop a packet, block access to an area of ​​the site, require double authentication, or end up with a pure blockage and simple. On the other hand, despite reading the documentation, there is no possibility of launching a drone attack equipped with missiles on the location of the IP.

    Once the IP address has been identified, it is accompanied by a timestamp and its scenario and then exported to the CrowdSec database.

    ⚠️ By default, CrowdSec does not block detected IPs so if you want to block them, you will have to go to the hub and install the blocker of your choice (Bouncers).

    There are Bouncers for WordPress, Cloudflare, Nginx or for Firewalls like iptables or nftables.

    Once again, it is you who determines the rules to apply according to the situations and the entry points, and CrowdSec is thus able to detect traditional attacks like a bruteforce or a web / ports scan until attacks much more elaborate and targeted, involving for example the use of 0day vulnerabilities, and this on all the services present on the machine (application, DNS, mail, web server, etc.). Too bad for the smart ones! 🙂

    As you will have understood, it is hyper granular in approach and if the IP address in question turns out to belong to bad bad guys, it is then shared with the whole of the benevolent community of CrowdSec via their database. 'IP.

    Regarding the community aspect of IP sharing, it is rather well thought out since IPs are evaluated by CrowdSec on the one hand and trust scores are gradually assigned to machines using CrowdSec. This prevents any poisoning of the database by spammers and limits false positives.

    CrowdSec is developed in Go and can be installed on a cloud or traditional server, deployed in a container or used via its API.

    Calls to the community IP list are indeed made via an API, which allows it to be used also on machines with little CPU/RAM as connected objects. Thus, a connected object, even if not very powerful, could simply take advantage of CrowdSec’s IP reputation system without having to sacrifice performance. Enough to make your home automation installation survive and avoid finding yourself in a house that behaves strangely in Exorcist mode.

    For example, CrowdSec helped one of its users, and therefore its entire community, avoid a botnet attack consisting of more than 7,000 zombie machines, in less than a minute, or a vast scam attempt on an e-commerce site with stolen bank cards carried out from more than 3,000 different IP addresses

    Regarding its license, CrowdSec has been under the MIT open source license since May 2020. It is therefore free to use and can be deployed under Unix / Linux system and soon under Windows and maCOS.

    The community database remains freely accessible for all CrowdSec users who contribute to it and the company finances itself by offering this database to other companies who do not contribute to it and who would like to integrate it directly into their workflows with additional private pool features for IP consensus, self-monitoring or even for broader monitoring (server fleet).

    The arrival of the Local API

    With the arrival of version 1.0 of CrowdSec, the architecture has evolved with the arrival of a Local REST API service.

    Thus all CrowdSec components (daemon, client and bouncers) can communicate with each other via a REST API, rather than reading/writing directly in the database.

    Only the Local API service interacts with the database (SQLite, PostgreSQL or MySQL) and this has several advantages, particularly at the level of the bouncers which can now operate completely independently, using the API and no longer direct access to the database. data. Thus, future changes in the database schema will no longer have any impact on the bouncers.

    245a3a89-b944-4a09-8a2e-587ef97ac5df-image.png

    This change also makes it possible to set up distributed configurations on several machines in a totally transparent way. We can imagine a machine running the local API, different CrowdSec agents analyzing the logs on several machines and bouncers also on other machines.

    This brings good modularity to the CrowdSec system and a certain durability in the continuation of the project and deployments among its users.

    #Link

    Website: https://crowdsec.net

    Documentation: https://docs.crowdsec.net/docs/getting_started/install_crowdsec

    My french tutorial installation here, liked by Phillipe Humeau himself, one of the designers of Crowdsec 🙂

    Very good security software, test it 😉

    PW and WS Spirit alive

    3
  • DownPWundefined DownPW marked this topic as a regular topic on
  • phenomlabundefined

    @DownPW this is a fantastic article - very well received indeed. I’ve actually read a lot about this particular product and always promised myself I’d take a good look at it once I had more time.

    As my professional role is Chief Information Security Officer for a financial institution, I cannot stress enough the importance of taking security seriously in today’s world.

    Author of OGProxy, Swatch, Threaded, Card, and most of Sudonix's bugs…

    0
  • DownPWundefined

    @phenomlab

    No they have a free and pro console instance.
    We can see alert with IP, Source AS, scenario attack etc…

    Installation on the NODEBB server without problems. Very good tools

    cf7e5a89-84f4-435b-82eb-434c0bfc895e-image.png
    cc82a10e-a1f1-4fd8-a433-7c9b2d31f254-image.png

    1b7147b0-37c6-4d87-b4f1-a0fe92e74afd-image.png

    7c21fc10-1825-48e1-a993-92b84455f074-image.png

    –
    We can also do research on IPs via the crowdsec analyzer

    I believe it’s 500 per month in the Free version

    43bc8265-a57c-4439-829c-0bb8602d99b4-image.png

    PW and WS Spirit alive

    1
  • DownPWundefined DownPW referenced this topic on

Related Topics
  • JACundefined

    Most privacy / security focused password manager

    Unsolved Privacy
    12
    7 Votes
    12 Posts
    84 Views
    JACundefined

    @DownPW this looks interesting! 😬😬.

  • phenomlabundefined

    Warning for Android users to urgently delete these apps if installed

    Security
    1
    1 Votes
    1 Posts
    180 Views
    No one has replied
  • phenomlabundefined

    Bad information security advice

    Security
    1
    1 Votes
    1 Posts
    141 Views
    No one has replied
  • DownPWundefined

    Setting for high load and prevent DDoS (sysctl, iptables, crowdsec or other)

    Security
    4
    0 Votes
    4 Posts
    668 Views
    phenomlabundefined

    @DownPW 🙂 most of this really depends on your desired security model. In all cases with firewalls, less is always more, although it’s never as clear cut as that, and there are always bespoke ports you’ll need to open periodically.

    Heztner’s DDoS protection is superior, and I know they have invested a lot of time, effort, and money into making it extremely effective. However, if you consider that the largest ever DDoS attack hit Cloudflare at 71m rps (and they were able to deflect it), and each attack can last anywhere between 8-24 hours which really depends on how determined the attacker(s) is/are, you can never be fully prepared - nor can you trace it’s true origin.

    DDoS attacks by their nature (Distributed Denial of Service) are conducted by large numbers of devices whom have become part of a “bot army” - and in most cases, the owners of these devices are blissfully unaware that they have been attacked and are under command and control from a nefarious resource. Given that the attacks originate from multiple sources, this allows the real attacker to observe from a distance whilst concealing their own identity and origin in the process.

    If you consider the desired effect of DDoS, it is not an attempt to access ports that are typically closed, but to flood (and eventually overwhelm) the target (such as a website) with millions of requests per second in an attempt to force it offline. Victims of DDoS attacks are often financial services for example, with either extortion or financial gain being the primary objective - in other words, pay for the originator to stop the attack.

    It’s even possible to get DDoS as a service these days - with a credit card, a few clicks of a mouse and a target IP, you can have your own proxy campaign running in minutes which typically involves “booters” or “stressers” - see below for more

    https://heimdalsecurity.com/blog/ddos-as-a-service-attacks-what-are-they-and-how-do-they-work

    @DownPW said in Setting for high load and prevent DDoS (sysctl, iptables, crowdsec or other):

    in short if you have any advice to give to secure the best.

    It’s not just about DDos or firewalls. There are a number of vulnerabilities on all systems that if not patched, will expose that same system to exploit. One of my favourite online testers which does a lot more than most basic ones is below

    https://www.immuniweb.com/websec/

    I’d start with the findings reported here and use that to branch outwards.

  • Mike Jonesundefined

    Securing javascript -> PHP mysql calls on Website

    Solved Security
    2
    1 Votes
    2 Posts
    371 Views
    phenomlabundefined

    @mike-jones Hi Mike,

    There are multiple answers to this, so I’m going to provide some of the most important ones here

    JS is a client side library, so you shouldn’t rely on it solely for validation. Any values collected by JS will need to be passed back to the PHP backend for processing, and will need to be fully sanitised first to ensure that your database is not exposed to SQL injection. In order to pass back those values into PHP, you’ll need to use something like

    <script> var myvalue = $('#id').val(); $(document).ready(function() { $.ajax({ type: "POST", url: "https://myserver/myfile.php?id=" + myvalue, success: function() { $("#targetdiv").load('myfile.php?id=myvalue #targetdiv', function() {}); }, //error: ajaxError }); return false; }); </script>

    Then collect that with PHP via a POST / GET request such as

    <?php $myvalue= $_GET['id']; echo "The value is " . $myvalue; ?>

    Of course, the above is a basic example, but is fully functional. Here, the risk level is low in the sense that you are not attempting to manipulate data, but simply request it. However, this in itself would still be vulnerable to SQL injection attack if the request is not sent as OOP (Object Orientated Programming). Here’s an example of how to get the data safely

    <?php function getid($theid) { global $db; $stmt = $db->prepare("SELECT *FROM data where id = ?"); $stmt->execute([$theid]); while ($result= $stmt->fetch(PDO::FETCH_ASSOC)){ $name = $result['name']; $address = $result['address']; $zip = $result['zip']; } return array( 'name' => $name, 'address' => $address, 'zip' => $zip ); } ?>

    Essentially, using the OOP method, we send placeholders rather than actual values. The job of the function is to check the request and automatically sanitise it to ensure we only return what is being asked for, and nothing else. This prevents typical injections such as “AND 1=1” which of course would land up returning everything which isn’t what you want at all for security reasons.

    When calling the function, you’d simply use

    <?php echo getid($myvalue); ?>

    @mike-jones said in Securing javascript -> PHP mysql calls on Website:

    i am pretty sure the user could just use the path to the php file and just type a web address into the search bar

    This is correct, although with no parameters, no data would be returned. You can actually prevent the PHP script from being called directly using something like

    <?php if(!defined('MyConst')) { die('Direct access not permitted'); } ?>

    then on the pages that you need to include it

    <?php define('MyConst', TRUE); ?>

    Obviously, access requests coming directly are not going via your chosen route, therefore, the connection will die because MyConst does not equal TRUE

    @mike-jones said in Securing javascript -> PHP mysql calls on Website:

    Would it be enough to just check if the number are a number 1-100 and if the drop down is one of the 5 specific words and then just not run the rest of the code if it doesn’t fit one of those perameters?

    In my view, no, as this will expose the PHP file to SQL injection attack without any server side checking.

    Hope this is of some use to start with. Happy to elaborate if you’d like.

  • phenomlabundefined

    Why you should never neglect physical security

    Blog
    1
    0 Votes
    1 Posts
    322 Views
    No one has replied
  • Hariundefined

    is my DMARC configured correctly?

    Solved Configure
    3
    3 Votes
    3 Posts
    466 Views
    Hariundefined

    @phenomlab said in is my DMARC configured correctly?:

    you’ll get one from every domain that receives email from yours.

    Today I have received another mail from outlook DMARC, i was referring to your reply again and found it very helpful/informative. thanks again.

    I wish sudonix 100 more great years ahead!

  • phenomlabundefined

    Browsing without a VPN? Think Twice...

    Moved Security
    12
    2 Votes
    12 Posts
    1k Views
    phenomlabundefined

    And if you ever needed another reason to use a VPN, here it is.

    https://news.sky.com/story/google-blinks-first-in-11-month-privacy-showdown-with-uk-regulator-12479198