Skip to content

Crowdsec: a replacement for Fail2ban

Security
  • What is Crowdsec?


    35751b4b-7306-44e7-bb48-0e51a2e0750a-image.png

    CrowdSec is a free, open-source and collaborative IPS.
    Analyze behaviors, respond to attacks & share signals across the community.

    Imagined by the French Philippe Humeau, ex-founder of a high security host and by Thibault Koechlin, pentester, and DevSecOps, CrowdSec in 2019 is a 100% open source tool that combines 2 ideas well known to server administrators.

    The first idea is to be able to automatically ban an IP connecting to your server, a bit like a Fail2ban. And the second idea is to offer a database of these IPs to the community of CrowdSec users, a bit like AbuseIPDB does, in order to circulate the information quickly and so that everyone can be protected in turn.

    It is therefore a clever trade-off between the behavior of an IP address on your server and its reputation that determines whether it is a legitimate IP or that of an attacker. A bit like arriving in front of a nightclub where you have never set foot and the bouncer already knows your reputation.

    This allows you to detect all types of attacks and respond to them in a proportionate way at any level of your infrastructure (from the CDN to the application level)

    Developed in Go, this open source tool is inspired by the philosophy of Fail2ban, and uses Grok models and YAML syntax to analyze your server logs. Once the malicious IP is intercepted, CrowdSec is able to remediate the threats using different bouncers (block, 403, Captchas, etc.) and the blocked @IPs are then shared among all users to further improve a little more safety for everyone.

    So how does it work?


    Well, your server produces logs and different data sources (cloud trails, data from your applications, etc.) containing the IP addresses of the machines that access this service.

    A CrowdSec service, super simple to set up on your machine, then takes care of observing all of this in real time or retroactively for post-analysis on cold logs.

    Then, using parsers (Parsers) that you can set up quite easily either by writing them yourself or by downloading them from the CrowdSec site, the behavior of each IP address found in the logs is analyzed.

    The CrowdSec agent normalizes (using the GROK scheme) and then enriches this IP address with additional information such as the country of origin of the IP, its AS of origin, the range to which it belongs, then searches in the CrowdSec community database if this IP has already been noticed on another server. Obviously, if the answer is yes, it is blocked.

    If it is a new IP address with malicious behavior, different remediation scenarios can be offered to it: request a captcha, drop a packet, block access to an area of ​​the site, require double authentication, or end up with a pure blockage and simple. On the other hand, despite reading the documentation, there is no possibility of launching a drone attack equipped with missiles on the location of the IP.

    Once the IP address has been identified, it is accompanied by a timestamp and its scenario and then exported to the CrowdSec database.

    ⚠️ By default, CrowdSec does not block detected IPs so if you want to block them, you will have to go to the hub and install the blocker of your choice (Bouncers).

    There are Bouncers for WordPress, Cloudflare, Nginx or for Firewalls like iptables or nftables.

    Once again, it is you who determines the rules to apply according to the situations and the entry points, and CrowdSec is thus able to detect traditional attacks like a bruteforce or a web / ports scan until attacks much more elaborate and targeted, involving for example the use of 0day vulnerabilities, and this on all the services present on the machine (application, DNS, mail, web server, etc.). Too bad for the smart ones! 🙂

    As you will have understood, it is hyper granular in approach and if the IP address in question turns out to belong to bad bad guys, it is then shared with the whole of the benevolent community of CrowdSec via their database. 'IP.

    Regarding the community aspect of IP sharing, it is rather well thought out since IPs are evaluated by CrowdSec on the one hand and trust scores are gradually assigned to machines using CrowdSec. This prevents any poisoning of the database by spammers and limits false positives.

    CrowdSec is developed in Go and can be installed on a cloud or traditional server, deployed in a container or used via its API.

    Calls to the community IP list are indeed made via an API, which allows it to be used also on machines with little CPU/RAM as connected objects. Thus, a connected object, even if not very powerful, could simply take advantage of CrowdSec’s IP reputation system without having to sacrifice performance. Enough to make your home automation installation survive and avoid finding yourself in a house that behaves strangely in Exorcist mode.

    For example, CrowdSec helped one of its users, and therefore its entire community, avoid a botnet attack consisting of more than 7,000 zombie machines, in less than a minute, or a vast scam attempt on an e-commerce site with stolen bank cards carried out from more than 3,000 different IP addresses

    Regarding its license, CrowdSec has been under the MIT open source license since May 2020. It is therefore free to use and can be deployed under Unix / Linux system and soon under Windows and maCOS.

    The community database remains freely accessible for all CrowdSec users who contribute to it and the company finances itself by offering this database to other companies who do not contribute to it and who would like to integrate it directly into their workflows with additional private pool features for IP consensus, self-monitoring or even for broader monitoring (server fleet).

    The arrival of the Local API


    With the arrival of version 1.0 of CrowdSec, the architecture has evolved with the arrival of a Local REST API service.

    Thus all CrowdSec components (daemon, client and bouncers) can communicate with each other via a REST API, rather than reading/writing directly in the database.

    Only the Local API service interacts with the database (SQLite, PostgreSQL or MySQL) and this has several advantages, particularly at the level of the bouncers which can now operate completely independently, using the API and no longer direct access to the database. data. Thus, future changes in the database schema will no longer have any impact on the bouncers.

    245a3a89-b944-4a09-8a2e-587ef97ac5df-image.png

    This change also makes it possible to set up distributed configurations on several machines in a totally transparent way. We can imagine a machine running the local API, different CrowdSec agents analyzing the logs on several machines and bouncers also on other machines.

    This brings good modularity to the CrowdSec system and a certain durability in the continuation of the project and deployments among its users.

    #Link


    Website: https://crowdsec.net

    Documentation: https://docs.crowdsec.net/docs/getting_started/install_crowdsec

    My french tutorial installation here, liked by Phillipe Humeau himself, one of the designers of Crowdsec 🙂

    Very good security software, test it 😉

  • DownPWundefined DownPW marked this topic as a regular topic on
  • @DownPW this is a fantastic article - very well received indeed. I’ve actually read a lot about this particular product and always promised myself I’d take a good look at it once I had more time.

    As my professional role is Chief Information Security Officer for a financial institution, I cannot stress enough the importance of taking security seriously in today’s world.

  • @phenomlab

    No they have a free and pro console instance.
    We can see alert with IP, Source AS, scenario attack etc…

    Installation on the NODEBB server without problems. Very good tools

    cf7e5a89-84f4-435b-82eb-434c0bfc895e-image.png
    cc82a10e-a1f1-4fd8-a433-7c9b2d31f254-image.png

    1b7147b0-37c6-4d87-b4f1-a0fe92e74afd-image.png

    7c21fc10-1825-48e1-a993-92b84455f074-image.png


    We can also do research on IPs via the crowdsec analyzer

    I believe it’s 500 per month in the Free version

    43bc8265-a57c-4439-829c-0bb8602d99b4-image.png

  • DownPWundefined DownPW referenced this topic on

  • 12 Votes
    8 Posts
    274 Views

    @crazycells good question. Gmail being provided by Google is going to be one of the more secure by default out of the box, although you have to bear in mind that you can have the best security in the world, but that is easily diluted by user decision.

    Obviously, it makes sense to secure all cloud based services with at least 2fa protection, or better still, biometric if available, but email still remains vastly unprotected (unless enforced in the sense of 2fa, which I know Sendgrid do) because of user choice (in the sense that users will always go for the path of least resistance when it comes to security to make their lives easier). The ultimate side effect of taking this route is being vulnerable to credentials theft via phishing attacks and social engineering.

    The same principle would easily apply to Proton Mail, who also (from memory) do not enforce 2fa. Based on this fact, neither product is more secure than the other without one form of additional authentication at least being imposed.

    In terms of direct attack on the servers holding mail accounts themselves, this is a far less common type of attack these days as tricking the user is so much simpler than brute forcing a server where you are very likely to be detected by perimeter security (IDS / IPS etc).

  • 0 Votes
    4 Posts
    400 Views

    @DownPW 🙂 most of this really depends on your desired security model. In all cases with firewalls, less is always more, although it’s never as clear cut as that, and there are always bespoke ports you’ll need to open periodically.

    Heztner’s DDoS protection is superior, and I know they have invested a lot of time, effort, and money into making it extremely effective. However, if you consider that the largest ever DDoS attack hit Cloudflare at 71m rps (and they were able to deflect it), and each attack can last anywhere between 8-24 hours which really depends on how determined the attacker(s) is/are, you can never be fully prepared - nor can you trace it’s true origin.

    DDoS attacks by their nature (Distributed Denial of Service) are conducted by large numbers of devices whom have become part of a “bot army” - and in most cases, the owners of these devices are blissfully unaware that they have been attacked and are under command and control from a nefarious resource. Given that the attacks originate from multiple sources, this allows the real attacker to observe from a distance whilst concealing their own identity and origin in the process.

    If you consider the desired effect of DDoS, it is not an attempt to access ports that are typically closed, but to flood (and eventually overwhelm) the target (such as a website) with millions of requests per second in an attempt to force it offline. Victims of DDoS attacks are often financial services for example, with either extortion or financial gain being the primary objective - in other words, pay for the originator to stop the attack.

    It’s even possible to get DDoS as a service these days - with a credit card, a few clicks of a mouse and a target IP, you can have your own proxy campaign running in minutes which typically involves “booters” or “stressers” - see below for more

    https://heimdalsecurity.com/blog/ddos-as-a-service-attacks-what-are-they-and-how-do-they-work

    @DownPW said in Setting for high load and prevent DDoS (sysctl, iptables, crowdsec or other):

    in short if you have any advice to give to secure the best.

    It’s not just about DDos or firewalls. There are a number of vulnerabilities on all systems that if not patched, will expose that same system to exploit. One of my favourite online testers which does a lot more than most basic ones is below

    https://www.immuniweb.com/websec/

    I’d start with the findings reported here and use that to branch outwards.

  • 4 Votes
    8 Posts
    624 Views

    @phenomlab
    Sorry to delay in responding, yes as i mentioned above, i had to remove my redis from docker and reinstall a new image with this command

    docker run --name=redis -p 127.0.0.1:6379:6379 -d -t redis:alpine

    and now when i test my ip and port on
    https://www.yougetsignal.com/tools/open-ports/

    the status of my redis port is closed. I think which to configure firewall in droplet digital ocean is a good idea too, and i will configure soon.
    Thanks for the help!

  • 0 Votes
    1 Posts
    216 Views
    No one has replied
  • 0 Votes
    1 Posts
    189 Views
    No one has replied
  • 0 Votes
    1 Posts
    212 Views
    No one has replied
  • 0 Votes
    1 Posts
    207 Views
    No one has replied
  • 0 Votes
    3 Posts
    315 Views

    @justoverclock yes, completely understand that. It’s a haven for criminal gangs and literally everything is on the table. Drugs, weapons, money laundering, cyber attacks for rent, and even murder for hire.

    Nothing it seems is off limits. The dark web is truly a place where the only limitation is the amount you are prepared to spend.