Crowdsec: a replacement for Fail2ban
-
What is Crowdsec?
CrowdSec is a free, open-source and collaborative IPS.
Analyze behaviors, respond to attacks & share signals across the community.Imagined by the French Philippe Humeau, ex-founder of a high security host and by Thibault Koechlin, pentester, and DevSecOps, CrowdSec in 2019 is a 100% open source tool that combines 2 ideas well known to server administrators.
The first idea is to be able to automatically ban an IP connecting to your server, a bit like a Fail2ban. And the second idea is to offer a database of these IPs to the community of CrowdSec users, a bit like AbuseIPDB does, in order to circulate the information quickly and so that everyone can be protected in turn.
It is therefore a clever trade-off between the behavior of an IP address on your server and its reputation that determines whether it is a legitimate IP or that of an attacker. A bit like arriving in front of a nightclub where you have never set foot and the bouncer already knows your reputation.
This allows you to detect all types of attacks and respond to them in a proportionate way at any level of your infrastructure (from the CDN to the application level)
Developed in Go, this open source tool is inspired by the philosophy of Fail2ban, and uses Grok models and YAML syntax to analyze your server logs. Once the malicious IP is intercepted, CrowdSec is able to remediate the threats using different bouncers (block, 403, Captchas, etc.) and the blocked @IPs are then shared among all users to further improve a little more safety for everyone.
So how does it work?
Well, your server produces logs and different data sources (cloud trails, data from your applications, etc.) containing the IP addresses of the machines that access this service.
A CrowdSec service, super simple to set up on your machine, then takes care of observing all of this in real time or retroactively for post-analysis on cold logs.
Then, using parsers (Parsers) that you can set up quite easily either by writing them yourself or by downloading them from the CrowdSec site, the behavior of each IP address found in the logs is analyzed.
The CrowdSec agent normalizes (using the GROK scheme) and then enriches this IP address with additional information such as the country of origin of the IP, its AS of origin, the range to which it belongs, then searches in the CrowdSec community database if this IP has already been noticed on another server. Obviously, if the answer is yes, it is blocked.
If it is a new IP address with malicious behavior, different remediation scenarios can be offered to it: request a captcha, drop a packet, block access to an area of the site, require double authentication, or end up with a pure blockage and simple. On the other hand, despite reading the documentation, there is no possibility of launching a drone attack equipped with missiles on the location of the IP.
Once the IP address has been identified, it is accompanied by a timestamp and its scenario and then exported to the CrowdSec database.
️ By default, CrowdSec does not block detected IPs so if you want to block them, you will have to go to the hub and install the blocker of your choice (Bouncers).There are Bouncers for WordPress, Cloudflare, Nginx or for Firewalls like iptables or nftables.
Once again, it is you who determines the rules to apply according to the situations and the entry points, and CrowdSec is thus able to detect traditional attacks like a bruteforce or a web / ports scan until attacks much more elaborate and targeted, involving for example the use of 0day vulnerabilities, and this on all the services present on the machine (application, DNS, mail, web server, etc.). Too bad for the smart ones!
As you will have understood, it is hyper granular in approach and if the IP address in question turns out to belong to bad bad guys, it is then shared with the whole of the benevolent community of CrowdSec via their database. 'IP.
Regarding the community aspect of IP sharing, it is rather well thought out since IPs are evaluated by CrowdSec on the one hand and trust scores are gradually assigned to machines using CrowdSec. This prevents any poisoning of the database by spammers and limits false positives.
CrowdSec is developed in Go and can be installed on a cloud or traditional server, deployed in a container or used via its API.
Calls to the community IP list are indeed made via an API, which allows it to be used also on machines with little CPU/RAM as connected objects. Thus, a connected object, even if not very powerful, could simply take advantage of CrowdSec’s IP reputation system without having to sacrifice performance. Enough to make your home automation installation survive and avoid finding yourself in a house that behaves strangely in Exorcist mode.
For example, CrowdSec helped one of its users, and therefore its entire community, avoid a botnet attack consisting of more than 7,000 zombie machines, in less than a minute, or a vast scam attempt on an e-commerce site with stolen bank cards carried out from more than 3,000 different IP addresses
Regarding its license, CrowdSec has been under the MIT open source license since May 2020. It is therefore free to use and can be deployed under Unix / Linux system and soon under Windows and maCOS.
The community database remains freely accessible for all CrowdSec users who contribute to it and the company finances itself by offering this database to other companies who do not contribute to it and who would like to integrate it directly into their workflows with additional private pool features for IP consensus, self-monitoring or even for broader monitoring (server fleet).
The arrival of the Local API
With the arrival of version 1.0 of CrowdSec, the architecture has evolved with the arrival of a Local REST API service.
Thus all CrowdSec components (daemon, client and bouncers) can communicate with each other via a REST API, rather than reading/writing directly in the database.
Only the Local API service interacts with the database (SQLite, PostgreSQL or MySQL) and this has several advantages, particularly at the level of the bouncers which can now operate completely independently, using the API and no longer direct access to the database. data. Thus, future changes in the database schema will no longer have any impact on the bouncers.
This change also makes it possible to set up distributed configurations on several machines in a totally transparent way. We can imagine a machine running the local API, different CrowdSec agents analyzing the logs on several machines and bouncers also on other machines.
This brings good modularity to the CrowdSec system and a certain durability in the continuation of the project and deployments among its users.
#Link
Website: https://crowdsec.net
Documentation: https://docs.crowdsec.net/docs/getting_started/install_crowdsec
My french tutorial installation here, liked by Phillipe Humeau himself, one of the designers of Crowdsec
Very good security software, test it
PW and WS Spirit alive
-
undefined DownPW marked this topic as a regular topic on
-
What is Crowdsec?
CrowdSec is a free, open-source and collaborative IPS.
Analyze behaviors, respond to attacks & share signals across the community.Imagined by the French Philippe Humeau, ex-founder of a high security host and by Thibault Koechlin, pentester, and DevSecOps, CrowdSec in 2019 is a 100% open source tool that combines 2 ideas well known to server administrators.
The first idea is to be able to automatically ban an IP connecting to your server, a bit like a Fail2ban. And the second idea is to offer a database of these IPs to the community of CrowdSec users, a bit like AbuseIPDB does, in order to circulate the information quickly and so that everyone can be protected in turn.
It is therefore a clever trade-off between the behavior of an IP address on your server and its reputation that determines whether it is a legitimate IP or that of an attacker. A bit like arriving in front of a nightclub where you have never set foot and the bouncer already knows your reputation.
This allows you to detect all types of attacks and respond to them in a proportionate way at any level of your infrastructure (from the CDN to the application level)
Developed in Go, this open source tool is inspired by the philosophy of Fail2ban, and uses Grok models and YAML syntax to analyze your server logs. Once the malicious IP is intercepted, CrowdSec is able to remediate the threats using different bouncers (block, 403, Captchas, etc.) and the blocked @IPs are then shared among all users to further improve a little more safety for everyone.
So how does it work?
Well, your server produces logs and different data sources (cloud trails, data from your applications, etc.) containing the IP addresses of the machines that access this service.
A CrowdSec service, super simple to set up on your machine, then takes care of observing all of this in real time or retroactively for post-analysis on cold logs.
Then, using parsers (Parsers) that you can set up quite easily either by writing them yourself or by downloading them from the CrowdSec site, the behavior of each IP address found in the logs is analyzed.
The CrowdSec agent normalizes (using the GROK scheme) and then enriches this IP address with additional information such as the country of origin of the IP, its AS of origin, the range to which it belongs, then searches in the CrowdSec community database if this IP has already been noticed on another server. Obviously, if the answer is yes, it is blocked.
If it is a new IP address with malicious behavior, different remediation scenarios can be offered to it: request a captcha, drop a packet, block access to an area of the site, require double authentication, or end up with a pure blockage and simple. On the other hand, despite reading the documentation, there is no possibility of launching a drone attack equipped with missiles on the location of the IP.
Once the IP address has been identified, it is accompanied by a timestamp and its scenario and then exported to the CrowdSec database.
️ By default, CrowdSec does not block detected IPs so if you want to block them, you will have to go to the hub and install the blocker of your choice (Bouncers).There are Bouncers for WordPress, Cloudflare, Nginx or for Firewalls like iptables or nftables.
Once again, it is you who determines the rules to apply according to the situations and the entry points, and CrowdSec is thus able to detect traditional attacks like a bruteforce or a web / ports scan until attacks much more elaborate and targeted, involving for example the use of 0day vulnerabilities, and this on all the services present on the machine (application, DNS, mail, web server, etc.). Too bad for the smart ones!
As you will have understood, it is hyper granular in approach and if the IP address in question turns out to belong to bad bad guys, it is then shared with the whole of the benevolent community of CrowdSec via their database. 'IP.
Regarding the community aspect of IP sharing, it is rather well thought out since IPs are evaluated by CrowdSec on the one hand and trust scores are gradually assigned to machines using CrowdSec. This prevents any poisoning of the database by spammers and limits false positives.
CrowdSec is developed in Go and can be installed on a cloud or traditional server, deployed in a container or used via its API.
Calls to the community IP list are indeed made via an API, which allows it to be used also on machines with little CPU/RAM as connected objects. Thus, a connected object, even if not very powerful, could simply take advantage of CrowdSec’s IP reputation system without having to sacrifice performance. Enough to make your home automation installation survive and avoid finding yourself in a house that behaves strangely in Exorcist mode.
For example, CrowdSec helped one of its users, and therefore its entire community, avoid a botnet attack consisting of more than 7,000 zombie machines, in less than a minute, or a vast scam attempt on an e-commerce site with stolen bank cards carried out from more than 3,000 different IP addresses
Regarding its license, CrowdSec has been under the MIT open source license since May 2020. It is therefore free to use and can be deployed under Unix / Linux system and soon under Windows and maCOS.
The community database remains freely accessible for all CrowdSec users who contribute to it and the company finances itself by offering this database to other companies who do not contribute to it and who would like to integrate it directly into their workflows with additional private pool features for IP consensus, self-monitoring or even for broader monitoring (server fleet).
The arrival of the Local API
With the arrival of version 1.0 of CrowdSec, the architecture has evolved with the arrival of a Local REST API service.
Thus all CrowdSec components (daemon, client and bouncers) can communicate with each other via a REST API, rather than reading/writing directly in the database.
Only the Local API service interacts with the database (SQLite, PostgreSQL or MySQL) and this has several advantages, particularly at the level of the bouncers which can now operate completely independently, using the API and no longer direct access to the database. data. Thus, future changes in the database schema will no longer have any impact on the bouncers.
This change also makes it possible to set up distributed configurations on several machines in a totally transparent way. We can imagine a machine running the local API, different CrowdSec agents analyzing the logs on several machines and bouncers also on other machines.
This brings good modularity to the CrowdSec system and a certain durability in the continuation of the project and deployments among its users.
#Link
Website: https://crowdsec.net
Documentation: https://docs.crowdsec.net/docs/getting_started/install_crowdsec
My french tutorial installation here, liked by Phillipe Humeau himself, one of the designers of Crowdsec
Very good security software, test it
@DownPW this is a fantastic article - very well received indeed. I’ve actually read a lot about this particular product and always promised myself I’d take a good look at it once I had more time.
As my professional role is Chief Information Security Officer for a financial institution, I cannot stress enough the importance of taking security seriously in today’s world.
Author of OGProxy, Swatch, Threaded, Card, and most of Sudonix's bugs…
-
@DownPW this is a fantastic article - very well received indeed. I’ve actually read a lot about this particular product and always promised myself I’d take a good look at it once I had more time.
As my professional role is Chief Information Security Officer for a financial institution, I cannot stress enough the importance of taking security seriously in today’s world.
No they have a free and pro console instance.
We can see alert with IP, Source AS, scenario attack etc…Installation on the NODEBB server without problems. Very good tools
–
We can also do research on IPs via the crowdsec analyzerI believe it’s 500 per month in the Free version
-
undefined DownPW referenced this topic on
