Performance

Lower grade VPS instances, whilst cheap, do have the inherent issue in the fact that they only have 1Gb of RAM. In most cases, this is enough for relatively small or minor projects, but when you need more RAM that you actually have, you’ll quickly find that instance exhausted, and your applications crashing as a result.

This is where the swap file comes into play. Adding a swap can significantly improve performance on low budget hosts, but without direct root access, this is not going to be possible. If you own a VPS that has root level access and need to add a swap, follow the below guide.

First, what exactly is a Swap?

swap is a section of hard disk space that has been set reserved for the operating system to temporarily store data that it is unable to hold in RAM. This step allows you increase the amount of information that your server can keep in its working memory (but not without with some caveats, which I’ll explain below). The swap space on the hard disk will be used mostly when there is no more sufficient space in RAM to host any in-use application data.

The information written to disk will be far slower than information kept in RAM (RAM is superior in terms of speed owing to its architecture), but the operating system will prefer to keep running application data in memory and only use the swap for the older data. Essentially, having swap space as a failsafe for when your system’s physical memory is depleted can be a good safety net against crashes on systems with non-SSD storage available.

Determine the size of the Swap we actually need.

This process is made so much easier by using the below calculator

https://pickwicksoft.github.io/swapcalc/

Admittedly, if you only had 1Gb RAM, the SWAP would be default at 1Gb. You can play with the various configurations here to get the results you need, but be honest - don’t make your system out to be something it isn’t, because otherwise, you’ll create more problems than you set out to resolve.

Swap space refers to a designated portion of hard drive storage that’s reserved for temporary data storage by the operating system when the RAM can’t accommodate it any longer. This allows for an expansion of the data that your server can hold in its active memory, though with certain conditions. The swap area on the hard drive comes into play primarily when there isn’t enough room left in the RAM to hold active application data.

The data that gets written to the disk is notably slower than the data stored in RAM. Nevertheless, the operating system prioritizes keeping currently used application data in memory and employs swap for older data. Having swap space as a fallback when your system’s RAM is exhausted can serve as a valuable safeguard against out-of-memory errors, especially on systems with traditional non-SSD storage.

Verifying the System for Swap Information

Before proceeding, it’s advisable to confirm whether your system already has existing swap space. While it’s possible to have multiple swap files or swap partitions, typically one should suffice.

You can check if your system has any configured swap by executing:

sudo swapon --show

If you receive no output, it means your system presently lacks swap space.

You can also confirm the absence of active swap using the free utility:

free -h

As evident in the output, there is no active swap on the system, as shown in the Swap row.

total used free shared buff/cache available Mem: 981Mi 122Mi 647Mi 0.0Ki 211Mi 714Mi SWAP: 0B 0B 0B Assessing Available Space on the Hard Drive Partition

Before creating a swap file, it’s essential to check the current disk usage to ensure you have enough available space. This can be done by entering

df -h Filesystem Size Used Avail Use% Mounted on tmpfs 1.6G 876K 1.6G 1% /run /dev/sda1 150G 65G 80G 45% / tmpfs 7.7G 0 7.7G 0% /dev/shm tmpfs 5.0M 0 5.0M 0% /run/lock /dev/sda15 253M 6.1M 246M 3% /boot/efi tmpfs 1.6G 0 1.6G 0% /run/user/1009

The device with / in the Mounted on column is our disk in this case. We have sufficient remaining space available - 65G used. Your availability will obviously be different.

The appropriate size of a swap space can vary according to personal preferences and application requirements. Typically, an amount equivalent to or double the system’s RAM is a good starting point. For a simple RAM fallback, anything over 4G of swap is usually deemed unnecessary.

Creating a Swap File

Now that you’ve determined the available hard drive space, you can generate a swap file on your file system. A file of your desired size, named ‘swapfile,’ will be allocated in your root directory (/).

The recommended method for creating a swap file is by using the fallocate program, which instantly generates a file of the specified size. For instance, if your server has 1G of RAM, you can create a 1G file as follows:

sudo fallocate -l 1G /swapfile

You can confirm the correct space allocation by running:

ls -lh /swapfile

The file will be created with the appropriate space allocation.

Activating the Swap File

Now that you have a correctly sized file, it’s time to turn it into swap space. Initially, you must restrict file access to only root users, enhancing security. To achieve this, execute:

sudo chmod 600 /swapfile

You can verify the permission change with:

ls -lh /swapfile

As seen in the output, only the root user has read and write permissions.

Next, mark the file as swap space with:

sudo mkswap /swapfile

Afterward, enable the swap file to allow your system to utilize it:

sudo swapon /swapfile

You can verify the availability of swap by executing:

sudo swapon --show

Finally, recheck the output of the free utility to confirm the setup:

free -h Making the Swap File Permanent

The changes made enable the swap file for the current session, but they won’t persist through a system reboot. To ensure your swap settings remain, you can add the swap file information to your /etc/fstab file. Here’s how you can do it:

Back up the /etc/fstab file as a precaution:

sudo cp /etc/fstab /etc/fstab.bak

Add the swap file information to the end of your /etc/fstab file with:

echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab Adjusting Swap Settings

There are several settings you can configure to influence your system’s performance with swap. Two key settings are the swappiness property and the cache pressure setting:

Swappiness Property: This parameter determines how often data is swapped from RAM to the swap space. A value between 0 and 100 represents a percentage. Lower values (close to 0) mean less frequent swapping, while higher values (closer to 100) encourage more swapping. You can check the current swappiness value with:

cat /proc/sys/vm/swappiness

You can set a different value using the sysctl command. For example, to set the swappiness to 10:

sudo sysctl vm.swappiness=10

This setting persists until the next reboot, but you can make it permanent by adding it to your /etc/sysctl.conf file.

Cache Pressure Setting: This setting affects how the system caches inode and dentry information over other data. Lower values, like 50, make the system cache this information more conservatively. You can check the current cache pressure value with:

cat /proc/sys/vm/vfs_cache_pressure

To set a different value, use the sysctl command and update your /etc/sysctl.conf file as you did with the swappiness setting.

Hello @phenomlab

Following this topic, I understood that CloudFlare in its Free Plan is not my friend but I wonder.

one of CloudFlare’s features of hiding the origin server’s IP is THE feature that makes me not really want to leave CloudFlare unless there’s another way to do it ?

That said, the problem being mainly the socket connections, wouldn’t it be possible to pass ONLY its connections without going through CloudFlare?

To do this, you will have to create a subdomain of the root domain and not proxy it via Cloudflare but just in DNS Only, then indicate this in nodebb’s config.json

If I’m not mistaken, this might fix the wss and error 400 issues.

I saw this topic on which you intervened with your position, but is this method still viable, is it functional?

At the end of this thread it reads:

@phenomlab if there are those who are using cloudflare, we do have it in our roadmap to eventually deprecate socket.io and use socket sent events instead.
This would solve a lot of problem with CloudFlare’s free plan no?

This would solve a lot of problem with CloudFlare’s free plan no?

Afterwards I am ready to leave CloudFlare if another possibility of hiding your IP is possible but in the meantime, could this be a solution ?

Thanks

Hello @mark

I have several disconeexion on v3.2.3 with socket error, csrf; error 400

with this on console :

4ff10158-ade2-40f0-8b5b-ca7f07a96831-image.png

an idea ?

I prefer to use vanilla js where possible, rather than library functions, (such a JQuery.)

I was looking at some Shopify code, and $.ajax was used for API call rather than the newer built in fetch() command.

I was curious why.
Also wonder why some people still use $.ajax in some nodebb plugin code?

Is there is any reason, rather than habit, to stay using the JQuery ajax method? (Apart from that working on old Internet Explorer browser version)

I was also curious if fetch() works from sites like Shopify - in theory theres no reason why not…
But I saw someone else ask this question on Shopify forum, where they could not get fetch working-

https://community.shopify.com/c/shopify-scripts/fetch-json/m-p/1606498

So my question is there any technical reason why people haven’t all moved to using fetch now, which is more modern, has advantages and doesn’t require JQuery?

Hi Mark
Chat area is very slow after uprading. Keyboard is very slow and it is locked some times.

I see more comments about Nodebb forums and the difficulty getting indexed.
Search Engine Optimisation is a whole field in itself, and I do hear empirically that NodeBB forums dont seem to do so well. Thats not a criticism of NodeBB, just an indicator of the complexity of SEO.

@phenomlab is your site indexing better now?

I saw in the output from ./nodebb build , 2WARNING s:
Firstly)

**Entrypoint ... combined size exceeds recommended limit (244 KiB) Entrypoints: -nodebb (355 KiB) -admin (294 KiB)

what does this mean, and is it a problem?

Secondly)

WARNING in asset size limit (and listed 4 min.js files)

Released, who is using and pls share with us your experincess 🙂

I noticed that my v3 instance of NodeBB in test was so much slower than live, but was using the same database etc. On closer inspection, the nginx configuration needed a tweak, so I’m posting my settings here so others can benefit from it. Note, that various aspects have been redacted for obvious privacy and security reasons, and to this end, you will need to substitute these values for those that exist in your own environment.

server { # Ensure you put your server name here, such as example.com www.example.com etc. server_name sservername; listen x.x.x.x:443 ssl http2; access_log /path/to/access.log; error_log /path/to/error.log; ssl_certificate /path/to/ssl.combined; ssl_certificate_key /path/to/ssl.key; # You may not need the below values, so feel free to remove these if not required rewrite ^\Q/mail/config-v1.1.xml\E(.*) $scheme://$host/cgi-bin/autoconfig.cgi$1 break; rewrite ^\Q/.well-known/autoconfig/mail/config-v1.1.xml\E(.*) $scheme://$host/cgi-bin/autoconfig.cgi$1 break; rewrite ^\Q/AutoDiscover/AutoDiscover.xml\E(.*) $scheme://$host/cgi-bin/autoconfig.cgi$1 break; rewrite ^\Q/Autodiscover/Autodiscover.xml\E(.*) $scheme://$host/cgi-bin/autoconfig.cgi$1 break; rewrite ^\Q/autodiscover/autodiscover.xml\E(.*) $scheme://$host/cgi-bin/autoconfig.cgi$1 break; # You may not need the above values, so feel free to remove these if not required client_body_buffer_size 10K; client_header_buffer_size 1k; client_max_body_size 8m; large_client_header_buffers 4 4k; client_body_timeout 12; client_header_timeout 12; keepalive_timeout 15; send_timeout 10; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header Host $http_host; proxy_set_header X-NginX-Proxy true; proxy_redirect off; # Socket.io Support proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; gzip on; gzip_disable "msie6"; gzip_vary on; gzip_proxied any; gzip_min_length 1024; gzip_comp_level 6; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript; add_header X-XSS-Protection "1; mode=block"; add_header X-Download-Options "noopen" always; add_header Content-Security-Policy "upgrade-insecure-requests" always; add_header Referrer-Policy 'no-referrer' always; add_header Permissions-Policy "accelerometer=(), camera=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), payment=(), usb=()" always; # This is the string that will show in the headers if requested, so you can put what you want in here. Keep it clean :) add_header X-Powered-By "<whatever you want here>" always; add_header X-Permitted-Cross-Domain-Policies "none" always; location / { # Don't forget to change the port to the one you use. I have a non-standard one :) proxy_pass http://127.0.0.1:5000; } location @nodebb { # Don't forget to change the port to the one you use. I have a non-standard one :) proxy_pass http://127.0.0.1:5000; } location ~ ^/assets/(.*) { root /path/to/nodebb/; try_files /build/public/$1 /public/$1 @nodebb; add_header Cache-Control "max-age=31536000"; } location /plugins/ { root /path/to/nodebb/build/public/; try_files $uri @nodebb; add_header Cache-Control "max-age=31536000"; } if ($scheme = http) { # Ensure you set your actual domain here rewrite ^/(?!.well-known)(.*) https://yourdomain/$1 break; } }

I’ve added comments at the obvious places where you need to make changes. Depending on how your server is configured, and it’s capabilities, this should improve performance no end.

There is a caveat though, and it’s an important one

Don’t use insane levels in the below section

client_body_buffer_size 10K; client_header_buffer_size 1k; client_max_body_size 8m; large_client_header_buffers 4 4k; client_body_timeout 12; client_header_timeout 12; keepalive_timeout 15; send_timeout 10;

Keep to these values, and if anything, adjust them DOWN to suit your server.

Hello

Since yesterday, we have been experiencing a massive influx of new users.

Today we have more than 1000 simultaneous connections (account creation, participation in topics, etc.)

I had a lot of issues with Nginx that I think I got fixed.

I have several “worker_connections is too long” & “too many openfiles” errors on nginx log :

48103f1b-f4f0-496b-bb5d-bd870b1807fd-image.png
f551f1f9-b9e5-4b57-a89f-af14e2675a3f-image.png

I think I solved his 2 problems in the following way :

systemctl edit --full nginx.service [Service] LimitNOFILE=70000 nano /etc/nginx/nginx.conf worker_rlimit_nofile 70000; events { # worker_connections 768; worker_connections 65535; # multi_accept on; multi_accept on; }

I have “client intended to send too large body” errors too :
bba58f12-0ca4-4fe7-9d49-ad0ef994b37d-image.png

I think I solved this problem in the following way :

nano /etc/nginx/nginx.conf http { ## # Basic Settings ## client_max_body_size 10M;

–> I hope this will be enough for Nginx.

But now, I have a problem of CPU performance (sometimes, not ) which causes nodebb 503 errors (not nginx) or latencies because nodeJS and mongodb takes a lot of CPU.

1ced94bf-f56c-4d2a-a701-8f39c8d2868d-image.png

– Would there be tweaks for nodejs, nodebb or mongodb to prevent the influx of users from draining the CPU - RAM ?

– If you have better values for nginx error (see above the topic) or better fix, tell me 🙂

Thanks in advance @phenomlab

Hello @phenomlab

I have a problem with Virtualmin and mySQL.

We have a server bug due to lack of storage space.

We cleaned up our files/backup and overnight the free space was drastically reduced.

By doing research it seems that mysql takes a huge space in a short time.

34c9b40f-7785-44da-a1b5-dea8b32fc236-image.png

It seems that the following tables cause problems

undo_001 undo_002 and mysql.ibd file

881273dc-04d7-46dd-8b2c-2b98fe6212b7-image.png

I noticed this error on the log too:

ae838c9f-d8d0-4beb-a1fe-08e01f18358f-image.png

I also noticed that mysql also eats all CPU resources.

What can we do to improve this situation?
What advice would you give?

I see this thread on virtualmin forum :
https://forum.virtualmin.com/t/mysql-files-occupied-whole-disk-which-file-to-delete-to-recover/116374/4

Do you think this is a solution?

Thanks in advance

You know how each time you have a new project online, the default is to go get a GAnalytics account, install the JS, and look at it every now and then?

If you are using cloudflare as a Global Content Delivery Network (CDN), it now seems to have analytics built in.

Not as advanced as Google’s but more than enough for most projects. It also hosts static sites and it’s really fast.

Cloudflare has its detractors, so do your research. But this could simplify one step in your next project.

I’ve noticed my Nord VPN has struggled to connect lately on the laptop, it does connect but sometimes it can be a few minutes before it connects or it will just continue to say connecting and then I have to shut the app down and then it may do the same thing again once it’s reopened.

Could this be antivirus software that is interfering with it? I wonder if anyone else has had similar issues to what I’m facing?

Would it be safe to turn off Bitdefender whilst the VPN is running?

look at this 😍🥰 mongo DB, redis, websockets support straight out of the box?

1288557c-0075-4c9a-b819-046ff9f77f1a-image.png

This is really really awsome 🔥🔥🔥

4c076895-f4bd-4ccd-9790-65d9fef969c7-image.png

Do i need more RAM? i could not understand what is the idle memory in this chart - is it just few MB?

currently, i am using 2cpu /4gb RAM

One thing I noticed was that after recovering the assets and database onto a test server: (see below)

https://sudonix.com/topic/239/testing-out-webdock-io

The performance of the replica NodeBB install was painfully slow. After much searching and hair pulling, I found that the indexes seemed to be missing - a point which is referenced by NodeBB themselves here

https://community.nodebb.org/topic/11299/slow-page-loading-on-digitalocean/12?_=1646060175987

When I checked /api/recent it was indeed slow - like a snail. So, at their suggestion, I opened up mongodb and added the missing indexes

db.objects.createIndex({ _key: 1, score: -1 }, { background: true }); db.objects.createIndex({ _key: 1, value: -1 }, { background: true, unique: true, sparse: true }); db.objects.createIndex({ expireAt: 1 }, { expireAfterSeconds: 0, background: true });

The difference is quite astonishing. Now it runs like the Ferrari I’ve come to love and respect ♥

Might be useful for someone else in the same boat…

Hello 🙂

I find that since version 1.19.3 of NodeBB, the refresh of the pages is much longer than usual.

I did a test with the same config on my VM and in 1.19.2, the refresh was much faster.
idem to connect to ACP.

Am I the only one who noticed this?

unable to upvote

db3b191c-1cc0-45af-8bdd-fa2f30dbf87e-image.png

what is the reason for choosing node BB over flarum?

is SEO of flarum not so great?

edit: i am reading this now 🙂

https://discuss.flarum.org/d/26838-flarum-is-slow-in-comparison-to-nodebb