Skip to content

NodeBB Global Tchat : Forum with many users performance issues

Solved Configure
  • @phenomlab said in NodeBB Global Tchat : Forum with many users performance issues:

    I’d say that this was still relevant, and would solve the short term issue (in most cases) but if anything, you’ll still experience slowness if you have say 600 users online.
    The right way to handle this in my view would be to pass the notification stream to a message queuing service (like RabbitMQ or Redis) and let that process in the back end rather than cause locking on the forum itself. I’m surprised that this is in fact the case as standard emails are being queued and aren’t sent in real time.
    I personally don’t use this plugin as you know, so I’d need to review the code first in order to understand it’s structure.

    Thank you for your analyse 👍

    I would have liked to try the thing all the same, just to see the result

    600 users is better than 4000 😉

    Here is the content of the /src/messsagin/notification.js file

    I put it here because I don’t see where to integrate the given code for test :

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    };
    

    @phenomlab

    An idea for where to put this code before I open a thread an NodeBB communauty ?

  • @phenomlab

    An idea for where to put this code before I open a thread an NodeBB communauty ?

    @DownPW not specifically, no, as there is an existing function with the same name. The comma at the end of the revised function would indicate part of an existing array but I’m not entirely sure of where it should be placed - or if it should override the existing function altogether (which I don’t think is the case).

  • @DownPW not specifically, no, as there is an existing function with the same name. The comma at the end of the revised function would indicate part of an existing array but I’m not entirely sure of where it should be placed - or if it should override the existing function altogether (which I don’t think is the case).

    @phenomlab arf I hope i have an answers in nodeBB 😞

    But it’s an async function maybe here :

    345a9b2c-26cf-491d-8ffe-047afa529d61-image.png

  • @phenomlab arf I hope i have an answers in nodeBB 😞

    But it’s an async function maybe here :

    345a9b2c-26cf-491d-8ffe-047afa529d61-image.png

    @DownPW you can always experiment 👍

  • @DownPW you can always experiment 👍

    @phenomlab no luck 😞

  • @DownPW what have you tried?

  • @phenomlab lot of things 😉

  • @DownPW can you provide some brief examples?

  • @DownPW can you provide some brief examples?

    @phenomlab .

    'use strict';
    const winston = require('winston');
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    module.exports = function (Messaging) {
    Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    uids = await user.blocks.filterUids(fromUid, uids);
    let data = {
    roomId: roomId,
    fromUid: fromUid,
    message: messageObj,
    uids: uids,
    };
    data = await plugins.hooks.fire('filter:messaging.notify', data);
    if (!data || !data.uids || !data.uids.length) {
    return;
    }
    uids = data.uids;
    uids.forEach((uid) => {
    data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    Messaging.pushUnreadCount(uid);
    sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    });
    if (messageObj.system) {
    return;
    }
    // Delayed notifications
    let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    if (queueObj) {
    queueObj.message.content += `\n${messageObj.content}`;
    clearTimeout(queueObj.timeout);
    } else {
    queueObj = {
    message: messageObj,
    };
    Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    }
    queueObj.timeout = setTimeout(async () => {
    try {
    await sendNotifications(fromUid, uids, roomId, queueObj.message);
    } catch (err) {
    winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    }
    }, meta.config.notificationSendDelay * 1000);
    if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
    Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
    } else {
    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
    }
    };
    async function sendNotifications(fromuid, uids, roomId, messageObj) {
    const isOnline = await user.isOnline(uids);
    uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    if (!uids.length) {
    return;
    }
    const { displayname } = messageObj.fromUser;
    const isGroupChat = await Messaging.isGroupChat(roomId);
    const notification = await notifications.create({
    type: isGroupChat ? 'new-group-chat' : 'new-chat',
    subject: `[[email:notif.chat.subject, ${displayname}]]`,
    bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    bodyLong: messageObj.content,
    nid: `chat_${fromuid}_${roomId}`,
    from: fromuid,
    path: `/chats/${messageObj.roomId}`,
    });
    delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    notifications.push(notification, uids);
    }
    };

    nodebb build is ok with this code but I don’t see any diiference of latencies

  • @phenomlab .

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    	
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    		
    		
               if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
                   Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
                } else {
                    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
                }
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    	
    
    		
    };
    

    nodebb build is ok with this code but I don’t see any diiference of latencies

    'use strict';
    const winston = require('winston');
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    module.exports = function (Messaging) {
    Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    uids = await user.blocks.filterUids(fromUid, uids);
    let data = {
    roomId: roomId,
    fromUid: fromUid,
    message: messageObj,
    uids: uids,
    };
    data = await plugins.hooks.fire('filter:messaging.notify', data);
    if (!data || !data.uids || !data.uids.length) {
    return;
    }
    uids = data.uids;
    uids.forEach((uid) => {
    data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    Messaging.pushUnreadCount(uid);
    sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    });
    if (messageObj.system) {
    return;
    }
    // Delayed notifications
    let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    if (queueObj) {
    queueObj.message.content += `\n${messageObj.content}`;
    clearTimeout(queueObj.timeout);
    } else {
    queueObj = {
    message: messageObj,
    };
    Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    }
    queueObj.timeout = setTimeout(async () => {
    try {
    await sendNotifications(fromUid, uids, roomId, queueObj.message);
    } catch (err) {
    winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    }
    }, meta.config.notificationSendDelay * 1000);
    };
    async function sendNotifications(fromuid, uids, roomId, messageObj) {
    const isOnline = await user.isOnline(uids);
    uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    if (!uids.length) {
    return;
    }
    if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
    Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
    } else {
    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
    }
    const { displayname } = messageObj.fromUser;
    const isGroupChat = await Messaging.isGroupChat(roomId);
    const notification = await notifications.create({
    type: isGroupChat ? 'new-group-chat' : 'new-chat',
    subject: `[[email:notif.chat.subject, ${displayname}]]`,
    bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    bodyLong: messageObj.content,
    nid: `chat_${fromuid}_${roomId}`,
    from: fromuid,
    path: `/chats/${messageObj.roomId}`,
    });
    delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    notifications.push(notification, uids);
    }
    };
  • undefined DownPW has marked this topic as solved on 27 Apr 2023, 21:22


12/13

8 Jan 2023, 16:36


Did this solution help you?
Did you find the suggested solution useful? Why not buy me a coffee? It's a nice gesture, and a great way to show your appreciation 💗

Related Topics
  • 4 Votes
    25 Posts
    2k Views
    @Panda said in Fixed background to nodebb forum: Chatgpt told me the ::before method. Go figure
  • NodeBB: hCaptcha

    Solved Configure 11 Apr 2023, 12:21
    2 Votes
    15 Posts
    553 Views
    @mventures none that I know of. I don’t recall selecting these either for mine.
  • NodeBB: Consent page

    Solved Configure 6 Apr 2023, 13:02
    4 Votes
    16 Posts
    924 Views
    @DownPW I still do not see any issues.
  • 0 Votes
    6 Posts
    333 Views
    @mventures You’d need to connect to the server and execute it directly - not on your local terminal. Review the guide below, which will show you how to gain access via SSH to your server https://docs.ovh.com/gb/en/dedicated/ssh-introduction/ Once you have access, you’ll need to navigate to the actual folder where NodeBB is installed You’ll then need to change to the directory as shown below /home/unbuntu/nodebb [image: 1680448167972-fdffe673-bf63-4b6d-a728-5506fddc1aff-image.png] In most cases, initial access takes you to the root of the file system. You can always issue pwd in a Linux terminal which will show you the Present Working Directory. From there, you can issue the command cd /home/ubuntu/nodebb Once in the NodeBB directory, you’d use the below commands ./nodebb stop git fetch && git checkout develop && git reset --hard origin/develop ./nodebb upgrade ./nodebb start Line 1 stops the NodeBB instance Line 2 gets the latest files from GIT (repository) and then checks out the development branch. It then resets the version you are using to the development branch ready for v3 Line 3 Runs the upgrade once the new branch is set, and code pulled Line 4 Restarts the NodeBB instance after the upgrade has completed Note that when you restart NodeBB and log back in, things will look very different to what you had in v2.
  • 1 Votes
    16 Posts
    701 Views
    @DownPW sure. Let me have a look at this in more detail. I know nginx plus has extensive support for this, but it’s not impossible to get somewhere near acceptable with the standard version. You might be better off handling this at the Cloudflare level given that it sits in between the requesting client and your server.
  • 3 Votes
    39 Posts
    2k Views
    @DownPW said in Virtualmin SQL problem: Finally problem solved simply but I still don’t understand why this service was installed. Glad to hear (and see) that this issue is now resolved. Virtualmin and Webmin are both derived from the origin of LAMP (Linux, Apache, MySQL, and PHP) - the earliest form would have been a project which was a fork of the original concept called “WAMP” (Windows, Apache, MySQL, and PHP) https://www.wampserver.com/en/ Scroll to the bottom, and you’ll see the packages it comes with [image: 1668368892094-cf3c0965-a699-4c6f-b89f-65e7bb381bbc-image.png] Over time, activity on this project dropped somewhat due to the rise of Virtualmin and Webmin - acting as “full blown” platforms designed to manage an entire web server from start to finish, and providing an easy way to do so with a GUI interface. Over time, the LAMP bundle became LEMP (Linux, NGINX [actually pronounced “engine X”], MySQL, and PHP). The issue with Apache is that it had limits, and compared to NGINX, was in fact much slower. Virtualmin and Webmin do not actually use MySQL for their core operation - they don’t even use Apache or NGINX for the core web services, which is why the admin port is set to a default of 10000 when you first complete the setup. Essentially, you can think of Virtualmin and Webmin as a central pane of glass to manage the underlying components that are required to run a website. MySQL doesn’t need to be installed if you are not actually using it, but is there as PHP typically is paired with either MySQL or MariaDB, so it makes sense to offer the installation of this itself, as well as support for managing it.
  • Configure SMTP for Nodebb

    Solved Configure 14 Apr 2022, 14:07
    5 Votes
    14 Posts
    1k Views
    @marusaky based on the work completed thus far (in relation to PM exchanges), I’m going to mark this completed. Sending email from the server itself works fine without issue, and DNS appears to be clean (valid SPF, DMARC, and DKIM records). It appears that only Gmail marks incoming messages from your domain as spam - perhaps because of the domain age, which there is nothing we can do to prevent this. Mail delivery to all other domains appears to work fine in al of my tests.
  • nodebb dropdown menu

    Solved Configure 26 Nov 2021, 15:17
    1
    0 Votes
    5 Posts
    549 Views
    @phenomlab said in nodebb dropdown menu: @kurulumu-net You set it like the below example taken from this site [image: 1637939951821-aae36790-3257-4bb2-ad5a-0d744309876a-image.png] Which presents this [image: 1637939983445-77f47260-2941-4afe-9614-8e17dcfc8c19-image.png] Very interesting… I actually thought this wasn’t possible, as I remember it being asked in the NodeBB forum. Is this something new that’s been implemented? I’ll 100% be doing that when I’m on the laptop over the weekend.