Skip to content

NodeBB Global Tchat : Forum with many users performance issues

Solved Configure
  • @phenomlab said in NodeBB Global Tchat : Forum with many users performance issues:

    I’d say that this was still relevant, and would solve the short term issue (in most cases) but if anything, you’ll still experience slowness if you have say 600 users online.
    The right way to handle this in my view would be to pass the notification stream to a message queuing service (like RabbitMQ or Redis) and let that process in the back end rather than cause locking on the forum itself. I’m surprised that this is in fact the case as standard emails are being queued and aren’t sent in real time.
    I personally don’t use this plugin as you know, so I’d need to review the code first in order to understand it’s structure.

    Thank you for your analyse 👍

    I would have liked to try the thing all the same, just to see the result

    600 users is better than 4000 😉

    Here is the content of the /src/messsagin/notification.js file

    I put it here because I don’t see where to integrate the given code for test :

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    };
    

    @phenomlab

    An idea for where to put this code before I open a thread an NodeBB communauty ?

  • @phenomlab

    An idea for where to put this code before I open a thread an NodeBB communauty ?

    @DownPW not specifically, no, as there is an existing function with the same name. The comma at the end of the revised function would indicate part of an existing array but I’m not entirely sure of where it should be placed - or if it should override the existing function altogether (which I don’t think is the case).

  • @DownPW not specifically, no, as there is an existing function with the same name. The comma at the end of the revised function would indicate part of an existing array but I’m not entirely sure of where it should be placed - or if it should override the existing function altogether (which I don’t think is the case).

    @phenomlab arf I hope i have an answers in nodeBB 😞

    But it’s an async function maybe here :

    345a9b2c-26cf-491d-8ffe-047afa529d61-image.png

  • @phenomlab arf I hope i have an answers in nodeBB 😞

    But it’s an async function maybe here :

    345a9b2c-26cf-491d-8ffe-047afa529d61-image.png

    @DownPW you can always experiment 👍

  • @DownPW you can always experiment 👍

    @phenomlab no luck 😞

  • @DownPW what have you tried?

  • @phenomlab lot of things 😉

  • @DownPW can you provide some brief examples?

  • @DownPW can you provide some brief examples?

    @phenomlab .

    'use strict';
    const winston = require('winston');
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    module.exports = function (Messaging) {
    Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    uids = await user.blocks.filterUids(fromUid, uids);
    let data = {
    roomId: roomId,
    fromUid: fromUid,
    message: messageObj,
    uids: uids,
    };
    data = await plugins.hooks.fire('filter:messaging.notify', data);
    if (!data || !data.uids || !data.uids.length) {
    return;
    }
    uids = data.uids;
    uids.forEach((uid) => {
    data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    Messaging.pushUnreadCount(uid);
    sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    });
    if (messageObj.system) {
    return;
    }
    // Delayed notifications
    let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    if (queueObj) {
    queueObj.message.content += `\n${messageObj.content}`;
    clearTimeout(queueObj.timeout);
    } else {
    queueObj = {
    message: messageObj,
    };
    Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    }
    queueObj.timeout = setTimeout(async () => {
    try {
    await sendNotifications(fromUid, uids, roomId, queueObj.message);
    } catch (err) {
    winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    }
    }, meta.config.notificationSendDelay * 1000);
    if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
    Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
    } else {
    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
    }
    };
    async function sendNotifications(fromuid, uids, roomId, messageObj) {
    const isOnline = await user.isOnline(uids);
    uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    if (!uids.length) {
    return;
    }
    const { displayname } = messageObj.fromUser;
    const isGroupChat = await Messaging.isGroupChat(roomId);
    const notification = await notifications.create({
    type: isGroupChat ? 'new-group-chat' : 'new-chat',
    subject: `[[email:notif.chat.subject, ${displayname}]]`,
    bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    bodyLong: messageObj.content,
    nid: `chat_${fromuid}_${roomId}`,
    from: fromuid,
    path: `/chats/${messageObj.roomId}`,
    });
    delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    notifications.push(notification, uids);
    }
    };

    nodebb build is ok with this code but I don’t see any diiference of latencies

  • @phenomlab .

    'use strict';
    
    const winston = require('winston');
    
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    
    module.exports = function (Messaging) {
    	Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    	
    	Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    		let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    		uids = await user.blocks.filterUids(fromUid, uids);
    
    		let data = {
    			roomId: roomId,
    			fromUid: fromUid,
    			message: messageObj,
    			uids: uids,
    		};
    		data = await plugins.hooks.fire('filter:messaging.notify', data);
    		if (!data || !data.uids || !data.uids.length) {
    			return;
    		}
    
    		uids = data.uids;
    		uids.forEach((uid) => {
    			data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    			Messaging.pushUnreadCount(uid);
    			sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    		});
    		if (messageObj.system) {
    			return;
    		}
    		// Delayed notifications
    		let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    		if (queueObj) {
    			queueObj.message.content += `\n${messageObj.content}`;
    			clearTimeout(queueObj.timeout);
    		} else {
    			queueObj = {
    				message: messageObj,
    			};
    			Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    		}
    
    		queueObj.timeout = setTimeout(async () => {
    			try {
    				await sendNotifications(fromUid, uids, roomId, queueObj.message);
    			} catch (err) {
    				winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    			}
    		}, meta.config.notificationSendDelay * 1000);
    		
    		
               if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
                   Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
                } else {
                    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
                }
    	};
    
    	async function sendNotifications(fromuid, uids, roomId, messageObj) {
    		const isOnline = await user.isOnline(uids);
    		uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    		if (!uids.length) {
    			return;
    		}
    
    
    		const { displayname } = messageObj.fromUser;
    
    		const isGroupChat = await Messaging.isGroupChat(roomId);
    		const notification = await notifications.create({
    			type: isGroupChat ? 'new-group-chat' : 'new-chat',
    			subject: `[[email:notif.chat.subject, ${displayname}]]`,
    			bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    			bodyLong: messageObj.content,
    			nid: `chat_${fromuid}_${roomId}`,
    			from: fromuid,
    			path: `/chats/${messageObj.roomId}`,
    		});
    
    		delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    		notifications.push(notification, uids);
    	}
    	
    
    		
    };
    

    nodebb build is ok with this code but I don’t see any diiference of latencies

    'use strict';
    const winston = require('winston');
    const user = require('../user');
    const notifications = require('../notifications');
    const sockets = require('../socket.io');
    const plugins = require('../plugins');
    const meta = require('../meta');
    module.exports = function (Messaging) {
    Messaging.notifyQueue = {}; // Only used to notify a user of a new chat message, see Messaging.notifyUser
    Messaging.notifyUsersInRoom = async (fromUid, roomId, messageObj) => {
    let uids = await Messaging.getUidsInRoom(roomId, 0, -1);
    uids = await user.blocks.filterUids(fromUid, uids);
    let data = {
    roomId: roomId,
    fromUid: fromUid,
    message: messageObj,
    uids: uids,
    };
    data = await plugins.hooks.fire('filter:messaging.notify', data);
    if (!data || !data.uids || !data.uids.length) {
    return;
    }
    uids = data.uids;
    uids.forEach((uid) => {
    data.self = parseInt(uid, 10) === parseInt(fromUid, 10) ? 1 : 0;
    Messaging.pushUnreadCount(uid);
    sockets.in(`uid_${uid}`).emit('event:chats.receive', data);
    });
    if (messageObj.system) {
    return;
    }
    // Delayed notifications
    let queueObj = Messaging.notifyQueue[`${fromUid}:${roomId}`];
    if (queueObj) {
    queueObj.message.content += `\n${messageObj.content}`;
    clearTimeout(queueObj.timeout);
    } else {
    queueObj = {
    message: messageObj,
    };
    Messaging.notifyQueue[`${fromUid}:${roomId}`] = queueObj;
    }
    queueObj.timeout = setTimeout(async () => {
    try {
    await sendNotifications(fromUid, uids, roomId, queueObj.message);
    } catch (err) {
    winston.error(`[messaging/notifications] Unabled to send notification\n${err.stack}`);
    }
    }, meta.config.notificationSendDelay * 1000);
    };
    async function sendNotifications(fromuid, uids, roomId, messageObj) {
    const isOnline = await user.isOnline(uids);
    uids = uids.filter((uid, index) => !isOnline[index] && parseInt(fromuid, 10) !== parseInt(uid, 10));
    if (!uids.length) {
    return;
    }
    if (roomId != 11) { // 5 Is the ID of the ID of the global chat room.
    Messaging.getUidsInRoom(roomId, 0, -1); // Proceed as normal.
    } else {
    user.getUidsFromSet('users:online', 0, -1); // Only notify online users.
    }
    const { displayname } = messageObj.fromUser;
    const isGroupChat = await Messaging.isGroupChat(roomId);
    const notification = await notifications.create({
    type: isGroupChat ? 'new-group-chat' : 'new-chat',
    subject: `[[email:notif.chat.subject, ${displayname}]]`,
    bodyShort: `[[notifications:new_message_from, ${displayname}]]`,
    bodyLong: messageObj.content,
    nid: `chat_${fromuid}_${roomId}`,
    from: fromuid,
    path: `/chats/${messageObj.roomId}`,
    });
    delete Messaging.notifyQueue[`${fromuid}:${roomId}`];
    notifications.push(notification, uids);
    }
    };
  • undefined DownPW has marked this topic as solved on 27 Apr 2023, 21:22


12/13

8 Jan 2023, 16:36


Did this solution help you?
Did you find the suggested solution useful? Why not buy me a coffee? It's a nice gesture, and a great way to show your appreciation 💗

Related Topics
  • 3 Votes
    8 Posts
    514 Views
    @Panda You should be able to expose the CSS for these using F12 to get into console [image: 1686833368846-3591518c-e3a3-4ada-a43c-6b32a5e0359c-image.png] [image: 1686833399483-a2b8ed46-4157-4ff2-85f0-576543380107-image.png] That should then expose the element once selected [image: 1686833431032-89d9c545-a47a-40d1-98f4-80cf3b958e8f-image.png] Here’s the below CSS you need based on the screenshot provided. .composer .formatting-bar .formatting-group li[data-format="picture-o"], .composer .formatting-bar .formatting-group li[data-format="spoiler"] { display: none; }
  • NodeBB: hCaptcha

    Solved Configure 11 Apr 2023, 12:21
    2 Votes
    15 Posts
    660 Views
    @mventures none that I know of. I don’t recall selecting these either for mine.
  • 0 Votes
    6 Posts
    426 Views
    @mventures You’d need to connect to the server and execute it directly - not on your local terminal. Review the guide below, which will show you how to gain access via SSH to your server https://docs.ovh.com/gb/en/dedicated/ssh-introduction/ Once you have access, you’ll need to navigate to the actual folder where NodeBB is installed You’ll then need to change to the directory as shown below /home/unbuntu/nodebb [image: 1680448167972-fdffe673-bf63-4b6d-a728-5506fddc1aff-image.png] In most cases, initial access takes you to the root of the file system. You can always issue pwd in a Linux terminal which will show you the Present Working Directory. From there, you can issue the command cd /home/ubuntu/nodebb Once in the NodeBB directory, you’d use the below commands ./nodebb stop git fetch && git checkout develop && git reset --hard origin/develop ./nodebb upgrade ./nodebb start Line 1 stops the NodeBB instance Line 2 gets the latest files from GIT (repository) and then checks out the development branch. It then resets the version you are using to the development branch ready for v3 Line 3 Runs the upgrade once the new branch is set, and code pulled Line 4 Restarts the NodeBB instance after the upgrade has completed Note that when you restart NodeBB and log back in, things will look very different to what you had in v2.
  • 0 Votes
    9 Posts
    609 Views
    OK, I think I have figured out how to place a link in the footer which will click to a new page.
  • restarting nodebb on boot

    Unsolved Configure 18 Dec 2022, 19:48
    1 Votes
    3 Posts
    383 Views
    @eeeee said in restarting nodebb on boot: can I just run nodebb under nodemon for auto restarts? It’s a better method. Nodemon just looks for file system changes and would effectively die if the server was rebooted meaning you’d have to start it again anyway. Systemd is the defacto standard which is how the operating system interacts in terms of services, scheduled tasks etc.
  • 3 Votes
    39 Posts
    2k Views
    @DownPW said in Virtualmin SQL problem: Finally problem solved simply but I still don’t understand why this service was installed. Glad to hear (and see) that this issue is now resolved. Virtualmin and Webmin are both derived from the origin of LAMP (Linux, Apache, MySQL, and PHP) - the earliest form would have been a project which was a fork of the original concept called “WAMP” (Windows, Apache, MySQL, and PHP) https://www.wampserver.com/en/ Scroll to the bottom, and you’ll see the packages it comes with [image: 1668368892094-cf3c0965-a699-4c6f-b89f-65e7bb381bbc-image.png] Over time, activity on this project dropped somewhat due to the rise of Virtualmin and Webmin - acting as “full blown” platforms designed to manage an entire web server from start to finish, and providing an easy way to do so with a GUI interface. Over time, the LAMP bundle became LEMP (Linux, NGINX [actually pronounced “engine X”], MySQL, and PHP). The issue with Apache is that it had limits, and compared to NGINX, was in fact much slower. Virtualmin and Webmin do not actually use MySQL for their core operation - they don’t even use Apache or NGINX for the core web services, which is why the admin port is set to a default of 10000 when you first complete the setup. Essentially, you can think of Virtualmin and Webmin as a central pane of glass to manage the underlying components that are required to run a website. MySQL doesn’t need to be installed if you are not actually using it, but is there as PHP typically is paired with either MySQL or MariaDB, so it makes sense to offer the installation of this itself, as well as support for managing it.
  • Configure SMTP for Nodebb

    Solved Configure 14 Apr 2022, 14:07
    5 Votes
    14 Posts
    1k Views
    @marusaky based on the work completed thus far (in relation to PM exchanges), I’m going to mark this completed. Sending email from the server itself works fine without issue, and DNS appears to be clean (valid SPF, DMARC, and DKIM records). It appears that only Gmail marks incoming messages from your domain as spam - perhaps because of the domain age, which there is nothing we can do to prevent this. Mail delivery to all other domains appears to work fine in al of my tests.
  • Forum not loading

    Solved Configure 6 Dec 2021, 14:26
    0 Votes
    27 Posts
    2k Views
    @phenomlab Brilliant!! Thanks ever so much!! Now I need to try pull in new members