OGProxy - a replacement for iFramely
-
Hi all,
Iโve been working on a self-coded replacement for iFramely, and I now have a workable solution that
- Is able to bypass CORS restrictions
- Gets data as the client, so zero server load
- Gets as close to iFramely hosted in terms of look and feel
- Uses a cors-proxy with cache, so any data held in cache is used before the browser makes any requests
- Super fast
- Uses 100% proprietary code written by Phenomlab
- Most importantly, itโs completely FREE.
Itโs not perfect, and has some bugs (youโll no doubt see these pop up from time to time) - but itโs working at around 97%, and active on this forum (yes, itโs running in PROD
)
Hereโs the CORS proxy running with
node-cache
Hereโs what the unfurled links look like
Itโll inherit the CSS of your site (potentially - you might have to make some minor changes).
You can see it in action by just visiting posts that have an external URL in them. A good example is this link
https://sudonix.org/topic/138/ai-a-new-dawn-or-the-demise-of-humanity
I know most members here want this, so decided to write my own
Itโs not quite โreadyโ yet, but will be soon.
Any takers?
-
OMG Mark
Very cool result. I take it for sure at 100% ;l
If you want to test it on another Platforms or help for anythings, lโm hรจreโฆ -
OMG Mark
Very cool result. I take it for sure at 100% ;l
If you want to test it on another Platforms or help for anythings, lโm hรจreโฆ@DownPW I can provide advance code for testing, but itโs subject to some refactoring. Up to you, but first come, first served, and I only want a limited amount of testers for obvious reasons whilst itโs in Alpha.
-
No problem
-
Mark i m always here as you know
-
Something of an updateโฆ
Iโve been looking at the overall design for the
CORS
proxy again today - perhaps with a fresher perspective (and less tired eyes) and found several ways to improve the overall efficiency of the proxy itself.I also changed the code base so it will now require the generation of an API key (donโt worry too much about this as there are tons of generators online that can easily convert a phrase to either a
sha256
(preferred) ormd5
hash. You simply copy this key and place it in both the proxy server and in the customjs
code in NodeBB, restart the proxy service, and youโre good to go.I suspect youโre wanting to know how the proxy part worksโฆ This is a relatively simple
nodeJS
based server that runs as a daemon (usingsystemd
) which takes input from thejs
code and processes it as if it were aGET
request from a browser - in short, it collectsOpenGraph (og)
data from the target URL and then returns that data back to thejs
function which then creates adiv
block based on the bootstrapcard
element containing the information about the website - typicallytitle
,description
,image
andurl
which is the used to render the preview card.Iโve switched development of the newer proxy to dev so it doesnโt impact production. As soon as this is ready, the current version running on this site will be replaced.
Once final testing is completed, I will be in a position to provide an installation guide so you can try this out for yourselves.
For info, the proxy will not render preview links pointing to your own domain by design. Iโd recommend you leave it that way otherwise youโll encounter issues with performance and odd artefacts that arenโt rendered properly because the lookup failed. Iโll be placing a check inside the code that ignores links that have the origin of your own domain.
Finally, Iโd recommend setting up a subdomain for the proxy service to run under. Itโs cleaner, easier to manage, and will run independent of the root domain where your NodeBB instance runs. A simple
nginx
reverse proxy is required, and I will provide this as part of the guide. -
@julian has just released
nodebb-plugin-link-preview
which might be a better fit depending on your needs. I will continue development for this solution though as it performs all requests client-side. -
@julian has just released
nodebb-plugin-link-preview
which might be a better fit depending on your needs. I will continue development for this solution though as it performs all requests client-side.I have installed the first version published
After a few tests, I have quite random results but the solution has the advantage of being easy to use -
I have installed the first version published
After a few tests, I have quite random results but the solution has the advantage of being easy to use@DownPW Yes, thatโs true, but I donโt think itโs as efficient. Iโve looked at the library being used in more detail last night, and whilst I like the approach, it has some nuances where I think performance could be an issue.
-
Another quick update. Julianโs image preview plugin has gone though numerous iterations and now appears to work very well.
However, there are some limitations from what I can see, given that the NodeBB plugin works server side. The implementation I am working on runs client side, and is extremely quick.
Iโve chosen to use an existing library as this made more sense than effectively reinventing the wheel. This one is updated frequently (in fact yesterday was the last update) so less of a concern from the longevity perspective.
I expect to have a fully functional beta available in the coming days. If you want to see progress around the latest code, this is available and running on https://sudonix.dev.
-
Another quick update. Julianโs image preview plugin has gone though numerous iterations and now appears to work very well.
However, there are some limitations from what I can see, given that the NodeBB plugin works server side. The implementation I am working on runs client side, and is extremely quick.
Iโve chosen to use an existing library as this made more sense than effectively reinventing the wheel. This one is updated frequently (in fact yesterday was the last update) so less of a concern from the longevity perspective.
I expect to have a fully functional beta available in the coming days. If you want to see progress around the latest code, this is available and running on https://sudonix.dev.
@phenomlab said in Potential replacement for iFramely:
and now appears to work very well.
Are you sure
After One hour !!
-
your solution working fine. It should be as simple to implement as Julianโs plugin and it would be perfect ^^
-
@phenomlab said in Potential replacement for iFramely:
and now appears to work very well.
Are you sure
After One hour !!
@DownPW yes, mine is doing the same now.
-
your solution working fine. It should be as simple to implement as Julianโs plugin and it would be perfect ^^
@DownPW itโs not going to be a plugin as it requires itโs own reverse proxy in a sub domain as I mentioned previously and everything is client side so a plugin would be pointless as it has no settings - only custom JS code.
-
I have literally one bug (a simple one) to resolve, then clean up the code and restyle done of the elements (@DownPW is right in the sense that they are a bit big
) and itโs ready for testing.
In reality, I expect this to work almost flawlessly on other user forums as the code has been though several iterations to ensure itโs both lean and efficient. If you test it in sudonix.dev youโll see what I mean. I found an ingenuous way to extract the
favicon
- believe it or not, despite the year being 2023, web site owners donโt understand how to structuremeta
tags properly ๐คฌ๐คฌ.I learned today that Google in fact has a hidden API that you can use to get the
favicon
from any site which not only works flawlessly, but itโs extremely efficient and so, Iโm using it in my code - pointless reinventing the wheel when you donโt have to.The final โbugโ isnโt really a โbugโ in the traditional sense. In fact, its nothing to do with my code, but one specific variable I look for in the headers of each URL being crawled is
ogSiteName
The โogโ part is short for OpenGraph - an industry standard for years, yet often missing in headers. This has the undesired effect of returning
undefined
when it doesnโt exist, meaning reviews look like thisNotice the โundefinedโ text where the site title will be - that looks bad, to Iโll likely replace the with the domain name of the site of the title is missing.
Seriously, if you havenโt tested this out yet, if suggest to do so on sudonix.dev as soon as you can.
Currently, this site is running
nodebb-plugin-link-previews
but will be switching to my client side version in the coming days.As soon as itโs running here, Iโll release the code and a guide.
-
I have literally one bug (a simple one) to resolve, then clean up the code and restyle done of the elements (@DownPW is right in the sense that they are a bit big
) and itโs ready for testing.
In reality, I expect this to work almost flawlessly on other user forums as the code has been though several iterations to ensure itโs both lean and efficient. If you test it in sudonix.dev youโll see what I mean. I found an ingenuous way to extract the
favicon
- believe it or not, despite the year being 2023, web site owners donโt understand how to structuremeta
tags properly ๐คฌ๐คฌ.I learned today that Google in fact has a hidden API that you can use to get the
favicon
from any site which not only works flawlessly, but itโs extremely efficient and so, Iโm using it in my code - pointless reinventing the wheel when you donโt have to.The final โbugโ isnโt really a โbugโ in the traditional sense. In fact, its nothing to do with my code, but one specific variable I look for in the headers of each URL being crawled is
ogSiteName
The โogโ part is short for OpenGraph - an industry standard for years, yet often missing in headers. This has the undesired effect of returning
undefined
when it doesnโt exist, meaning reviews look like thisNotice the โundefinedโ text where the site title will be - that looks bad, to Iโll likely replace the with the domain name of the site of the title is missing.
Seriously, if you havenโt tested this out yet, if suggest to do so on sudonix.dev as soon as you can.
Currently, this site is running
nodebb-plugin-link-previews
but will be switching to my client side version in the coming days.As soon as itโs running here, Iโll release the code and a guide.
@phenomlab said in Potential replacement for iFramely:
believe it or not, despite the year being 2023, web site owners donโt understand how to structure meta tags properly ๐คฌ๐คฌ.
Ha ha I believe you !!
@phenomlab said in Potential replacement for iFramely:
Notice the โundefinedโ text where the site title will be - that looks bad, to Iโll likely replace the with the domain name of the site of the title is missing.
yep be clearly better with domain name when OpenGraphi when he is absent in headers
@phenomlab said in Potential replacement for iFramely:
Seriously, if you havenโt tested this out yet, if suggest to do so on sudonix.dev as soon as you can.
Already tested, very fast
@phenomlab said in Potential replacement for iFramely:
Currently, this site is running nodebb-plugin-link-previews but will be switching to my client side version in the coming days.
As soon as itโs running here, Iโll release the code and a guide.
Hell yeah
-
Ok, spent some time on this late last night, and the good news is that itโs finished, and ready for you to try out.
Please do remember that this isnโt a plugin, but a client side
js
function with a server side proxy. However, donโt be put off as the installation is simple, and you should be up and running in around 30 minutes maximum.It does require some technical knowledge and ability, but if youโve setup NodeBB, then you can do this easily - besides, there will be full documentation so you are taken through each step.
Some other points to note are that not every site returns valid Open Graph data - in fact, some donโt even return an image (yes, Reddit, Iโm taking about you) when they are closed to the public, or behind a registration form / membership grant, or in some cases, a paywall.
When this scenario is met, the problem arises that no valid image is being returned. I did toy with the idea of using a free random image API , and even wrote the code for it - then realised nature scenery didnโt quite align with a tech site like Reddit.
Ok - the only thing to do here is to generate your own image then, and bundle that in with the proxy. For this purpose, I chose an image (I have a subscription to stockphotosecrets.com, which is an annual cost to me. Iโve cancelled the subscription as I donโt use it, but provided I downloaded the image before the term expires, I have the right to use any images after the fact) and then added some text parts to it so that it could then be used as a placeholder for when the image is absent.
Hereโs that image
Itโs sparse, but functional. And given my comment earlier around membership and paywalls, hereโs what that specific scenario would look like when encountered
Before I go ahead and provide the documentation, code, and proxy server, Iโd recommend you try out the latest code on sudonix.dev.
Enjoy.
-
Mark could you please install to my site also?
-
@cagatay Let me put the guide together first, and see if you can get it working by yourself. Itโs not a difficult installation, and once you understand how the components intersect and work together, I think youโll be fine. If the worst comes to it however, Iโm always happy to help.
-
UPDATE: OGProxy is now live on this forum
https://sudonix.org/topic/498/setup-ogproxy-for-use-in-nodebb
Related Topics
-
-
-
-
-
-
Link Not Working
Solved Customisation -
-